Writing to MySQL database with pandas using SQLAlchemy, to_sql

问题:

trying to write pandas dataframe to MySQL table using to_sql. Previously been using flavor='mysql', however it will be depreciated in the future and wanted to start the transition to using SQLAlchemy engine.

sample code:

import pandas as pd import mysql.connector from sqlalchemy import create_engine  engine = create_engine('mysql+mysqlconnector://[user]:[pass]@[host]:[port]/[schema]', echo=False) cnx = engine.raw_connection() data = pd.read_sql('SELECT * FROM sample_table', cnx) data.to_sql(name='sample_table2', con=cnx, if_exists = 'append', index=False)

The read works fine but the to_sql has an error:

DatabaseError: Execution failed on sql 'SELECT name FROM sqlite_master WHERE type='table' AND name=?;': Wrong number of arguments during string formatting

Why does it look like it is trying to use sqlite? What is the correct use of a sqlalchemy connection with mysql and specifically mysql.connector?

I also tried passing the engine in as the connection as well, and that gave me an error referencing no cursor object.

data.to_sql(name='sample_table2', con=engine, if_exists = 'append', index=False) >>AttributeError: 'Engine' object has no attribute 'cursor'

回答1:

Using the engine in place of the raw_connection() worked:

import pandas as pd import mysql.connector from sqlalchemy import create_engine  engine = create_engine('mysql+mysqlconnector://[user]:[pass]@[host]:[port]/[schema]', echo=False) data.to_sql(name='sample_table2', con=engine, if_exists = 'append', index=False)

not clear on why when I tried this yesterday it gave me the earlier error

回答2:

Alternatively, use pymysql package...

import pymysql from sqlalchemy import create_engine cnx = create_engine('mysql+pymysql://[user]:[pass]@[host]:[port]/[schema]', echo=False)  data = pd.read_sql('SELECT * FROM sample_table', cnx) data.to_sql(name='sample_table2', con=cnx, if_exists = 'append', index=False)

回答3:

I know in the title of the question is included the word SQLAlchemy, however I see in the questions and answers the need to import pymysql or mysql.connector, and also is possible to do the job with pymysql, withouth calling SQLAlchemy.

import pymysql user = 'root' passw = 'my-secret-pw-for-mysql-12ud' # In previous posts variable "pass" host =  '172.17.0.2' port = 3306  database = 'sample_table' # In previous posts similar to "schema"  conn = pymysql.connect(host=host,                        port=port,                        user=user,                         passwd=passw,                          db=database)  data.to_sql(name=database, con=conn, if_exists = 'append', index=False, flavor = 'mysql')

I think this solution could be good althought it is not using SQLAlchemy.

回答4:

Using pymysql and sqlalchemy, this works for Pandas v0.22:

import pandas as pd import pymysql from sqlalchemy import create_engine  user = 'yourUserName' passw = 'password' host =  'hostName'  # either localhost or ip e.g. '172.17.0.2' or hostname address  port = 3306  database = 'dataBaseName'  mydb = create_engine('mysql+pymysql://' + user + ':' + passw + '@' + host + ':' + str(port) + '/' + database , echo=False)  directory = r'directoryLocation'  # path of csv file csvFileName = 'something.csv'  df = pd.read_csv(os.path.join(directory, csvFileName ))  df.to_sql(name=csvFileName[:-4], con=mydb, if_exists = 'replace', index=False)  """ if_exists: {'fail', 'replace', 'append'}, default 'fail'      fail: If table exists, do nothing.      replace: If table exists, drop it, recreate it, and insert data.      append: If table exists, insert data. Create if does not exist. """

转载请标明出处:Writing to MySQL database with pandas using SQLAlchemy, to_sql

文章来源: Writing to MySQL database with pandas using SQLAlchemy, to_sql