Inserting JSON into MySQL using Python

Question

I have a JSON object in Python. I am Using Python DB-API and SimpleJson. I am trying to insert the json into a MySQL table.

At moment am getting errors and I believ

Answer 1

You need to get a look at the actual SQL string, try something like this:

sqlstr = "INSERT INTO tweets_unprocessed (text, created_at, twitter_id, user_id, user_screen_name, json) VALUES (%s,%s,%s,%s,%s,%s)", (status.get('text'), status.get('created_at'), status.get('id'), status.get('user', {}).get('id'), status.get('user', {}).get('screen_name'), status)
print "about to execute(%s)" % sqlstr
twitdb.execute(sqlstr)

I imagine you are going to find some stray quotes, brackets or parenthesis in there.

Answer 2

The most straightforward way to insert a python map into a MySQL JSON field...

python_map = { "foo": "bar", [ "baz", "biz" ] }

sql = "INSERT INTO your_table (json_column_name) VALUES (%s)"
cursor.execute( sql, (json.dumps(python_map),) )

Answer 3

One example, how add a JSON file into MySQL using Python. This means that it is necessary to convert the JSON file to sql insert, if there are several JSON objects then it is better to have only one call INSERT than multiple calls, ie for each object to call the function INSERT INTO.

# import Python's JSON lib
import json

# use JSON loads to create a list of records
test_json = json.loads('''
[
    {
    "COL_ID": "id1",
    "COL_INT_VAULE": 7,
    "COL_BOOL_VALUE": true,
    "COL_FLOAT_VALUE": 3.14159,
    "COL_STRING_VAULE": "stackoverflow answer"
    },
    {
    "COL_ID": "id2",
    "COL_INT_VAULE": 10,
    "COL_BOOL_VALUE": false,
    "COL_FLOAT_VALUE": 2.71828,
    "COL_STRING_VAULE": "http://stackoverflow.com/"
    },
    {
    "COL_ID": "id3",
    "COL_INT_VAULE": 2020,
    "COL_BOOL_VALUE": true,
    "COL_FLOAT_VALUE": 1.41421,
    "COL_STRING_VAULE": "GIRL: Do you drink? PROGRAMMER: No. GIRL: Have Girlfriend? PROGRAMMER: No. GIRL: Then how do you enjoy life? PROGRAMMER: I am Programmer"
    }
]
''')

# create a nested list of the records' values
values = [list(x.values()) for x in test_json]
# print(values)

# get the column names
columns = [list(x.keys()) for x in test_json][0]

# value string for the SQL string
values_str = ""

# enumerate over the records' values
for i, record in enumerate(values):

    # declare empty list for values
    val_list = []
   
    # append each value to a new list of values
    for v, val in enumerate(record):
        if type(val) == str:
            val = "'{}'".format(val.replace("'", "''"))
        val_list += [ str(val) ]

    # put parenthesis around each record string
    values_str += "(" + ', '.join( val_list ) + "),\n"

# remove the last comma and end SQL with a semicolon
values_str = values_str[:-2] + ";"

# concatenate the SQL string
table_name = "json_data"
sql_string = "INSERT INTO %s (%s)\nVALUES\n%s" % (
    table_name,
    ', '.join(columns),
    values_str
)

print("\nSQL string:\n\n")
print(sql_string)

output:

SQL string:


INSERT INTO json_data (COL_ID, COL_INT_VAULE, COL_BOOL_VALUE, COL_FLOAT_VALUE, COL_STRING_VAULE)
VALUES
('id1', 7, True, 3.14159, 'stackoverflow answer'),
('id2', 10, False, 2.71828, 'http://stackoverflow.com/'),
('id3', 2020, True, 1.41421, 'GIRL: Do you drink? PROGRAMMER: No. GIRL: Have Girlfriend? PROGRAMMER: No. GIRL: Then how do you enjoy life? PROGRAMMER: I am Programmer.');