How to open and convert sqlite database to pandas dataframe

问题

I have downloaded some datas as a sqlite database (data.db) and I want to open this database in python and then convert it into pandas dataframe.

This is so far I have done

import sqlite3
import pandas    
dat = sqlite3.connect('data.db') #connected to database with out error
pandas.DataFrame.from_records(dat, index=None, exclude=None, columns=None, coerce_float=False, nrows=None)

But its throwing this error

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 980, in from_records
    coerce_float=coerce_float)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 5353, in _to_arrays
    if not len(data):
TypeError: object of type 'sqlite3.Connection' has no len()

How to convert sqlite database to pandas dataframe

回答1:

Despite sqlite being part of the Python Standard Library and is a nice and easy interface to SQLite databases, the Pandas tutorial states:

Note In order to use read_sql_table(), you must have the SQLAlchemy optional dependency installed. http://pandas.pydata.org/pandas-docs/stable/io.html#reading-tables

But Pandas still supports sqlite3 access if you want to avoid installing SQLAlchemy:

import sqlite3
import pandas as pd
# Create your connection.
cnx = sqlite3.connect('file.db')

df = pd.read_sql_query("SELECT * FROM table_name", cnx)

But you need to know the name of the used table in advance.

Hope it helps!

http://pandas.pydata.org/pandas-docs/stable/io.html#sqlite-fallback

回答2:

The line

data = sqlite3.connect('data.db')

opens a connection to the database. There are no records queried up to this. So you have to execute a query afterward and provide this to the pandas DataFrame constructor.

It should look similar to this

import sqlite3
import pandas as pd

dat = sqlite3.connect('data.db')
query = dat.execute("SELECT * From <TABLENAME>")
cols = [column[0] for column in query.description]
results= pd.DataFrame.from_records(data = query.fetchall(), columns = cols)

I am not really firm with SQL commands, so you should check the correctness of the query. should be the name of the table in your database.

回答3:

Search sqlalchemy, engine and database name in google (sqlite in this case):

import pandas as pd
import sqlalchemy

db_name = "data.db"
table_name = "LITTLE_BOBBY_TABLES"

engine = sqlalchemy.create_engine("sqlite:///%s" % db_name, execution_options={"sqlite_raw_colnames": True})
df = pd.read_sql_table(table_name, engine)

回答4:

i have stored my data in database.sqlite table name is Reviews

import sqlite3
con=sqlite3.connect("database.sqlite")

data=pd.read_sql_query("SELECT * FROM Reviews",con)
print(data)

来源：https://stackoverflow.com/questions/36028759/how-to-open-and-convert-sqlite-database-to-pandas-dataframe

标签

python

database

sqlite

pandas

dataframe