Using IBM_DB with Pandas

Question

I am trying to use the data analysis tool Pandas in Python Language. I am trying to read data from a IBM DB, using ibm_db package. According to the documentation in Pandas website we need to provide at least 2 arguments, one would be the sql that would be executed and other would be the connection object of the database. But when i do that, it gives me error that the connection object does not have a cursor() method in it. I figured maybe this is not how this particular DB Package worked. I tried to find a few workarounds but was not successfull.

Code:

print "hello PyDev"
con = db.connect("DATABASE=db;HOSTNAME=localhost;PORT=50000;PROTOCOL=TCPIP;UID=admin;PWD=admin;", "", "")
sql = "select * from Maximo.PLUSPCUSTOMER"
stmt = db.exec_immediate(con,sql)
pd.read_sql(sql, db)
print "done here"

Error:

hello PyDev
Traceback (most recent call last):
  File "C:\Users\ray\workspace\Firstproject\pack\test.py", line 15, in <module>
    pd.read_sql(sql, con)
  File "D:\etl\lib\site-packages\pandas\io\sql.py", line 478, in read_sql
    chunksize=chunksize)
  File "D:\etl\lib\site-packages\pandas\io\sql.py", line 1504, in read_query
    cursor = self.execute(*args)
  File "D:\etl\lib\site-packages\pandas\io\sql.py", line 1467, in execute
    cur = self.con.cursor()
AttributeError: 'ibm_db.IBM_DBConnection' object has no attribute 'cursor'

I am able to fetch data if i fetch it from the database but i need to read into a dataframe and need to write back to the database after processing data.

Code for fetching from DB

stmt = db.exec_immediate(con,sql)
 tpl=db.fetch_tuple(stmt)
 while tpl:
     print(tpl)
     tpl=db.fetch_tuple(stmt)

Answer 1

On doing further studying the package, i found that I need to wrap the IBM_DB connection object in a ibm_db_dbi connection object, which is part of the same package.

So

conn = ibm_db_dbi.Connection(con)
df = pd.read_sql(sql, conn)

The above code works and pandas fetches data into dataframe successfully.

Answer 2

you can also check out https://pypi.python.org/pypi/ibmdbpy

It provides Pandas style API without pulling out all data into Python memory.

Documentation is here: http://pythonhosted.org/ibmdbpy/index.html Here is a quick demo how to use it in Bluemix Notebooks: https://www.youtube.com/watch?v=tk9T1yPkn4c