问题
What is the best way to save a pandas dataframe in kdb? Are there any libraries that can make it easier?
The below code can apparently be used to loas something from kdb, but how do I save a dataframe into it?
from qpython import qconnection
with qconnection.QConnection(host = 'localhost', port = 5001, pandas = True) as q:
ds = q('(1i;0Ni;3i)', pandas = True)
print(ds)
回答1:
To save a dataframe in KDB+ with qPython one can use the sync method of the QConnection class. Set the first parameter to a string defining a q function that assigns its parameter to a global variable and send the dataframe as the second parameter. Something like this:
from qpython import qconnection
import pandas as pd
df = pd.DataFrame({'sym':['abc','def','ghi'],'price':[10.1,10.2,10.3]})
with qconnection.QConnection(host = 'localhost', port = 5001, pandas = True) as q:
q.sync('{t::x}',df)
Note that you need to use a double colon :: in the function definition so that the parameter is assigned to a global variable t rather that a local variable.
回答2:
Using a mock table from Pandas Integration
with qconnection.QConnection(host = 'localhost', port = 5000, pandas = True) as q:
df = q('flip `name`iq`fullname!(`Dent`Beeblebrox`Prefect;98 42 126;("Arthur Dent";"Zaphod Beeblebrox"; "Ford Prefect"))')
This pulls a table built in kdb back in python as a pandas dataframe. The following method saves the table down in memory:
with qconnection.QConnection(host = 'localhost', port = 5000, pandas = True) as q:
q('{`t set x}', df)
Some more information on how data is saved in kdb can be found: kdb overview
More examples of using set to save data can be found here: set
There is another method of integrating python and q that may be of use to you; PyQ brings the Python and q interpreters into the same process, so that code written in either of the languages operates on the same data.
回答3:
One option is to use embedPy which allows kdb+ and Python to share the same process and memory space
You can find documation on this library in the provide link http://code.kx.com/q/ml/embedpy/
See below an example implemented in kdb+
q)/ Create kdb+ table
q)n:100;show 5#qtab:([]scol:n?`AAA`BBB`CCC;icol:n?100;fcol:n?1f)
scol icol vcol
-------------------
CCC 59 0.3927524
AAA 30 0.5170911
CCC 85 0.5159796
AAA 89 0.4066642
AAA 23 0.1780839
q)
q)/ Load embedPy and import pandas/DataFrame
q)\l p.q
q)df:(pd:.p.import`pandas)`:DataFrame
q)
q)/ Initialize DataFrame object
q)/ print is built in embedpy
q)print df[qtab][`:head]5
fcol icol scol
0 0.392752 59 CCC
1 0.517091 30 AAA
2 0.515980 85 CCC
3 0.406664 89 AAA
4 0.178084 23 AAA
q)/ need to reorder the columns
q)print ptab:df[qtab][@;cols qtab]
scol icol fcol
0 CCC 59 0.392752
1 AAA 30 0.517091
2 CCC 85 0.515980
3 AAA 89 0.406664
4 AAA 23 0.178084
q)/ and converting back to kdb+
q)/ need to convert it to a dict like
q)5#flip ptab[`:to_dict;`list]`
scol icol fcol
--------------------
"CCC" 59 0.3927524
"AAA" 30 0.5170911
"CCC" 85 0.5159796
"AAA" 89 0.4066642
"AAA" 23 0.1780839
q)/ can also use but slower
q)5#ptab[`:to_dict;`records]`
scol icol fcol
--------------------
"CCC" 59 0.3927524
"AAA" 30 0.5170911
"CCC" 85 0.5159796
"AAA" 89 0.4066642
"AAA" 23 0.1780839
q)/ Consider keyed table
q)show qktab:select by scol from qtab
scol| icol fcol
----| ---------------
AAA | 35 0.3410485
BBB | 61 0.5548864
CCC | 0 0.07347808
q)/ print as dataframe format
q)/ reordering columns to be the same as qktab
q)print pktab:df[qktab][@;cols qktab]
scol icol fcol
0 AAA 35 0.341049
1 BBB 61 0.554886
2 CCC 0 0.073478
q) / convert it to a dataframe keyed table
q) / setting the index to become the keyed column
q)
q)print pktab:pktab[`:set_index]keys qktab
icol fcol
scol
AAA 35 0.341049
BBB 61 0.554886
CCC 0 0.073478
q) / converting back to kdb+
q) / need to `reset_index` to return full table
q) / then key the table
q)(`$pktab[`:index.names]`)xkey flip pktab[`:reset_index][][`:to_dict;`list]`
scol | icol fcol
-----| ---------------
"AAA"| 35 0.3410485
"BBB"| 61 0.5548864
"CCC"| 0 0.07347808
来源：https://stackoverflow.com/questions/49101675/save-pandas-dataframe-in-kdb-q