How to insert timeseries data into SQLite using SQLalchemy?

∥☆過路亽.° 提交于 2019-12-20 06:37:38

问题


I am developing an app which obtains financial data from the Federal Reserve, stores it in a SQlite database using Flask-SQlalchemy, and then serves the data to my front-end dashboard (Bokeh).

The data I am using is structured as follows:

             date    realtime_end  realtime_start  value
0     1962-01-02   2018-08-02     2018-08-02   4.06
1     1962-01-03   2018-08-02     2018-08-02   4.03
2     1962-01-04   2018-08-02     2018-08-02   3.99
3     1962-01-05   2018-08-02     2018-08-02   4.02
4     1962-01-08   2018-08-02     2018-08-02   4.03
5     1962-01-09   2018-08-02     2018-08-02   4.05
6     1962-01-10   2018-08-02     2018-08-02   4.07
7     1962-01-11   2018-08-02     2018-08-02   4.08
8     1962-01-12   2018-08-02     2018-08-02   4.08
9     1962-01-15   2018-08-02     2018-08-02   4.10
10    1962-01-16   2018-08-02     2018-08-02   4.13
11    1962-01-17   2018-08-02     2018-08-02   4.12
12    1962-01-18   2018-08-02     2018-08-02   4.11
13    1962-01-19   2018-08-02     2018-08-02   4.11
14    1962-01-22   2018-08-02     2018-08-02   4.09
15    1962-01-23   2018-08-02     2018-08-02   4.11
16    1962-01-24   2018-08-02     2018-08-02   4.10
17    1962-01-25   2018-08-02     2018-08-02   4.11
18    1962-01-26   2018-08-02     2018-08-02   4.11
19    1962-01-29   2018-08-02     2018-08-02   4.12
20    1962-01-30   2018-08-02     2018-08-02   4.11
21    1962-01-31   2018-08-02     2018-08-02   4.10
22    1962-02-01   2018-08-02     2018-08-02   4.09
23    1962-02-02   2018-08-02     2018-08-02   4.08
24    1962-02-05   2018-08-02     2018-08-02   4.07
25    1962-02-06   2018-08-02     2018-08-02   4.06
26    1962-02-07   2018-08-02     2018-08-02   4.07
27    1962-02-08   2018-08-02     2018-08-02   4.07
28    1962-02-09   2018-08-02     2018-08-02   4.05
29    1962-02-12   2018-08-02     2018-08-02    NaN

In my SQLalchemy model, I am trying to store data from the "value" column as follows:

value = db.Column(db.Float) 

It is currently in the float64 datatype, but SQLite needs it to be in a generic Python float type (rather than NumPy, which is float64). I am trying to convert the "value" column into generic Python float values as follows:

ValueData = GeneralData['value'].astype(float) #slices the 'value' column from the dataframe 

ValueForSQL = Graph(value = ValueData) #stores values in the 'value' column of SQLite table.

I am however getting the following error when running the code: "StatementError cannot convert the series to class 'float'".

The issue appears to be that SQlite cannot process series or lists of data. If I use GeneralData['value'].iloc[3], for example, it works fine. This is because the data is already in float64 format.

SQlite accepts this because it is an individual data point, rather than a list or series. 'Value' is column of thousands of values, however. How can get SQlite to accept the entire series?


回答1:


astype will still be converting it to float64. How about converting the series to a list and mapping it to float values

list(map(float,GeneralData.value))


来源:https://stackoverflow.com/questions/51661061/how-to-insert-timeseries-data-into-sqlite-using-sqlalchemy

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!