reading millisecond data into pandas

问题

I have a file with data like this, and want to load it, and use timestamp column (which denotes milliseconds) as a DateTimeIndex.

                  x           y   
timestamp                                                                   
0                 50          90    
125               37          87    
234               37          87     
344               37          87     
453               37          87     
562               26          78    
656               26          78    
766               26          78   
875               26          78     
984               30          77

when I specify timestamp as index, it becomes FloatIndex

cur_df = pd.read_csv(cur_file, sep=',', comment='#', index_col = 'timestamp', parse_dates=True)

EDIT: I added a function to parse dates, adding a dummy date:

def convert_time(a):
    sec = int(math.floor(a/1000))
    millisec = int(((a/1000.0)-int(math.floor(a/1000.0)))*1000)
    time = '2012-01-01 00:00:%d.%d' % (sec, millisec)
    return parser.parse(time)


cur_df = pd.read_csv(cur_file, sep=',', comment='#', index_col = 'timestamp', parse_dates=True, date_parser=convert_time)

now it works ok!

i'd be grateful for any suggestions how could I accomplish this better ;)

回答1:

Something similar, but simpler I think (python datetime.datetime uses microseconds, so therefore the factor 1000):

In [12]: import datetime

In [13]: def convert_time(a):
    ...:     ms = int(a)
    ...:     return datetime.datetime(2012, 1, 1, 0, 0, 0, ms*1000)

In [14]: pd.read_csv(cur_file, sep=',', index_col = 'timestamp', parse_dates=True, date_parser=convert_time)
Out[14]: 
                             x   y
timestamp                         
2012-01-01 00:00:00         50  90
2012-01-01 00:00:00.125000  37  87
2012-01-01 00:00:00.234000  37  87
2012-01-01 00:00:00.344000  37  87
2012-01-01 00:00:00.453000  37  87
2012-01-01 00:00:00.562000  26  78
2012-01-01 00:00:00.656000  26  78
2012-01-01 00:00:00.766000  26  78
2012-01-01 00:00:00.875000  26  78
2012-01-01 00:00:00.984000  30  77

来源：https://stackoverflow.com/questions/24036028/reading-millisecond-data-into-pandas

标签

python

pandas