Calculate time difference between Pandas Dataframe indices

Question

I am trying to add a column of deltaT to a dataframe where deltaT is the time difference between the successive rows (as indexed in the timeseries).

time             


        

Answer 1

>= Numpy version 1.7.0.

Also can typecast df.index.to_series().diff() from timedelta64[ns](nano seconds- default dtype) to timedelta64[m](minutes) [Frequency conversion (astyping is equivalent of floor division)]

df['ΔT'] = df.index.to_series().diff().astype('timedelta64[m]')

                     value      ΔT
time                              
2012-03-16 23:50:00      1     NaN
2012-03-16 23:56:00      2     6.0
2012-03-17 00:08:00      3    12.0
2012-03-17 00:10:00      4     2.0
2012-03-17 00:12:00      5     2.0
2012-03-17 00:20:00      6     8.0
2012-03-20 00:43:00      7  4343.0

(ΔT dtype: float64)

if you want to convert to int, fill na values with 0 before converting

>>> df.index.to_series().diff().fillna(0).astype('timedelta64[m]').astype('int')

time
2012-03-16 23:50:00       0
2012-03-16 23:56:00       6
2012-03-17 00:08:00      12
2012-03-17 00:10:00       2
2012-03-17 00:12:00       2
2012-03-17 00:20:00       8
2012-03-20 00:43:00    4343
Name: time, dtype: int64

Timedelta data types support a large number of time units, as well as generic units which can be coerced into any of the other units.

Below are the date units:

Y   year
M   month
W   week
D   day

below are the time units:

h   hour
m   minute
s   second
ms  millisecond
us  microsecond
ns  nanosecond
ps  picosecond
fs  femtosecond
as  attosecond

---

if you want difference upto decimals use true division, i.e., divide by np.timedelta64(1, 'm')
e.g. if df is as below,

                     value
time                      
2012-03-16 23:50:21      1
2012-03-16 23:56:28      2
2012-03-17 00:08:08      3
2012-03-17 00:10:56      4
2012-03-17 00:12:12      5
2012-03-17 00:20:00      6
2012-03-20 00:43:43      7

check the difference between asyping(floor division) and true division below.

>>> df.index.to_series().diff().astype('timedelta64[m]')
time
2012-03-16 23:50:21       NaN
2012-03-16 23:56:28       6.0
2012-03-17 00:08:08      11.0
2012-03-17 00:10:56       2.0
2012-03-17 00:12:12       1.0
2012-03-17 00:20:00       7.0
2012-03-20 00:43:43    4343.0
Name: time, dtype: float64

>>> df.index.to_series().diff()/np.timedelta64(1, 'm')
time
2012-03-16 23:50:21            NaN
2012-03-16 23:56:28       6.116667
2012-03-17 00:08:08      11.666667
2012-03-17 00:10:56       2.800000
2012-03-17 00:12:12       1.266667
2012-03-17 00:20:00       7.800000
2012-03-20 00:43:43    4343.716667
Name: time, dtype: float64