Converting Matlab's datenum format to Python

Question

I just started moving from Matlab to Python 2.7 and I have some trouble reading my .mat-files. Time information is stored in Matlab\'s datenum format. For those who are not

Answer 1

Just in case it's useful to others, here is a full example of loading time series data from a Matlab mat file, converting a vector of Matlab datenums to a list of datetime objects using carlosdc's answer (defined as a function), and then plotting as time series with Pandas:

from scipy.io import loadmat
import pandas as pd
import datetime as dt
import urllib

# In Matlab, I created this sample 20-day time series:
# t = datenum(2013,8,15,17,11,31) + [0:0.1:20];
# x = sin(t)
# y = cos(t)
# plot(t,x)
# datetick
# save sine.mat

urllib.urlretrieve('http://geoport.whoi.edu/data/sine.mat','sine.mat');

# If you don't use squeeze_me = True, then Pandas doesn't like 
# the arrays in the dictionary, because they look like an arrays
# of 1-element arrays.  squeeze_me=True fixes that.

mat_dict = loadmat('sine.mat',squeeze_me=True)

# make a new dictionary with just dependent variables we want
# (we handle the time variable separately, below)
my_dict = { k: mat_dict[k] for k in ['x','y']}

def matlab2datetime(matlab_datenum):
    day = dt.datetime.fromordinal(int(matlab_datenum))
    dayfrac = dt.timedelta(days=matlab_datenum%1) - dt.timedelta(days = 366)
    return day + dayfrac

# convert Matlab variable "t" into list of python datetime objects
my_dict['date_time'] = [matlab2datetime(tval) for tval in mat_dict['t']]

# print df
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 201 entries, 2013-08-15 17:11:30.999997 to 2013-09-04 17:11:30.999997
Data columns (total 2 columns):
x    201  non-null values
y    201  non-null values
dtypes: float64(2)

# plot with Pandas
df = pd.DataFrame(my_dict)
df = df.set_index('date_time')
df.plot()

Answer 2

Here's a way to convert these using numpy.datetime64, rather than datetime.

origin = np.datetime64('0000-01-01', 'D') - np.timedelta64(1, 'D')
date = serdate * np.timedelta64(1, 'D') + origin

This works for serdate either a single integer or an integer array.

Answer 3

Using pandas, you can convert a whole array of datenum values with fractional parts:

import numpy as np
import pandas as pd
datenums = np.array([737125, 737124.8, 737124.6, 737124.4, 737124.2, 737124])
timestamps = pd.to_datetime(datenums-719529, unit='D')

The value 719529 is the datenum value of the Unix epoch start (1970-01-01), which is the default origin for pd.to_datetime().

I used the following Matlab code to set this up:

datenum('1970-01-01')  % gives 719529
datenums = datenum('06-Mar-2018') - linspace(0,1,6)  % test data
datestr(datenums)  % human readable format

Answer 4

You link to the solution, it has a small issue. It is this:

python_datetime = datetime.fromordinal(int(matlab_datenum)) + timedelta(days=matlab_datenum%1) - timedelta(days = 366)

a longer explanation can be found here

Answer 5

Just building on and adding to previous comments. The key is in the day counting as carried out by the method toordinal and constructor fromordinal in the class datetime and related subclasses. For example, from the Python Library Reference for 2.7, one reads that fromordinal

Return the date corresponding to the proleptic Gregorian ordinal, where January 1 of year 1 has ordinal 1. ValueError is raised unless 1 <= ordinal <= date.max.toordinal().

However, year 0 AD is still one (leap) year to count in, so there are still 366 days that need to be taken into account. (Leap year it was, like 2016 that is exactly 504 four-year cycles ago.)

These are two functions that I have been using for similar purposes:

import datetime 

def datetime_pytom(d,t):
'''
Input
    d   Date as an instance of type datetime.date
    t   Time as an instance of type datetime.time
Output
    The fractional day count since 0-Jan-0000 (proleptic ISO calendar)
    This is the 'datenum' datatype in matlab
Notes on day counting
    matlab: day one is 1 Jan 0000 
    python: day one is 1 Jan 0001
    hence an increase of 366 days, for year 0 AD was a leap year
'''
dd = d.toordinal() + 366
tt = datetime.timedelta(hours=t.hour,minutes=t.minute,
                       seconds=t.second)
tt = datetime.timedelta.total_seconds(tt) / 86400
return dd + tt

def datetime_mtopy(datenum):
'''
Input
    The fractional day count according to datenum datatype in matlab
Output
    The date and time as a instance of type datetime in python
Notes on day counting
    matlab: day one is 1 Jan 0000 
    python: day one is 1 Jan 0001
    hence a reduction of 366 days, for year 0 AD was a leap year
'''
ii = datetime.datetime.fromordinal(int(datenum) - 366)
ff = datetime.timedelta(days=datenum%1)
return ii + ff 

Hope this helps and happy to be corrected.