pytz timezone conversion performance

问题

I have more than 1 million datetime object from database, and I want to convert each of them to timezone-aware datetime objects. Here is my helper funcion conv_tz:

# dt is python datetime object, src_tz and dest_tz and pytz.timezone objects
def conv_tz(dt, src_tz, dest_tz):
    if not dt: return None
    sdt = src_tz.localize(dt)
    return sdt.astimezone(dest_tz)

Here is the result from the profiler:

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1101475    1.166    0.000   44.440    0.000 ../release/python/lib/dtutil.py:128(conv_tz)
1101475    9.092    0.000   35.656    0.000 /app/python/lib/python3.4/site-packages/pytz/tzinfo.py:244(localize)

Question 1: Is there anyway to make it run faster? Each datetime object from the database is assumed to be in pytz.timezone('America/New_York'), and the destination timezone varies by each datetime object (or each row in the database)

In fact, after I get the timezone-aware datetime object, what I really want to achieve is to convert these datetime object to matlab time (which is not timezone aware.) So here is the to_mat function I use:

def to_mat(dt):
    if not dt:  return None
    val = dt.toordinal() + 366
    t = dt.time()
    return val + (((t.hour * 60) + t.minute) * 60 + t.second) / float(_seconds_day) + t.microsecond / 1.0e6 / _seconds_day

I am calling these 2 functions together for more than 1million datetime objects:

matdt = dtutil.to_mat(dtutil.conv_tz(dt, pytz.timezone('America/New_York'), dst_tz))

Question2: Maybe there is a better way to do these conversions together? Here is the profiler of to_mat, which seems less time consuming than conv_tz:

3304425    5.067    0.000    5.662    0.000 ../release/python/lib/dtutil.py:8(to_mat)

Environment: CentOS6 x64 + Python3.4.3 x64

回答1:

Thanks for J.F. Sebastian comment! Here is what I decide to use, assuming the default timezone for these datatime objects are consistent with the OS timezone:

def conv_tz2(dt, dest_tz):
    if not dt: return None
    return datetime.fromtimestamp(dt.timestamp(), dest_tz)

And it runs a small fraction of the original conv_tz. Here is a test based on half million conversion:

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
567669    0.664    0.000   23.354    0.000 ../test/test_tz.py:17(conv_tz)
567669    4.831    0.000   18.732    0.000 /app/python/lib/python3.4/site-packages/pytz/tzinfo.py:244(localize)
567669    0.472    0.000    5.786    0.000 ../test/test_tz.py:22(conv_tz2)

来源：https://stackoverflow.com/questions/33532457/pytz-timezone-conversion-performance