Python JSON encoder to support datetime?

问题

is there any elegant way to make Python JSON encoder support datetime? some 3rd party module or easy hack?

I am using tornado's database wrapper to fetch some rows from db to generate a json. The query result includes a regular MySQL timestamp column.

It's quite annoying that Python's default json encoder doesn't support its own datetime type, which is so common in all kinds of database queries.

I don't want to modify Python's own json encoder. any good practice? Thanks a lot!

ps: I found a dirty hack by modifying the Python JSON encoder default method:

Change:

def default(self, o):
    raise TypeError(repr(o) + " is not JSON serializable")

To:

def default(self, o):
    from datetime import date
    from datetime import datetime
    if isinstance(o, datetime):
        return o.isoformat()
    elif isinstance(o, date):
        return o.isoformat()
    else:
        raise TypeError(repr(o) + " is not JSON serializable")

well, it will be a temporary solution just for dev environment.

But for long term solution or production environment, this is quite ugly, and I have to do the modification every time I deploy to a new server.

Is there a better way? I do not want to modify Python code itself, neither Tornado source code. Is there something I can do with my own project code to make this happen? preferably in one pace.

Thanks a lot!

回答1:

The docs suggest subclassing JSONEncoder and implementing your own default method. Seems like you're basically there, and it's not a "dirty hack".

The reason dates aren't handled by the default encoder is there is no standard representation of a date in JSON. Some people are using the format /Date(1198908717056)/, but I prefer ISO format personally.

import json
import datetime


class DateTimeEncoder(json.JSONEncoder):

    def default(self, obj):
        if isinstance(obj, (datetime.datetime, datetime.date, datetime.time)):
            return obj.isoformat()
        elif isinstance(obj, datetime.timedelta):
            return (datetime.datetime.min + obj).time().isoformat()

        return super(DateTimeEncoder, self).default(obj)

now = datetime.datetime.now()
encoder = DateTimeEncoder()
encoder.encode({"datetime": now, "date": now.date(), "time": now.time()})
> {"datetime": "2019-07-02T16:17:09.990126", "date": "2019-07-02", "time": "16:17:09.990126"}

回答2:

json.dumps(thing, default=str)

回答3:

I made my own classes for my project:

import datetime
import decimal
import json
import sys

class EnhancedJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            ARGS = ('year', 'month', 'day', 'hour', 'minute',
                     'second', 'microsecond')
            return {'__type__': 'datetime.datetime',
                    'args': [getattr(obj, a) for a in ARGS]}
        elif isinstance(obj, datetime.date):
            ARGS = ('year', 'month', 'day')
            return {'__type__': 'datetime.date',
                    'args': [getattr(obj, a) for a in ARGS]}
        elif isinstance(obj, datetime.time):
            ARGS = ('hour', 'minute', 'second', 'microsecond')
            return {'__type__': 'datetime.time',
                    'args': [getattr(obj, a) for a in ARGS]}
        elif isinstance(obj, datetime.timedelta):
            ARGS = ('days', 'seconds', 'microseconds')
            return {'__type__': 'datetime.timedelta',
                    'args': [getattr(obj, a) for a in ARGS]}
        elif isinstance(obj, decimal.Decimal):
            return {'__type__': 'decimal.Decimal',
                    'args': [str(obj),]}
        else:
            return super().default(obj)


class EnhancedJSONDecoder(json.JSONDecoder):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, object_hook=self.object_hook,
                         **kwargs)

    def object_hook(self, d): 
        if '__type__' not in d:
            return d
        o = sys.modules[__name__]
        for e in d['__type__'].split('.'):
            o = getattr(o, e)
        args, kwargs = d.get('args', ()), d.get('kwargs', {})
        return o(*args, **kwargs)

if __name__ == '__main__':
    j1 = json.dumps({'now': datetime.datetime.now(),
        'val': decimal.Decimal('9.3456789098765434987654567')},
        cls=EnhancedJSONEncoder)
    print(j1)
    o1 = json.loads(j1, cls=EnhancedJSONDecoder)
    print(o1)

Result:

{"val": {"args": ["9.3456789098765434987654567"], "__type__": "decimal.Decimal"}, "now": {"args": [2014, 4, 29, 11, 44, 57, 971600], "__type__": "datetime.datetime"}}
{'val': Decimal('9.3456789098765434987654567'), 'now': datetime.datetime(2014, 4, 29, 11, 44, 57, 971600)}

References:

json Documentation
Mark Hildreth -- Subclassing JSONEncoder and JSONDecoder
Cédric Krier -- trytond.protocols.jsonrpc source code

Note: It can be made more flexible by passing a custom dictionary with types as keys and args, kwargs as values to the encoder's __init__() and use that (or a default dictionary) in the default() method.

回答4:

json.dumps(r, default=lambda o: o.isoformat() if hasattr(o, 'isoformat') else o)

回答5:

The Tryton project has a JSONEncoder implementation for datetime.datetime, datetime.date and datetime.time objects (with others). It is used for JSON RPC communication between the server and client.

See http://hg.tryton.org/2.4/trytond/file/ade5432ac476/trytond/protocols/jsonrpc.py#l53

回答6:

Convert the datetime type into a unix timestamp, then encode the contents into a json.

e.g. : http://codepad.org/k3qF09Kr

回答7:

Create a custom decoder/encoder:

class CustomJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return http_date(obj)
        if isinstance(obj, uuid.UUID):
            return str(obj)
        return json.JSONEncoder.default(self, obj)

class CustomJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        json.JSONDecoder.__init__(self, object_hook=self.object_hook, *args, **kwargs)

    def object_hook(self, source):
        for k, v in source.items():
            if isinstance(v, str):
                try:
                    source[k] = datetime.datetime.strptime(str(v), '%a, %d %b %Y %H:%M:%S %Z')
                except:
                    pass
        return source

回答8:

Just create a custom encoder

(the small but important addition to Cole's answer is the handling of pd.NaT (or null/empty timestamp values), since without the addition you will get very weird timestamp conversions for NaT/missing timestamp data)

class CustomEncoder(json.JSONEncoder):
    def default(self, obj):
        if pd.isnull(obj):
            return None
        elif isinstance(obj, datetime):
            return obj.isoformat()
        elif isinstance(obj, date):
            return obj.isoformat()
        elif isinstance(obj, timedelta):
            return (datetime.min + obj).time().isoformat()
        else:
            return super(CustomEncoder, self).default(obj)

Then use it to encode a dataframe:

df_as_dict = df.to_dict(outtype = 'records')  # transform to dict

df_as_json = CustomEncoder().encode(df_as_dict) #transform to json

Since the encoder standardized the data, the regular decoder will act fine in transforming it back to a dataframe:

result_as_dict = json.JSONDecoder().decode(df_as_json) # decode back to dict

result_df = pd.DataFrame(result)  # transform dict back to dataframe

Of course this will also work if you put the dataframe into a larger dict before encoding, e.g

input_dict = {'key_1':val_1,'key_2':val_2,...,'df_as_dict':df_as_dict}
input_json = CustomEncoder().encode(input_dict)
input_json_back_as_dict = json.JSONDecoder().decode(input_json)
input_df_back_as_dict = input_json_back_as_dict['df_as_dict']
input_df_back_as_df = pd.DataFrame(input_df_back_as_dict)

来源：https://stackoverflow.com/questions/12122007/python-json-encoder-to-support-datetime

标签

python

mysql

json

ios5

tornado