JSON serialization of Google App Engine models

Question

I\'ve been searching for quite a while with no success. My project isn\'t using Django, is there a simple way to serialize App Engine models (google.appengine.ext.db.Model)

Answer 1

You don't need to write your own "parser" (a parser would presumably turn JSON into a Python object), but you can still serialize your Python object yourself.

Using simplejson:

import simplejson as json
serialized = json.dumps({
    'filename': self.filename,
    'title': self.title,
    'date_taken': date_taken.isoformat(),
    # etc.
})

Answer 2

To serialize models, add a custom json encoder as in the following python:

import datetime
from google.appengine.api import users
from google.appengine.ext import db
from django.utils import simplejson

class jsonEncoder(simplejson.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return obj.isoformat()

        elif isinstance(obj, db.Model):
            return dict((p, getattr(obj, p)) 
                        for p in obj.properties())

        elif isinstance(obj, users.User):
            return obj.email()

        else:
            return simplejson.JSONEncoder.default(self, obj)


# use the encoder as: 
simplejson.dumps(model, cls=jsonEncoder)

This will encode:

• a date as as isoformat string (per this suggestion),
• a model as a dict of its properties,
• a user as his email.

To decode the date you can use this javascript:

function decodeJsonDate(s){
  return new Date( s.slice(0,19).replace('T',' ') + ' GMT' );
} // Note that this function truncates milliseconds.

Note: Thanks to user pydave who edited this code to make it more readable. I had originally had used python's if/else expressions to express jsonEncoder in fewer lines as follows: (I've added some comments and used google.appengine.ext.db.to_dict, to make it clearer than the original.)

class jsonEncoder(simplejson.JSONEncoder):
  def default(self, obj):
    isa=lambda x: isinstance(obj, x) # isa(<type>)==True if obj is of type <type>
    return obj.isoformat() if isa(datetime.datetime) else \
           db.to_dict(obj) if isa(db.Model) else \
           obj.email()     if isa(users.User) else \
           simplejson.JSONEncoder.default(self, obj)

Answer 3

To serialize a Datastore Model instance you can't use json.dumps (haven't tested but Lorenzo pointed it out). Maybe in the future the following will work.

http://docs.python.org/2/library/json.html

import json
string = json.dumps(['foo', {'bar': ('baz', None, 1.0, 2)}])
object = json.loads(self.request.body)

Answer 4

I've extended the JSON Encoder class written by dpatru to support:

• Query results properties (e.g. car.owner_set)
• ReferenceProperty - recursively turn it into JSON

• Filtering properties - only properties with a verbose_name will be encoded into JSON

  class DBModelJSONEncoder(json.JSONEncoder):
      """Encodes a db.Model into JSON"""
  
      def default(self, obj):
          if (isinstance(obj, db.Query)):
              # It's a reference query (holding several model instances)
              return [self.default(item) for item in obj]
  
          elif (isinstance(obj, db.Model)):
              # Only properties with a verbose name will be displayed in the JSON output
              properties = obj.properties()
              filtered_properties = filter(lambda p: properties[p].verbose_name != None, properties)
  
              # Turn each property of the DB model into a JSON-serializeable entity
              json_dict = dict([(
                      p,
                      getattr(obj, p)
                          if (not isinstance(getattr(obj, p), db.Model))
                          else
                      self.default(getattr(obj, p)) # A referenced model property
                  ) for p in filtered_properties])
  
              json_dict['id'] = obj.key().id() # Add the model instance's ID (optional - delete this if you do not use it)
  
              return json_dict
  
          else:
              # Use original JSON encoding
              return json.JSONEncoder.default(self, obj)
  

Answer 5

There's a method, "Model.properties()", defined for all Model classes. It returns the dict you seek.

from django.utils import simplejson
class Photo(db.Model):
  # ...

my_photo = Photo(...)
simplejson.dumps(my_photo.properties())

See Model properties in the docs.

Answer 6

In the latest (1.5.2) release of the App Engine SDK, a to_dict() function that converts model instances to dictionaries was introduced in db.py. See the release notes.

There is no reference to this function in the documentation as of yet, but I have tried it myself and it works as expected.