SQLAlchemy: Knowing the field names and values of a model object?

Question

I'm trying to keep to SOLID object oriented programming principles, stay DRY, etc, but my newness to Python/SQLAlchemy/Pyramid is making it very hard.

I'm trying to take what I now know to be a SQLAlchemy model used to create a simple Pyramid Framework object and use what I know to be "reflection" in C#, it may be called something different in Python (Introspection? Not sure as this is only my second week with python but I have lots of experience in other languages (C/C++/C#,Java, etc) so the trouble seems to be mapping my knowledge to the vocabulary of python, sorry), to find out the field names of the database table, and most importantly, the current field values, when I do not know the column names or ANY of the shape of the object in advance.

Thats right; I don't know that the 'derp' instance has a field named id or name, just that it has columns and a value in each of them. And thats all I care about.

The goal is to be able to take any SQLAlchemy defined data model, and convert it to a dictionary of column_name -> column_value fields of simple data types of the kind found in JSON as I want to ultimately serialize any object I create in SQLAlchemy to a json object, but I will settle for a dictionary as from there its trivial as long as the dictionary holds the correct types of data. Doing this for every object by hand is a violation of too many good clean code rules and will create too much work over time; I could spend another week on this and still save time and effort by doing it the right way.

So if I have a class defined in SQLAlchemy as:

class SimpleFooModel(Base):
    id = Column(Integer, primary_key=True, autoincrement=True, nullable=False)
    name = Column(VARCHAR(length=12), nullable=False, index=True)

.. and I have an instance of this equal to (in python):

derp = SimpleFooModel(id=7, name="Foobar")

I want to be able to having ONLY the 'derp' instance variable described above, and NO OTHER KNOWLEDGE of how the model is shaped, and be able to flatten it out to a python key->value dictionary for that simple object, where every value in that dictionary can be serialized to JSON using import json from python syslib.

The problem is , I have been up for 2 days looking at this and I cant find an answer that gives me the results I want in my unit tests ANYWHERE; Google keeps taking me to really old posts here on SO about really old versions of the library that either use interfaces that no longer apply, or have accepted answers that do not actually work at all; and since none of them are recent that does surprise me (but why Stack Overflow keeps them when they are wrong and allows google to mislead people does surprise me)

I know I could wire every object manually for every object to json, etc, but thats not only NOT ELEGANT, its inefficient because it just creates more work for me as I create more objects and could lead to big bugs down the road. I want to know how to do this the correct way, with introspection/reflection, but nobody seems to know, and the people who claim to have all given examples here on stack overflow that actually do not work at all (at least with the current versions of things)

This seems like a really common use case for me; and getting the column field list and then iterating through it with getattr - like many of the answers say to do - doesn't work as expected either; it just creates what look like namespaces that never return the actual value of the column, and don't actually exist in any code as none of the fields created by sqlalchmy are singleton/static.

So:

    from sqlalchemy.inspection import inspect

    obj = inspect(derp, raiseerr=True)

    for key in obj.attrs.keys():
        fields[key] = getattr(derp, key)
        print fields[key]

Just gives me:

    [Class Name].[Column Name]

.. or in this case just gives me:

    SimpleFooModel.id
    SimpleFooModel.name

NOT the values of 7 and "Foobar" for id and name respectively, that I actually expected in my tests.

In fact it seems like I cant even find WHERE the values are being stored in the object model; or I could brute force the issue and get them from there as an ugly, evil hack I would be ashamed to look at. All I get through the "official public api" is a lot of objects that seem to have no clue where the real data is being stored, but will happily tell me the name of the path used by the column name and type, restrictions, etc... just not the actual data that I want.

Yet since my requirement is that I do not know the field names in advance, using a call to derp.id or derp.name to collect the value is not an option since that would violate SOLID and force me to duplicate work for every single class. So its not an option.

Maybe its the fact I have not slept in 2 days but its really hard for me to not see this as a serious design flaw in these libs; I just want to serialize a SQLAlchemy defined Model object representing a single row in a table into a python dictionary without having to know the names of the fields in advance, and while many other languages make this easy or even trivial, this seems to be far too hard than it should be.

Can somebody please explain either a working solution or why I am wrong to want to apply SOLID to my code?

EDIT: Updated spelling.

Answer 1

Extend your model with following class:

class BaseModel(object):

    @classmethod
    def _get_keys(cls):
        return sa.orm.class_mapper(cls).c.keys()

    def get_dict(self):
        d = {}
        for k in self._get_keys():
            d[k] = getattr(self, k)
        return d

This will do exactly what you want, return a dict in form of {'column_name':'value'} pairs.