Python + Twisted + sqlanydb = abort()

Question

I'm using Twisted 11 together with SQLAnywhere 12 via the official sqlanydb driver.

Generally, it works fine.

But occasionally the application crashes with an abort on the first query.

If one query worked, all following work too. However my tests run only seldom through.

That's awful to develop with and strace doesn't tell me anything informative too. Sometimes it crashes inside of select(), sometimes in mmap()...

I'm running 64bit Linux and run locally the Sybase as dbeng12 for testing.

Is anyone working successfully using these components? Any suggestions how to solve that? I used sqlanydb with Django before and it never crashed.

Using prints, I found out it crashes inside of the DeferredList, the important code is basically the following:

class WhoisDb(object):
    # ... shortened ...
    def _get_contacts(self, dom):
        if not dom:
            self.d.errback(UnknownDomain(self._get_limit()))
            return
        self.dom = Domain._make(dom[0])

        dl = defer.DeferredList( [
            self.dbpool.runQuery(CON_SQL, (self.dom.dom_owner,)),
            self.dbpool.runQuery(CON_SQL, (self.dom.dom_admin,)),
            self.dbpool.runQuery(CON_SQL, (self.dom.dom_tech,)),
            self.dbpool.runQuery(
                LAST_UPDATE_SQL,
                ( self.dom.domName, )), ] ).addCallback(self._fmt_string)

    def get_whois(self, domain):
        self.d = defer.Deferred()
        if not self._check_limit():
            self.d.errback(LimitExceeded(MAX_PER_HOUR))
        elif not RE_ALLOWED_TLDS.match(domain):
            self.d.errback(UnknownDomain(self._get_limit()))
        else: 
            self.dbpool.runQuery(
                    'select ' + DOM_FIELDS + ' from domains where '
                    'domain = ? or domain_idn = ?',
                    ( domain, domain, )) \
                            .addCallback(self._get_contacts)

        return self.d

_fmt_string() is not called if it crashes.

Inside gdb, it's a simple SIGSEV:

(gdb) run ~/.virtualenvs/whois/bin/trial test.test_protocol.ProtocolTestCase.test_correct_domain
Starting program: /home/hynek/.virtualenvs/whois/bin/python ~/.virtualenvs/whois/bin/trial test.test_protocol.ProtocolTestCase.test_correct_domain
[Thread debugging using libthread_db enabled]
test.test_protocol
  ProtocolTestCase
    test_correct_domain ... [New Thread 0x7ffff311a700 (LWP 6685)]
[New Thread 0x7ffff3099700 (LWP 6686)]
[New Thread 0x7ffff27dc700 (LWP 6723)]
[New Thread 0x7ffff1fdb700 (LWP 6724)]
[New Thread 0x7ffff17da700 (LWP 6725)]
[New Thread 0x7ffff0fd9700 (LWP 6729)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff1fdb700 (LWP 6724)]
0x00007ffff4d4167c in ?? () from /opt/sqlanywhere12/lib64/libdbcapi_r.so

Answer 1

It looks like your database library is not threadsafe. In order to make it a stable connection, do this:

self.dbpool = ConnectionPool(..., cp_min=1, cp_max=1)

This will set the maximum concurrency to 1, and the ThreadPool will be limited to 1 thread, meaning that no queries will run simultaneously. This should stop your non-threadsafe library from causing you any drama, while still running the queries in a thread and not blocking the mainloop.

Answer 2

Yeah, your deferred list looks like it's not going to do what you want. Each runQuery is going to be run in a adbapi threadpool so there's no guarantee of the ordering of those queries. The "LAST_UPDATE_SQL" being the last thing in the DeferredList is not necessarily going to make it happen last. Are the queries in the deferred list supposed to be part of a single transaction?

Not knowing exactly what the SQL queries are here I'm assuming that sometimes a transaction has been setup for your LAST_UPDATE_SQL and sometimes it hasn't been setup depending on the order those runQuery's end up actually running.

Here's how to replace the deferred list with a single adbapi thread using adbapi.runInteraction. I'm not 100% convinced this will fix your issues but I think it's the correct way to write the sort of database interaction you're attempting to do.

class WhoisDb(object):
    # ... shortened ...
    def _get_contacts(self, dom):
        if not dom:
            self.d.errback(UnknownDomain(self._get_limit()))
            return
        self.dom = Domain._make(dom[0])

        d = self.dbpool.runInteraction(
                 self._get_stuff_from_db
            )
        d.addCallback(self._fmt_string)
        d.addErrback(self._fmt_string) # don't forget to add an errback!
        return d

    def _get_stuff_from_db(self, cursor):
        cursor.execute(CON_SQL, (self.dom.dom_owner,)),
        cursor.execute(CON_SQL, (self.dom.dom_admin,)),
        cursor.execute(CON_SQL, (self.dom.dom_tech,)),
        cursor.execute(
            LAST_UPDATE_SQL,
            ( self.dom.domName, )), ] )
        return cursor.fetchall() # or whatever you need to return obviously