Python, MySQLdb and escaping table names?

Question

I may be missing something obvious, but I can't figure out how my code is different from various examples I see in the online documentation for MySQLdb.

I'm fairly new to programming python, more experienced with perl. What I'm trying to do is get into certain good habits early on (like in perl I always start with 'use strict; use warnings'), so I'm trying to ensure I create re-usable functions in an independent file (funct.py) to save me time later on down the line.

I've got this function for selecting random rows from a table:

def returnRandom(conn,countcol,table,field):
    cursor = conn.cursor()
    cursor.execute('SELECT MAX(%s) FROM %s',(countcol,table))
    maxRow = cursor.fetchone()[0]
    cursor.execute("SELECT MIN(%s) FROM %s",(countcol,table))
    minRow = cursor.fetchone()[0]
    randomId = random.randrange(minRow,maxRow+1,1)
    cursor.execute("SELECT ? FROM ? WHERE id >=? LIMIT 1",field,table,randomId)
    return cursor.fetchone()[0]

It's being called like this:

msg = funct.returnRandom(conn,"id","testtable","data")

Unfortunately it errors out with: _mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''testtable'' at line 1")

If I put testtable in in place of the %s on the execute line, The query will run but it looks like it runs

SELECT MAX('id') FROM testtable;

which of course returns 'id'.

Given both of those it looks like it's quoting the %s entries when it tries to execute them. I was wondering if anyone could explain how I get it to stop doing that, or how I should actually being doing what I'm trying to achieve? I want the function to be as generic as possible, so relying on data being passed to it by when it's called.

edit: I should add, if I substitute in ? marks:

....
    cursor.execute('SELECT MAX(?) FROM ?',(countcol,table))
File "/usr/lib/pymodules/python2.7/MySQLdb/cursors.py", line 151, in execute
    query = query % db.literal(args)
TypeError: not all arguments converted during string formatting

Answer 1

You cannot use DB-API for metadata; you will need to make replacements yourself outside of the execute() call.

query = 'SELECT MAX(%%s) FROM `%s`' % (table,)
cursor.execute(query, (countcol,))

Obviously you should not do this if table comes from an outside source.

Answer 2

MySQLdb is probably quoting your table names with single quotes instead of backticks. Try this

cursor.execute('SELECT MAX(%%s) FROM `%s`' % table,(countcol))

Answer 3

Interesting. But in the manual there are a couple of examples. Maybe it is something similar.

c=db.cursor()
max_price=5
c.execute("""SELECT spam, eggs, sausage FROM breakfast
          WHERE price < %s""", (max_price,))

In this example, max_price=5 Why, then, use %s in the string? Because MySQLdb will convert it to a SQL literal value, which is the string '5'. When it's finished, the query will actually say, "...WHERE price < 5".

And

c.executemany(
      """INSERT INTO breakfast (name, spam, eggs, sausage, price)
      VALUES (%s, %s, %s, %s, %s)""",
      [
      ("Spam and Sausage Lover's Plate", 5, 1, 8, 7.95 ),
      ("Not So Much Spam Plate", 3, 2, 0, 3.95 ),
      ("Don't Wany ANY SPAM! Plate", 0, 4, 3, 5.95 )
      ] )

Here we are inserting three rows of five values. Notice that there is a mix of types (strings, ints, floats) though we still only use %s. And also note that we only included format strings for one row. MySQLdb picks those out and duplicates them for each row.