Escaping special characters in SphinxSE

Question

Im using sphinx storage engine implementation for searching on my site, which works fairly well, however when a search includes characters such as & and @, the search fails with the following error:

There was a problem processing the query on the foreign data source. Data source error: search query already specified

and php throws this error:

Warning: mysql_query() [function.mysql-query]: Unable to save result set in /home/path/to/file.php on line 100

Im escaping the user's input with mysql_real_escape_string

Whats interesting is if I copy the query and run it in phpmyadmin directly, I get no errors.

 query = '@title("cheese & cake");limit=1000filter=type=1;ranker=sph04;mode=extended;sort=extended:@weight desc;'

Answer 1

Character escaping in Sphinxql is a tricky subject... I'm not sure if it is fully officially resolved. mysql_real_escape_string won't handle the special Sphinx query characters.

They do provide an escape function in sphinxapi.php:

function EscapeString ( $string )
{
    $from = array ( '\\', '(',')','|','-','!','@','~','"','&', '/', '^', '$', '=' );
    $to   = array ( '\\\\', '\(','\)','\|','\-','\!','\@','\~','\"', '\&', '\/', '\^', '\$', '\=' );
    return str_replace ( $from, $to, $string );
 }

Note that this won't specifically handle the SQL escape characters (for example, no single quote replacement). Actually, I tested it, and it doesn't even work just for Sphinx characters.

You need this function:

function EscapeSphinxQL ( $string )
{
    $from = array ( '\\', '(',')','|','-','!','@','~','"','&', '/', '^', '$', '=', "'", "\x00", "\n", "\r", "\x1a" );
    $to   = array ( '\\\\', '\\\(','\\\)','\\\|','\\\-','\\\!','\\\@','\\\~','\\\"', '\\\&', '\\\/', '\\\^', '\\\$', '\\\=', "\\'", "\\x00", "\\n", "\\r", "\\x1a" );
    return str_replace ( $from, $to, $string );
}

Note the extra backslashes on the Sphinx-specific characters. I think what happens is that they put your whole query through an SQL parser, which removes escape backslashes 'extraneous' for SQL purposes (i.e. '\&' -> '&'). Then, it puts the MATCH clause through the fulltext parser, and suddenly '&' is a special character. So, you need the extra backslashes in the beginning.