SQLite Order By places umlauts & speical chars at end

Question

I'm using Phonegap to do a dictionary app for iOS.

When querying the database for an alphabetical list I use COLLATE NOCASE:

ORDER BY term COLLATE NOCASE ASC

This solved the problem that terms starting with a lower case letter where appended to the end (Picked it up from that question).

However non-standard characters as öäüéêè still get sorted in the end ~ here 2 examples:

    Expected: Öffnungszeiten     Oberved: Zuzahlung
              Zuzahlung                   Öffnungszeiten

(or)          clé                         cliquer sur
              cliquer sur                 clé

I looked around and found similar matters discussed here or here but it seems the general advice is to install some type of extension

This extension can probably help you ...

...use ICU either as an extension

SQLite supports integration with ICU ...

But I'm not sure if this is applicable in my situation where the database is not hosted by myself but running on the customers device. So I'd guess I'd to ship this extension w/ my app-package.
I'm not very familiar with iOS but I've got the feeling that would be complicated - at least.

Also in the official forum I've found that hint:

SQLite does not properly handle accented characters.

and a little bit down in the text the poster mentions a bug in SQLite.

All the links I've found haven't been active for >= 1 year and non of them seems to deal with the mobile environment I'm currently developing in.
So I was wondering if anyone else found a solution on their iOS projects.

The documentation states they're only 3 default COLLATION option:

6.0 Collating Sequences

When SQLite compares two strings, it uses a collating sequence or collating function (two words for the same thing) to determine which string is greater or if the two strings are equal. SQLite has three built-in collating functions: BINARY, NOCASE, and RTRIM.

BINARY - Compares string data using memcmp(), regardless of text encoding.
NOCASE - The same as binary, except the 26 upper case characters of ASCII are folded to their lower case equivalents before the

comparison is performed. Note that only ASCII characters are case folded. SQLite does not attempt to do full UTF case folding due to the size of the tables required. RTRIM - The same as binary, except that trailing space characters are ignored.

For now my best guess would be to do the sorting in JavaScript but I suspect that this wouldn't do anything well to overall performance.

Answer 1

The reason is that the SQLite on iOS doesn't come with ICU enabled. So you need to build your own SQLite version with ICU enabled + your own ICU version as static lib + add the ICU .dat and make SQLite load this .dat file. Then you can load any collation via a simple SQL command (i.e. 'icu_load_collation("de_DE", "DEUTSCH")', once after the db was opened)

It doesn't only sound like it's dirt work, it really is. Try to find a version of SQLite + ICU with all of it done already.