Checksum for a SQLite database?

Question

I'd like to be able to tell whether a SQLite database file has been updated in any way. How would I go about implementing that?

The first solution I think of is comparing checksums, but I don't really have any experience working with checksums.

Answer 1

According to http://www.sqlite.org/fileformat.html SQLite3 maintains a file change counter in byte 24..27 of the database file. It is independent of the file change time which, for example, can change after a binary restore or rollback while nothing changed at all:

$ sqlite3 test.sqlite 'create table test ( test )'
$ od --skip-bytes 24 --read-bytes=4 -tx1  test.sqlite | sed -n '1s/[^[:space:]]*[[:space:]]//p' | tr -d ' '
00000001
$ sqlite3 test.sqlite "insert into test values ( 'hello world');"
$ od --skip-bytes 24 --read-bytes=4 -tx1  test.sqlite | sed -n '1s/[^[:space:]]*[[:space:]]//p' | tr -d ' '
00000002
$ sqlite3 test.sqlite "delete from test;"
$ od --skip-bytes 24 --read-bytes=4 -tx1  test.sqlite | sed -n '1s/[^[:space:]]*[[:space:]]//p' | tr -d ' '
00000003
$ sqlite3 test.sqlite "begin exclusive; insert into test values (1); rollback;"
$ od --skip-bytes 24 --read-bytes=4 -tx1  test.sqlite | sed -n '1s/[^[:space:]]*[[:space:]]//p' | tr -d ' '
00000003

To be really sure that two database files are "equal" you can only be sure after dumping the files (.dump), reducing the output to the INSERT statements and sorting that result for compare (perhaps by some cryptographically secure checksum). But that is plain overkill.

Answer 2

Depending on the size of the database, continually polling and generating a checksum may be a bit too intensive of the machine.

Have you considered monitoring the last modified meta data stored on the OS file system instead?

Answer 3

If one of the sqlite databases is used only as a copy (read only) and you want to check whether the original database file has been updated so you can update the copy (from the web for example, without having to download the orginal if it is not different from the copy), then you may just compare the first 100 bytes of both database files (database headers) http://www.sqlite.org/fileformat.html

For bytes 24..27 of the database header, SQlite doc says:

24..27 4
The file change counter. Each time a database transaction is committed, the value of the 32-bit unsigned integer stored in this field is incremented

After some testing it appears that the file change counter is not incremented when a database transaction is committed but only contains select statements, which is the behavior you want in case you wrap your selects in a transaction and commit to end the transaction.