I am experimenting with creating a simple message system (PHP) page that uses a MySQL table to store the entries. The rough outline of the columns I\'ll use in the table are
So I had an idea that might accomplish this, but it's all conceptual.
Suppose that you had the value "Lorem ipsum dolor sit amet", and you wanted to do a search for "lorem". One way is that you could take the original and break it up into chunks (lowercased), and put them in a second table. The whole (original) value is in the original table column with row_id 123, but a new table called "chunks" might have:
row_id | chunk | foreign_row_id
1 | lo | 123
2 | or | 123
3 | re | 123
4 | em | 123
5 | m | 123
6 | i | 123
7 | ip | 123
Think of it like a substring index, where every substring is 2 characters long.
Now, when a user wanted to perform a search, you would similarly chunk that up, then do a lookup. If they typed "lo", then you see which foreign row IDs matched. But if they enter "lore", then you do a search for all foreign row IDs that had a matching chunk for "lo", "or", AND "re".
So far, not too practical. However, if the original value "Lorem ipsum dolor sit amet" is encrypted or hashed, then you could ALSO chunk up the 2-char substrings, encrypt/hash them, then do a lookup on the chunks instead or the full string. No decryption or unhashing necessary.
The logic would be:
Any match can then be fetched from the original table. This would protect the data at rest, because if the chunk table was compromised, they can't do anything with a bunch of encrypted/hashed 2-char values. You can't take 2 encrypted/hashed substrings and recombine them or get anything meaningful from them.
If I am the inventor and got to name this, since it's similar but not quite the same as making a Rainbow Table, I would called this "Fruity Pebbles Tables". Because of the chunks.