Is it possible to populate a large set at compile time?

Question

We have a \'delete all my data\' feature. I\'d like to delete a set of IPs from many many web log files.

Currently at runtime I open a CSV with the IP addresses to d

Answer 1

I would recommend to simply use a Build Script to read the CSV and produce a source file containing the initialized of a standard HashSet with a custom hasher (FxHash, for example).

This would let you keep the convenience of editing a CSV file, while still baking all the data into a binary. It would require some initialization time (unlike PHF), but the ability to specify a custom hash is quite beneficial.

Also, depending on the format of IPs in the logs, you may want to store either &'static str or u32; the latter is more efficient (search-wise), but the gain may be negated if a conversion is required.