crc32

I've seen 8-bit, 16-bit, and 32-bit CRCs. At what point do I need to jump to a wider CRC? My gut reaction is that it is based on the data length: 1-100 bytes: 8-bit CRC 101 - 1000 bytes: 16-bit CRC 1001 - ??? bytes: 32-bit CRC EDIT: Looking at the Wikipedia page about CRC and Lott's answer, here' what we have: <64 bytes: 8-bit CRC <16K bytes: 16-bit CRC <512M bytes: 32-bit CRC It's not a research topic. It's really well understood: http://en.wikipedia.org/wiki/Cyclic_redundancy_check The math is pretty simple. An 8-bit CRC boils all messages down to one of 256 values. If your message is more

Can CRC32 be used as a hash function? Any drawbacks to this approach? Any tradedeoffs? CRC32 works very well as a hash algorithm. The whole point of a CRC is to hash a stream of bytes with as few collisions as possible. That said, there are a few points to consider: CRC's are not secure. For secure hashing you need a much more computationally expensive algorithm. For a simple bucket hasher, security is usually a non-issue. Different CRC flavors exist with different properties. Make sure you use the right algorithm, e.g. with hash polynomial 0x11EDC6F41 (CRC32C) which is the optimal general

I'm writing a Python program to extract data from the middle of a 6 GB bz2 file. A bzip2 file is made up of independently decryptable blocks of data, so I only need to find a block (they are delimited by magic bits), then create a temporary one-block bzip2 file from it in memory, and finally pass that to the bz2.decompress function. Easy, no? The bzip2 format has a crc32 checksum for the file at the end. No problem, binascii.crc32 to the rescue. But wait. The data to be checksummed does not necessarily end on a byte boundary, and the crc32 function operates on a whole number of bytes. My plan:

I'm looking for a modern JavaScript implementation of CRC32. This implementation , which may have originated from here , and is now here, there and everywhere , is unacceptable because it's slow (500ms/MB), and depends on over 2KB of space delimited table, accessed using substr. Yuck! There appears to be a few variations of CRC32, so I need to match this output: mysql> SELECT CRC32('abcde'); > 2240272485 Function doesn't actually need to accept a string however, since I'm working with byte arrays. Update I added a helper function to create the CRCTable instead of having this enormous literal

Maybe I'm just not seeing it, but CRC32 seems either needlessly complicated, or insufficiently explained anywhere I could find on the web. I understand that it is the remainder from a non-carry-based arithmetic division of the message value, divided by the (generator) polynomial, but the actual implementation of it escapes me. I've read A Painless Guide To CRC Error Detection Algorithms , and I must say it was not painless. It goes over the theory rather well, but the author never gets to a simple "this is it." He does say what the parameters are for the standard CRC32 algorithm, but he