murmurhash

Fast hash function with collision possibility near SHA-1

核能气质少年 提交于 2019-12-05 17:30:26
问题 I'm using SHA-1 to detect duplicates in a program handling files. It is not required to be cryptographic strong and may be reversible. I found this list of fast hash functions https://code.google.com/p/xxhash/ What do I choose if I want a faster function and collision on random data near to SHA-1? Maybe a 128 bit hash is good enough for file deduplication? (vs 160 bit sha-1) In my program the hash is calculated on chuncks from 0 - 512 KB. 回答1: Maybe this will help you: https:/

How to create a custom Murmur Avalanche Mixer?

寵の児 提交于 2019-12-05 15:29:11
I'm trying to use an Avalanche mixer to hash integer coordinates. I've been using Murmur3's 32bit and 64bit avalanche mixers to do so (and not the actual total hash function). For my application the entire hash function is not needed, only the Avalanche Mixer seen here: uint32_t murmurmix32( uint32_t h ) { h ^= h >> 16; h *= 0x85ebca6b; h ^= h >> 13; h *= 0xc2b2ae35; h ^= h >> 16; return h; } uint64_t murmurmix64( uint64_t h ) { h ^= h >> 33; h *= 0xff51afd7ed558ccdULL; h ^= h >> 33; h *= 0xc4ceb9fe1a85ec53ULL; h ^= h >> 33; return h; } These appear fast on my machine, I take two uint32_ts and

Fast hash function with collision possibility near SHA-1

China☆狼群 提交于 2019-12-04 02:10:05
I'm using SHA-1 to detect duplicates in a program handling files. It is not required to be cryptographic strong and may be reversible. I found this list of fast hash functions https://code.google.com/p/xxhash/ What do I choose if I want a faster function and collision on random data near to SHA-1? Maybe a 128 bit hash is good enough for file deduplication? (vs 160 bit sha-1) In my program the hash is calculated on chuncks from 0 - 512 KB. A. Binzxxxxxx Maybe this will help you: https://softwareengineering.stackexchange.com/questions/49550/which-hashing-algorithm-is-best-for-uniqueness-and

Murmur3 hash different result between Python and Java implementation

◇◆丶佛笑我妖孽 提交于 2019-12-01 09:41:54
问题 I have two different program that wish to hash same string using Murmur3 in Python and Java respectively. Python version 2.7.9: mmh3.hash128('abc') Gives 79267961763742113019008347020647561319L. Java is Guava 18.0: HashCode hashCode = Hashing.murmur3_128().newHasher().putString("abc", StandardCharsets.UTF_8).hash(); Gives string "6778ad3f3f3f96b4522dca264174a23b", converting to BigInterger gives 137537073056680613988840834069010096699. How to get same result from both? Thanks 回答1: Here's how

Is any 64-bit portion of a 128-bit hash as collision-proof as a 64-bit hash?

南楼画角 提交于 2019-11-28 23:17:42
We're trying to settle an internal debate on our dev team: We're looking for a 64-bit PHP hash function. We found a PHP implementation of MurmurHash3 , but MurmurHash3 is either 32-bit or 128-bit, not 64-bit. Co-worker #1 believes that to produce a 64-bit hash from MurmurHash3, we can simply slice the first (or last, or any) 64 bits of the 128-bit hash and that it will be as collision-proof as a native 64-bit hash function. Co-worker #2 believes that we must find a native 64-bit hash function to reduce collisions and that 64-bit slices of a 128-bit hash will not be as collision proof as a

MurmurHash - what is it?

时光总嘲笑我的痴心妄想 提交于 2019-11-27 16:49:28
I've been trying to get a high level understanding of what MurmurHash does. I've read a basic description but have yet to find a good explanation of when to use it and why. I know its very fast but want to know a bit more. I asked a related question about how I could fit a UUID into a Redis bitset, and someone suggested using MurmurHash. It works but I'd like to understand the risks/benefits. Murmur is a family of good general purpose hashing functions, suitable for non-cryptographic usage. As stated by Austin Appleby, MurmurHash provides the following benefits: simple (in term of number of

MurmurHash - what is it?

妖精的绣舞 提交于 2019-11-26 18:46:00
问题 I've been trying to get a high level understanding of what MurmurHash does. I've read a basic description but have yet to find a good explanation of when to use it and why. I know its very fast but want to know a bit more. I asked a related question about how I could fit a UUID into a Redis bitset, and someone suggested using MurmurHash. It works but I'd like to understand the risks/benefits. 回答1: Murmur is a family of good general purpose hashing functions, suitable for non-cryptographic