How does SHA generate unique codes for big files in git

Question

Using Git I don\'t understand how using SHA you can generate just a 40 hexadecimal digit code that can then be mapped to any file which could be hundreds of lines long.

Answer 1

In fact, what I call your "margin of safety" determines how many objects you can store.

The widely quoted "about 2⁸⁰" number is the point at which you have approximately a 50% chance of a hash collision. To keep the chance below about 1 out of 10¹⁸, the number of distinct objects in the repository should not exceed about 1.7 quadrillion (1.71x10¹⁵).

(I did some math for a book I'm working on; I haven't had it checked by a real mathematician, but when I ran the same sort of numbers against other hash sizes, my outputs agreed with those on Wikipedia, for whatever that's worth. :-) )

Edit to add: here's the approximation formula. Let r be the cardinality of the hash function (so r is 2¹⁶⁰ for SHA-1) and U be the desired probability-of-uniqueness (so U is 0.5 for the usual "50% chance of safety, 50% chance of collision" statistic. The maximum number of hash inputs is:

(1 + sqrt(1 + 8r ln (1 / U)) / 2

The natural log of 1 / .5 is about 0.693, so we have about sqrt(4r)/2, which is of course just about sqrt(r). Hence for a k-bit hash, "50% probability of uniqueness" occurs after about k/2 hashes.

To see (ballpark) how I get my number—in the neighborhood of 10¹⁵ objects—let U = 1 - 10^-18. The natural log of this number is basically the original 10^-18, which means we knock most of 2⁶⁰ off the range r, leaving about 2¹⁰⁰. The square root of that is about 2⁵⁰ which is about 10¹⁵.