How to calculate the entropy of a file?

Question

How to calculate the entropy of a file? (Or let\'s just say a bunch of bytes)
I have an idea, but I\'m not sure that it\'s mathematically correct.

My id

Answer 1

For what it's worth, here's the traditional (bits of entropy) calculation represented in C#:

/// 

/// returns bits of entropy represented in a given string, per 
/// http://en.wikipedia.org/wiki/Entropy_(information_theory) 
/// 
public static double ShannonEntropy(string s)
{
    var map = new Dictionary();
    foreach (char c in s)
    {
        if (!map.ContainsKey(c))
            map.Add(c, 1);
        else
            map[c] += 1;
    }

    double result = 0.0;
    int len = s.Length;
    foreach (var item in map)
    {
        var frequency = (double)item.Value / len;
        result -= frequency * (Math.Log(frequency) / Math.Log(2));
    }

    return result;
}