C# Protobuf-net: Dictionary of decimals: Zeroes don't get roundtrip properly

问题

I've found a weird bug around serialization/deserialization of decimal zeroes in protobuf-net, wondering if anyone has found a good workaround for this, or if this is actually a feature.

Given a dictionary like above, if i run in linqpad:

void Main()
{
    {
        Dictionary<string, decimal> dict = new Dictionary<string, decimal>();
        dict.Add("one", 0.0000000m);
        DumpStreamed(dict);
    }

    {
        Dictionary<string, decimal> dict = new Dictionary<string, decimal>();
        dict.Add("one", 0m);
        DumpStreamed(dict);
    }
}

public static void DumpStreamed<T>(T val)
{
    using (var stream = new MemoryStream())
    {
        Console.Write("Stream1: ");
        ProtoBuf.Serializer.Serialize(stream, val);
        foreach (var by in stream.ToArray())
        {
            Console.Write(by);
        }

        Console.WriteLine();
        Console.Write("Stream2: ");
        stream.Position = 0;
        var item = ProtoBuf.Serializer.Deserialize<T>(stream);
        using(var stream2 = new MemoryStream())
        {
            ProtoBuf.Serializer.Serialize(stream2, item);
            foreach (var by in stream2.ToArray())
            {
                Console.Write(by);
            }

        }
    }

    Console.WriteLine();
    Console.WriteLine("----");
}

I'll get two different streams:

First serialization: 1091031111101011822414

Second serialization: 107103111110101180

(The 0.0000000m gets converted to 0 on deserialization).

I've found this is due to this line of code in ReadDecimal:

 if (low == 0 && high == 0) return decimal.Zero;

Does anyone know why zeroes are getting normalized only during deserialization, and not on serialization?

Or any workaround for either consistently normalizing or consistently not normalizing decimal zero in a dictionary on serialization/deserialization?

回答1:

Yep; the problem is this well-meaning but potentially harmful line:

    if (low == 0 && high == 0) return decimal.Zero;

which neglects to check signScale. It should really be:

    if (low == 0 && high == 0 && signScale == 0) return decimal.Zero;

I'll tweak that for the next build.

(edit: I ended up removing that check completely - the rest of the code is just some integer shifts etc, so having the "branch" is probably more expensive than not having it)

回答2:

Floating point data types are actually structures with several elements. Among them are base value and an exponent to which the base value is to be raised. The c# documentation for decimal states the following:

The binary representation of a Decimal number consists of a 1-bit sign, a 96-bit integer number, and a scaling factor used to divide the integer number and specify what portion of it is a decimal fraction. The scaling factor is implicitly the number 10, raised to an exponent ranging from 0 to 28

So for example you could represent 1234000 as

A base value of 1234000 x 10 ^ 0
A base value of 123000 x 10 ^1
A base value of 12300 x 10 ^ 2

etc.

So this problem isn't just limited to zero. All decimal values could be represented more than one way. If you are relying on the byte streams to check for equivalence, you're in for a lot of problems. You really shouldn't be doing that, as you will definitely get false positives, not just for zero either.

As for normalization while serializing, I think that is a problem specific to ProtoBuf. You could certainly write your own serialization that takes steps to normalize the data, although it might be tricky to figure out. Another option is to convert the decimals to some custom class before storage, or store them as their string representations, as odd as that may sound.

If you are interested in monkeying around with some decimals and inspecting the raw data, see the GetBits() method. Or you could use this extension method to view the in-memory representation and see for yourself:

public static unsafe string ToBinaryHex(this decimal This)
{
    byte* pb = (byte*)&This;
    var bytes = Enumerable.Range(0, 16).Select(i => (*(pb + i)).ToString("X2"));
    return string.Join("-", bytes);
}

来源：https://stackoverflow.com/questions/50807668/c-sharp-protobuf-net-dictionary-of-decimals-zeroes-dont-get-roundtrip-properl

标签

decimal

protobuf-net