Protobuf-net memcache provider null type error on deserialize

问题

I'm using lastest protobuf-net lib with protobuf-net memcache provider. I need to serialize list of custom type MyClass

[ProtoContract]
public class MyClass{
  [ProtoMember(1)]
  public int a {get; set;}
  [ProtoMember(2)]
  public int b {get; set;}
}

So I need to store/retreive:

List<MyClass> myList

When values are stored through protobuf and then retreived from cache all goes well. But if value is stored in memcache (in correct protobuf format) i.e. from another thread/app and after that that value is retreived from cache deserialization fails due to NullReferenceExceptions of type field.

So when set goes first, type of serialized value is stored in typeCache variable and then retreived from that dictionary. But if value is present in memcache but not set in current thread typeCache var doesn't contain that type and throws NullReference on deserialization.

Are there any ways to fix this or some workaround?

Deeper investigations: Serialization/deserialization process for enyim implemented in ProtoTranscoder.cs. It contains NetTranscoder class that has Dictionary<ArraySegment<byte>, Type> typeCache. So when set goes first serialized type (i.e. List<MyClass>) is stored in typeCache var and all goes well. If value is present in memcache, but not set in current app/thread on deserialize it retreived by this code:

type = Type.GetType(enc.GetString(buffer, keyOffset, len));
byte[] standaloneBuffer = new byte[len];
Buffer.BlockCopy(buffer, keyOffset, standaloneBuffer, 0, len);
key = new ArraySegment<byte>(standaloneBuffer, 0, len);
sync.EnterWriteLock();
try
   {
        // did somebody beat us to it?
        Type tmp;
        if (typeCache.TryGetValue(key, out tmp)) return tmp;
        typeCache.Add(key, type);
        return type;   <-- Here it returns null, if type not present in typeCache
   }
finally
   {
        sync.ExitWriteLock();
   }

To reproduce that error:

List item
Create and store some List in memcache (with configured prototranscoder)
Restart current app (or start another thread)
Try to get value by key from memcache from those "another thread"

Here is stack trace of this error: [ArgumentNullException: Value cannot be null. Parameter name: type]

   ProtoBuf.Meta.TypeModel.PrepareDeserialize(Object value, Type& type) in c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:592
   ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, Type type, SerializationContext context) in c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:577
   ProtoBuf.Caching.Enyim.NetTranscoder.Enyim.Caching.Memcached.ITranscoder.Deserialize(CacheItem item) in c:\Users\akureniov\work\protobuf-net-1\protobuf-net.Enyim\protobuf-net.Enyim\ProtoTranscoder.cs:109
   Enyim.Caching.MemcachedClient.PerformTryGet(String key, UInt64& cas, Object& value) +179
   Enyim.Caching.MemcachedClient.TryGet(String key, Object& value) +42
   Enyim.Caching.MemcachedClient.Get(String key) +15

回答1:

With help of Marc Gravell we found "bug" in type trimming in ProtoTranscoder.cs: NetTranscoder: void WriteType(MemoryStream ms, Type type). This code block cause error, because it cuts too much:

int i = typeName.IndexOf(','); // first split
if (i >= 0) { i = typeName.IndexOf(',', i + 1); } // second split
if (i >= 0) { typeName = typeName.Substring(0, i); } // extract type/assembly only

It worked well for simple types, but fails on List, Dictionary, etc.

To avoid this it's better to use regexp to cut not nessesary info (like culture, publickkeytoken, etc). So here is a replacement (need to replace above rows with this), quiet rude, but working in most cases:

typeName = Regex.Replace(typeName, @", Version=\d+.\d+.\d+.\d+", string.Empty);
typeName = Regex.Replace(typeName, @", Culture=\w+", string.Empty);
typeName = Regex.Replace(typeName, @", PublicKeyToken=\w+", string.Empty);

This regexp doesn't cut type's assembly, which is needed for custom types. But for standart types it's mscorlib and it can be deleted safely in most cases by adding another line:

typeName = Regex.Replace(typeName, @", mscorlib", string.Empty);

回答2:

i think that memcache does not understand ProtoContract, try to mark them as DataContract

来源：https://stackoverflow.com/questions/14862333/protobuf-net-memcache-provider-null-type-error-on-deserialize

标签

.net

memcached

protobuf-net