Kafka consumer and consumerbuild difference in encoding

问题

I was using old kafka where i was using piece of code below where mEncoding could be utf-7, utf-8, unicode etc.

new Consumer<Ignore, string>(mConfig, null, new StringDeserializer(mEncoding)))

I am upgrading my kafka to 1.4.0 version.

I found that Consumer is replaced by ConsumerBuilder where method SetValueDeserializer is available, but its accepting only utf-8 (Deserializers.Utf8). Is there any way i can send other encoding also?

回答1:

You should just implement your own deserializer. It could look something like this:

public class MyValueDeserializer : IDeserializer<string>
{
    private readonly Encoding _encoding;

    public MyValueDeserializer(Encoding encoding)
    {
        _encoding = encoding;
    }

    public string Deserialize(ReadOnlySpan<byte> data, bool isNull, SerializationContext context)
    {
        if (isNull)
        {
            return null;
        }

        var array = data.ToArray();

        using var stream = new MemoryStream(array);
        using var sr = new StreamReader(stream, _encoding);

        return sr.ReadToEnd();
    }
}

Then you could use SetValueDeserializer(new MyValueDeserializer(Encoding.UTF7))

来源：https://stackoverflow.com/questions/62403591/kafka-consumer-and-consumerbuild-difference-in-encoding