Avro schema doesn't honor backward compatibilty

Question

I have this avro schema

{
 \"namespace\": \"xx.xxxx.xxxxx.xxxxx\",
 \"type\": \"record\",
 \"name\": \"MyPayLoad\",
 \"fields\": [
     {\"name\": \"filed1\"         


        

Answer 1

I am facing this exact situation. Data written by the older schema fails when trying to read it with the newer schema. The newer schema has just one additional field with union and default set. "type":["null","string"],"doc":"","default":null

In spite of setting default, the null doesn't get filled in automatically during reading. Both the writer and the reader schemas need to be provided during reading. My understanding was avro is backward compatible and it should be able to support newer columns without the need for the older schema.