Is it possible to have an optional field in an Avro schema (i.e. the field does not appear at all in the .json file)?

问题

Is it possible to have an optional field in an Avro schema (i.e. the field does not appear at all in the .JSON file)?

In my Avro schema, I have two fields:

{"name": "author", "type": ["null", "string"], "default": null},
{"name": "importance", "type": ["null", "string"], "default": null},

And in my JSON files those two fields can exist or not.

However, when they do not exist, I receive an error (e.g. when I test such a JSON file using avro-tools command line client):

Expected field name not found: author

I understand that as long as the field name exists in a JSON, it can be null, or a string value, but what I'm trying to express is something like "this JSON is valid if the those field names do not exist, OR if they exist and they are null or string".

Is this possible to express in an Avro schema? If so, how?

回答1:

you can define the default attribute as undefined example. so the field can be skipped.

{ "name": "first_name", "type": "string", "default": "undefined" },

Also all field are manadatory in avro. if you want it to be optional, then union its type with null. example:

  {
    "name": "username",
    "type": [
      "null",
      "string"
    ],
    "default": null
  },

回答2:

According to avro specification this is possible, using the default attribute.

See https://avro.apache.org/docs/1.8.2/spec.html

default: A default value for this field, used when reading instances that lack this field (optional). Permitted values depend on the field's schema type, according to the table below. Default values for union fields correspond to the first schema in the union.

At the example you gave, you do add the default attribute with value "null", so this should work. However, supporting this depends also on the library you use for reading the avro message (there are libraries at c,c++,python,java,c#,ruby etc.). Maybe (probably) the library you use lack this feature.

来源：https://stackoverflow.com/questions/29299610/is-it-possible-to-have-an-optional-field-in-an-avro-schema-i-e-the-field-does

标签

json

avro