Kafka Connect API error when send default value of STRUCT with both JsonConvertor and AvroConvertor

旧城冷巷雨未停 提交于 2020-01-16 11:59:54

问题


here is the code:

            SchemaBuilder schemaBuilder = SchemaBuilder.struct()
                    .field("province", SchemaBuilder.STRING_SCHEMA)
                    .field("city", SchemaBuilder.STRING_SCHEMA);

            Struct defaultValue = new Struct(schemaBuilder)
                    .put("province", "aaa")
                    .put("city", "aaaa");

            Schema addressSchema = schemaBuilder.defaultValue(defaultValue).build();

            Schema dataSchema = SchemaBuilder.struct().name("personMessage")
                    .field("address", addressSchema).build();
            Struct normalValue = new Struct(addressSchema)
                    .put("province", "bbb")
                    .put("city", "bbbb");


            Struct struct = new Struct(dataSchema).put("address", normalValue);

            BsonTimestamp timestamp = new BsonTimestamp();
            records.add(new SourceRecord(offsetKey(replicaSetName), offsetValue(timestamp.getValue()),
                    topic, struct.schema(), struct));

i'm using JsonConvertor and getting error:

Caused by: org.apache.kafka.connect.errors.DataException: Mismatching schema.
    at org.apache.kafka.connect.json.JsonConverter.convertToJson(JsonConverter.java:711)
    at org.apache.kafka.connect.json.JsonConverter.asJsonSchema(JsonConverter.java:458)
    at org.apache.kafka.connect.json.JsonConverter.asJsonSchema(JsonConverter.java:431)

the relative code is

                    Struct struct = (Struct) value;
                    if (!struct.schema().equals(schema))
                        throw new DataException("Mismatching schema.");
                    ObjectNode obj = JsonNodeFactory.instance.objectNode();
                    for (Field field : schema.fields()) {
                        obj.set(field.name(), convertToJson(field.schema(), struct.get(field)));
                    }
                    return obj;

above code will be call by below code:

                jsonSchema = JsonNodeFactory.instance.objectNode().put(JsonSchema.SCHEMA_TYPE_FIELD_NAME, JsonSchema.STRUCT_TYPE_NAME);
                ArrayNode fields = JsonNodeFactory.instance.arrayNode();
                for (Field field : schema.fields()) {
                    ObjectNode fieldJsonSchema = asJsonSchema(field.schema()).deepCopy();
                    fieldJsonSchema.put(JsonSchema.STRUCT_FIELD_NAME_FIELD_NAME, field.name());
                    fields.add(fieldJsonSchema);
                }
                jsonSchema.set(JsonSchema.STRUCT_FIELDS_FIELD_NAME, fields);

i found struct.schema() is SchemaBuilder and schema is Schema{Struct} when field = "address".

So result is i can set default value of STRUCT of record, but i cannot sent record out...

来源:https://stackoverflow.com/questions/59729574/kafka-connect-api-error-when-send-default-value-of-struct-with-both-jsonconverto

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!