avro-tools

来源： https://stackoverflow.com/questions/57965646/how-to-convert-json-schema-to-avro-schema

来源： https://stackoverflow.com/questions/63642870/avro-is-not-able-to-deserialize-union-with-logical-types-in-fields

问题 I have some json data that looks like this: { "id": 1998983092, "name": "Test Name 1", "type": "search string", "creationDate": "2017-06-06T13:49:15.091+0000", "lastModificationDate": "2017-06-28T14:53:19.698+0000", "lastModifiedUsername": "testuser@test.com", "lockedQuery": false, "lockedByUsername": null } I am able to add the lockedQuery null value to a GenericRecord object without issue. GenericRecord record = new GenericData.Record(schema); if(json.isNull("lockedQuery")){ record.put(

问题 I have some json data that looks like this: { "id": 1998983092, "name": "Test Name 1", "type": "search string", "creationDate": "2017-06-06T13:49:15.091+0000", "lastModificationDate": "2017-06-28T14:53:19.698+0000", "lastModifiedUsername": "testuser@test.com", "lockedQuery": false, "lockedByUsername": null } I am able to add the lockedQuery null value to a GenericRecord object without issue. GenericRecord record = new GenericData.Record(schema); if(json.isNull("lockedQuery")){ record.put(

问题 I'm using this Avro schema: prices-state.avsc { "namespace": "com.company.model", "name": "Product", "type": "record", "fields": [ { "name": "product_id", "type": "string" }, { "name": "sale_prices", "type": { "name": "sale_prices", "type": "record", "fields": [ { "name": "default", "type": { "name": "default", "type": "record", "fields": [ { "name": "order_by_item_price_by_item", "type": [ "null", { "name": "markup_strategy", "type": "record", "fields": [ { "name": "type", "type": { "name":

问题 I'm using this Avro schema: prices-state.avsc { "namespace": "com.company.model", "name": "Product", "type": "record", "fields": [ { "name": "product_id", "type": "string" }, { "name": "sale_prices", "type": { "name": "sale_prices", "type": "record", "fields": [ { "name": "default", "type": { "name": "default", "type": "record", "fields": [ { "name": "order_by_item_price_by_item", "type": [ "null", { "name": "markup_strategy", "type": "record", "fields": [ { "name": "type", "type": { "name":

问题 I am trying to create a sample .avro file containing bytes as type and decimal as logicalType, But the avro file when loaded to hive table results in a different value. What could be the reason? schema.avsc: { "type" : "record", "name" : "example", "namespace" : "com.xyz.avro", "fields" : [ { "name" : "cost", "type" : { "type" : "bytes", "logicalType" : "decimal", "precision" : 38, "scale" : 10 } }] } data.json: { "cost" : "0.0" } Converted to .avro using avro-tools : java -jar avro-tools-1.8

问题 I am trying to create a sample .avro file containing bytes as type and decimal as logicalType, But the avro file when loaded to hive table results in a different value. What could be the reason? schema.avsc: { "type" : "record", "name" : "example", "namespace" : "com.xyz.avro", "fields" : [ { "name" : "cost", "type" : { "type" : "bytes", "logicalType" : "decimal", "precision" : 38, "scale" : 10 } }] } data.json: { "cost" : "0.0" } Converted to .avro using avro-tools : java -jar avro-tools-1.8

问题 I am trying to create a sample .avro file containing bytes as type and decimal as logicalType, But the avro file when loaded to hive table results in a different value. What could be the reason? schema.avsc: { "type" : "record", "name" : "example", "namespace" : "com.xyz.avro", "fields" : [ { "name" : "cost", "type" : { "type" : "bytes", "logicalType" : "decimal", "precision" : 38, "scale" : 10 } }] } data.json: { "cost" : "0.0" } Converted to .avro using avro-tools : java -jar avro-tools-1.8

问题 I have a JSON file and an avro schema file, which correctly describes it's structure. I then convert the JSON file with the Avro tools into an avro file, without getting an error, like this: java -jar .\avro-tools-1.7.7.jar fromjson --schema-file .\data.avsc .\data.json > .\data.avro I then convert the generated Avro file back to JSON to verify that I got a valid Avro file like this: java -jar .\avro-tools-1.7.7.jar tojson .\data.avro > .\data.json This throws the error: Exception in thread