Bigquery importing map's json data into a table

陌路散爱 提交于 2019-12-11 19:23:55

问题


I have a json file which has data from a pojo, which is basically

String
Map<String, Set<OBJ>>

=> where object has

int
int 
string
timestamp

Here is a sample row: {"key":"123qwe123","mapData":{"3539":[{"id":36,"type":1,"os":"WINDOWS","lastSeenDate":"2015-06-03 22:46:38 UTC"}],"16878":[{"id":36,"type":1,"os":"WINDOWS","lastSeenDate":"2015-06-03 22:26:34 UTC"}],"17312":[{"id":36,"type":1,"os":"WINDOWS","lastSeenDate":"2015-06-03 22:26:48 UTC"}]}}

I tried to do following schema, but thats not working:

[
{
     "name" : "key",
     "type" : "string"
},
{
    "name" : "mapData",
    "type" : "record",
    "mode": "repeated",
"fields": [
{
    "name": "some_id",
   "type": "record",
    "mode" : "repeated",
    "fields" : [
    {
            "name": "id",
            "type": "integer",
            "mode": "nullable"
    },
    {
            "name": "type",
            "type": "integer",
            "mode": "nullable"
    },
    {
            "name": "os",
            "type": "string",
            "mode": "nullable"
    },
    {
            "name": "lastSeenDate",
            "type": "timestamp",
            "mode": "nullable"
    }
    ]
}
] } ]

When i run i get: repeated record must be imported as a JSON array I know something is up with schema but not figured out yet.

来源:https://stackoverflow.com/questions/31150177/bigquery-importing-maps-json-data-into-a-table

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!