Updating a table schema while streaming inserts

问题

I have a table that is continuously receiving streaming inserts (potentially thousands per second).

I am interested in using the Update functionality (via API calls) to add a column. Can I call Update to add a column to an existing table, while data is still being inserted, without concern for loss of data?

For reference, here is the code I am planning on using to add a column to the table:

func addColumnToTable(service *bigquery.Service, project, dataset, table string, newCols map[string]string) bool {
    resp, err := service.Tables.Get(project, dataset, table).Do()
    if err != nil {
        fmt.Println(err)
        return false
    }

    for col, col_type := range newCols {
        this_col := &bigquery.TableFieldSchema{}
        this_col.Name = col
        this_col.Type = col_type
        resp.Schema.Fields = append(resp.Schema.Fields, this_col)
    }

    _, err = service.Tables.Update(project, dataset, table, resp).Do()
    if err != nil {
        fmt.Println(err)
        return false
    }

    return true
}

回答1:

You can update the schema of a table using tables.update (the only allowed updates are adding a column and relaxing requiredness of a column).

There is a slight delay with which schema updates become visible to streaming. So, you will need to wait for a small amount of time (upto 5 minutes) before streaming data containing new column (assuming your update is that it added a column). Old data will not be lost.

来源：https://stackoverflow.com/questions/39086636/updating-a-table-schema-while-streaming-inserts

标签

google-bigquery