Google BQ - how to upsert existing data in tables?

Question

I\'m using Python client library for loading data in BigQuery tables. I need to update some changed rows in those tables. But I couldn\'t figure out how to correctly update

Answer 1

BigQuery is by design append-only preferred. That means that you better let duplicate rows from the same entity in the table and write your queries to always read most recent row.

Updating rows as you know in transactional tables is not possible in BQ. You have only 100 updates per table per day. That's very limited and their purpose is totally different.

Since BQ is used as data lake, you should just stream new rows every time the user eg: updates their profile. You will end up having from 20 saves 20 rows for the same user. Later you can rematerilize your table to have unique rows by removing duplicate data.

See the most question for the later: BigQuery - DELETE statement to remove duplicates