google-bigquery

BigQuery add new columns into nested STRUCT in subquery

非 Y 不嫁゛ 提交于 2020-03-24 01:58:27
问题 I have the following table that contains nested STRUCT s, and in a subquery, I am trying to add additional columns at the struct level . I've created a reproducible example of my efforts thus far: WITH wide_stats AS ( ( SELECT 'joe' name, 'bills' team, struct(struct(7 as fga, 5 as fgm) as o, struct(8 as fga, 3 as fgm) as d) as t1, struct(struct(3 as fga, 4 as fgm) as o, struct(9 as fga, 2 as fgm) as d) as t2 ) UNION ALL ( SELECT 'nick' name, 'jets' team, struct(struct(12 as fga, 7 as fgm) as

Calculate HMAC in Google BigQuery SQL

落爺英雄遲暮 提交于 2020-03-23 08:25:18
问题 How do I calculate an HMAC in Google BigQuery? BigQuery includes a number of crypto related functions, such as hash functions and encryption functions, but a HMAC function (that calculates a signature) is missing. 回答1: The BigQuery developers apparently forgot to add HMAC functions, but fortunately that can be built on what is available. HMAC is defined as HMAC(Key, Message) = H((Key xor opad) concat H((Key xor ipad) concat Message)) where H is a hash function and Key , ipad and opad are all

Create ranges based on the data

心不动则不痛 提交于 2020-03-23 07:52:29
问题 I have products sold at different prices. I want to see how many products were sold in a particular price range. For this, I need to go through the data and see how to divide the ranges and then get the count of products in that range. Data looks like below- Product Price sold A 4.5 B 45.7 C 20 D 20.1 E 36.8 F 50 For example, for the above data I see that the min is 4.5 and max is 50. So, I decide to divide the price range like- 0-10$, 11-20$, 21-30$, 30-40$, 40-50$ So, the result should look

How to delete/update nested data in bigquery

我们两清 提交于 2020-03-23 07:13:08
问题 Is there a way to delete/update nested field in bigquery? Let's say I have this data wives.age wives.name name 21 angel adam 20 kale 21 victoria rossi 20 jessica or in json: {"name":"adam","wives":[{"name":"angel","age":21},{"name":"kale","age":20}]} {"name":"rossi","wives":[{"name":"victoria","age":21},{"name":"jessica","age":20}]} As you can see from the data above. Adam has 2 wives, named angel and kale. How to: Delete kale record. Update jessica to dessica I tried to google this, but can

Retrieving first and last records of each group

喜夏-厌秋 提交于 2020-03-21 10:33:18
问题 Distance travelled between points: I have a set of gps points travelled by each vehicle. I am trying to retrieve the first and last records for each trip. Data: VehicleId TripId Latitude Longitude 121 131 33.645 -84.424 121 131 33.452 -84.409 121 131 33.635 -84.424 121 131 35.717 -85.121 121 131 35.111 -85.111 In the above dataset, I need the resultset to be the first and last points of each trip. VehicleId TripId StartLat StartLong EndLat EndLong 121 131 33.645 -84.424 35.111 -85.111 I tried

Bigquery input delay when typing when query length is past 300 lines

旧巷老猫 提交于 2020-03-16 08:09:12
问题 Google BigQuery recently deprecated their legacy UI in favor of their new UI. While the new UI is cleaner looking, has a linter, and has better autocomplete, I had avoided using it because the performance when typing crashes when queries are longer than around 300 lines. For example, if I paste in a query it will take ~30 seconds for the linter to run during which time I cannot do anything. Then if I try to type anything new in, there will be an input delay of a few seconds. This issue did

Bigquery input delay when typing when query length is past 300 lines

☆樱花仙子☆ 提交于 2020-03-16 08:08:32
问题 Google BigQuery recently deprecated their legacy UI in favor of their new UI. While the new UI is cleaner looking, has a linter, and has better autocomplete, I had avoided using it because the performance when typing crashes when queries are longer than around 300 lines. For example, if I paste in a query it will take ~30 seconds for the linter to run during which time I cannot do anything. Then if I try to type anything new in, there will be an input delay of a few seconds. This issue did

BigQuery - Extract data from an array of Json files

故事扮演 提交于 2020-03-05 04:27:10
问题 Good morning, In big query I have a table with one column on the following form (I show 3 lines): [{'a':1,'b':4, 'c':5}, {'a':0,'b':7, 'c':8},{'a':4,'b':9, 'c':12}] [{'a':9,'b':10, 'c':9}] [{ 'a':5,'b':10, 'c':9}, {'a':1,'b':10, 'c':9}, {'a':7,'b':10, 'c':9}] that is, I have an array (Not fixed length) of Jsons. For every row I would like to extract (create a new column), if it is present, the value of the key 'b' when the key 'a'=1 (for every row the key 'a' can be equal 1 just one time). I

BigQuery - Extract data from an array of Json files

江枫思渺然 提交于 2020-03-05 04:27:06
问题 Good morning, In big query I have a table with one column on the following form (I show 3 lines): [{'a':1,'b':4, 'c':5}, {'a':0,'b':7, 'c':8},{'a':4,'b':9, 'c':12}] [{'a':9,'b':10, 'c':9}] [{ 'a':5,'b':10, 'c':9}, {'a':1,'b':10, 'c':9}, {'a':7,'b':10, 'c':9}] that is, I have an array (Not fixed length) of Jsons. For every row I would like to extract (create a new column), if it is present, the value of the key 'b' when the key 'a'=1 (for every row the key 'a' can be equal 1 just one time). I

Why doesn't BigQuery UI allow setting a destination table if a query uses variables?

ε祈祈猫儿з 提交于 2020-03-05 04:22:28
问题 In BigQuery, I can compose a query and then set a destination table under More > Query Settings. This works as expected for queries without variables, for example: SELECT * FROM foo.bar WHERE PARSE_TIMESTAMP("%a, %d %b %Y %X %z", date_created) > '2020-01-01 00:00:00'; However, when I try to replace that formatting string with a variable, suddenly the options to set a destination table do not exist under More > Query Settings. For example: DECLARE date_format STRING DEFAULT "%a, %d %b %Y %X %z