bigquery-standard-sql

Build array based on add and remove event rows in BigQuery

僤鯓⒐⒋嵵緔 提交于 2020-02-02 15:26:06
问题 I have a table in BigQuery with the following structure: datetime | event | value ========================== 1 | add | 1 ---------+--------+------- 2 | remove | 1 ---------+--------+------- 6 | add | 2 ---------+--------+------- 8 | add | 3 ---------+--------+------- 11 | add | 4 ---------+--------+------- 23 | remove | 3 ---------+--------+------- I'm trying to build a view which adds a list column to each row containing the current state of the array. The array will never contain duplicate

How to convert an Epoch timestamp to a Date in Standard SQL

余生长醉 提交于 2020-01-13 08:38:07
问题 I didn't find any simple answer to this while I was looking around, so I thought I'd put it up here in case anyone was having the same problem as me with what should have been a trivial issue. I was using ReDash analytics with Google's BigQuery and had turned on Standard SQL in the datasource settings. For the purposes of my query, I needed to convert a timestamp - unix time in milliseconds, as a string - to a Date format so that I could use the DATE_DIFF method. As an example...

How to transform IP addresses into geolocation in BigQuery standard SQL?

偶尔善良 提交于 2020-01-04 00:45:19
问题 So I have read https://cloudplatform.googleblog.com/2014/03/geoip-geolocation-with-google-bigquery.html But I was wondering if there was a #standardSQL way of doing it. So far, I have a lot of challenge converting PARSE_IP and NTH() since the suggested changes in the migration docs have limitations. Going from PARSE_IP(contributor_ip) to NET.IPV4_TO_INT64(NET.SAFE_IP_FROM_STRING(contributor_ip)) does not work for IPv6 IP addresses. Going from NTH(1, latitude) lat to latitude[SAFE_ORDINAL(1)]

How to transform IP addresses into geolocation in BigQuery standard SQL?

旧城冷巷雨未停 提交于 2020-01-04 00:44:47
问题 So I have read https://cloudplatform.googleblog.com/2014/03/geoip-geolocation-with-google-bigquery.html But I was wondering if there was a #standardSQL way of doing it. So far, I have a lot of challenge converting PARSE_IP and NTH() since the suggested changes in the migration docs have limitations. Going from PARSE_IP(contributor_ip) to NET.IPV4_TO_INT64(NET.SAFE_IP_FROM_STRING(contributor_ip)) does not work for IPv6 IP addresses. Going from NTH(1, latitude) lat to latitude[SAFE_ORDINAL(1)]

How to transform IP addresses into geolocation in BigQuery standard SQL?

一笑奈何 提交于 2020-01-04 00:44:17
问题 So I have read https://cloudplatform.googleblog.com/2014/03/geoip-geolocation-with-google-bigquery.html But I was wondering if there was a #standardSQL way of doing it. So far, I have a lot of challenge converting PARSE_IP and NTH() since the suggested changes in the migration docs have limitations. Going from PARSE_IP(contributor_ip) to NET.IPV4_TO_INT64(NET.SAFE_IP_FROM_STRING(contributor_ip)) does not work for IPv6 IP addresses. Going from NTH(1, latitude) lat to latitude[SAFE_ORDINAL(1)]

BigQuery: dynamically select table by the current date in Standard SQL?

折月煮酒 提交于 2019-12-30 07:48:31
问题 I am trying to find the table of the current date of SELECT * FROM `da`.`m`.`ga_realtime_20190306` but not working SELECT * FROM `da`.`m`.`CONCAT('ga_realtime_', FORMAT_DATE('%Y%m%d', CURRENT_DATE())` How can I dynamically select a table with CURRENT_DATE and the BigQuery Standard query? 回答1: Use wildcard and _TABLE_SUFFIX SELECT field1, field2, field3 FROM `my_dataset.ga_realtime_*` WHERE _TABLE_SUFFIX = FORMAT_DATE('%Y%m%d', CURRENT_DATE()) 来源: https://stackoverflow.com/questions/57817881

Extracting date from timestamp in Bigquery: a preferable method

天大地大妈咪最大 提交于 2019-12-23 10:56:33
问题 A quick question to Bigquery gurus. Here are two methods for extracting date from a timestamp in Bigquery using standardSQL #standardSQL #1 DATE(TIMESTAMP_MILLIS(CAST((timestamp) AS INT64))) #2 EXTRACT(DATE FROM TIMESTAMP_MILLIS(timestamp)) Which one is more preferable and why? Thanks! 回答1: It really comes down to personal preference; one isn't superior to the other since they have the same semantics and performance. The argument in favor of using EXTRACT is that if you are extracting other

BigQuery - rank rows by desc order, based on values in one of the columns, removing duplicates

喜夏-厌秋 提交于 2019-12-23 04:40:53
问题 For every 20 minute period interval, I am trying to find/rank the unique ip addresses, with their corresponding port number, which generate the highest volume of traffic, in mbps (megabits per second), descending order. Each IP address may or may be recorded more than once in each 20-minute period. Each time an IP address gets recorded in the 20-minute period interval, it may or may not have the same port number listed. For example, in the table below, the ip address 192.168.10.1 shows up

BigQuery: Union two different tables which are based on federated Google Spreadsheet

淺唱寂寞╮ 提交于 2019-12-22 18:13:18
问题 I have two different Google Spreadsheet: One with 4 columns +------+------+------+------+ | Col1 | Col2 | Col5 | Col6 | +------+------+------+------+ | ID1 | A | B | C | | ID2 | D | E | F | +------+------+------+------+ One with the 4 columns of the previous file, and 2 more columns +------+------+------+------+------+------+ | Col1 | Col2 | Col3 | Col4 | Col5 | Col6 | +------+------+------+------+------+------+ | ID3 | G | H | J | K | L | | ID4 | M | N | O | P | Q | +------+------+------+---

DATE_ADD or DATE_DIFF error when grouping dates in BigQuery

帅比萌擦擦* 提交于 2019-12-13 20:24:13
问题 Try and search as I might, I still cannot figure this out and although https://cloud.google.com/bigquery/docs/reference/standard-sql/date_functions#date_add has assisted a bit, I am still stuck. I am trying to group dates into weeks but keep getting either one of the two errors below the code. day bitcoin_total dash_total 2009-01-03 1 0 2009-01-09 14 0 2009-01-10 61 0 The desirable outcome would be the date at the start of the week (could be Monday or Sunday, whichever) day bitcoin_total dash