google-bigquery

问题 I vaguely remember there being a function that does this, but I think I may be going crazy. Say I have a datatable, call it table1. It has three columns: column1, column2, column3. The query SELECT * FROM table1 returns all rows/columns from table1. Isn't there some type of EXPAND function that allows me to duplicate that result? For example, if I want to duplicate everything from the SELECT * FROM table1 query three times, I can do something like EXPAND(3) ? 回答1: In BigQuery, I would

问题 I know how to get the date a table was last modified, but not accessed. Is it possible to get the last time a table was read ? Is there query or an API to get this ? 回答1: If you have audit logs in BigQuery, you can write a query like this: WITH tables AS ( SELECT FORMAT("%s.%s.%s", table.projectId, table.datasetId, table.tableId) table , MAX(timestamp) last_access FROM ( SELECT timestamp , protopayload_auditlog.servicedata_v1_bigquery.jobCompletedEvent.job.jobStatistics.referencedTables FROM

问题 I know how to get the date a table was last modified, but not accessed. Is it possible to get the last time a table was read ? Is there query or an API to get this ? 回答1: If you have audit logs in BigQuery, you can write a query like this: WITH tables AS ( SELECT FORMAT("%s.%s.%s", table.projectId, table.datasetId, table.tableId) table , MAX(timestamp) last_access FROM ( SELECT timestamp , protopayload_auditlog.servicedata_v1_bigquery.jobCompletedEvent.job.jobStatistics.referencedTables FROM

问题 I have a SQL below which is able to get the interval average of timestamp column grouped by icao_address, flight_number, flight_date. I'm trying to do the same for standard deviation and although I get a figure, it is wrong. The standard deviation that I get back is 14.06 (look at image below to see) while it should be around 1.8. Below is what I'm using for stddev calculation. STDDEV_POP(UNIX_SECONDS(timestamp))as standard_deviation Below is my SQL #standardSQL select DATE(timestamp) as

问题 I have a SQL below which is able to get the interval average of timestamp column grouped by icao_address, flight_number, flight_date. I'm trying to do the same for standard deviation and although I get a figure, it is wrong. The standard deviation that I get back is 14.06 (look at image below to see) while it should be around 1.8. Below is what I'm using for stddev calculation. STDDEV_POP(UNIX_SECONDS(timestamp))as standard_deviation Below is my SQL #standardSQL select DATE(timestamp) as

问题 I have a SQL below which is able to get the interval average of timestamp column grouped by icao_address, flight_number, flight_date. I'm trying to do the same for standard deviation and although I get a figure, it is wrong. The standard deviation that I get back is 14.06 (look at image below to see) while it should be around 1.8. Below is what I'm using for stddev calculation. STDDEV_POP(UNIX_SECONDS(timestamp))as standard_deviation Below is my SQL #standardSQL select DATE(timestamp) as

问题 I'm trying to access Google BigQuery using Service Account approach. My code is as follows: private static final HttpTransport HTTP_TRANSPORT = new NetHttpTransport(); private static final JsonFactory JSON_FACTORY = new JacksonFactory(); GoogleCredential credentials = new GoogleCredential.Builder() .setTransport(HTTP_TRANSPORT) .setJsonFactory(JSON_FACTORY) .setServiceAccountId("XXXXX@developer.gserviceaccount.com") .setServiceAccountScopes(BigqueryScopes.BIGQUERY)

问题 I am trying to execute the value of the variable, but I can't find documentation about it in Google BigQuery. DECLARE SQL STRING; SELECT SQL = CONCAT( "CREATE TABLE IF NOT EXISTS first.rdds_", REPLACE(CAST(T.actime AS STRING), " 00:00:00+00", ""), " PARTITION BY actime ", " CLUSTER BY id ", " OPTIONS( ", " partition_expiration_days=365 ", " ) ", " AS ", "SELECT * ", "FROM first.rdds AS rd ", "WHERE rd.actime = ", "'", CAST(T.actime AS STRING), "'", " AND ", "EXISTS ( ", "SELECT 1 ", "FROM

问题 I am trying to execute the value of the variable, but I can't find documentation about it in Google BigQuery. DECLARE SQL STRING; SELECT SQL = CONCAT( "CREATE TABLE IF NOT EXISTS first.rdds_", REPLACE(CAST(T.actime AS STRING), " 00:00:00+00", ""), " PARTITION BY actime ", " CLUSTER BY id ", " OPTIONS( ", " partition_expiration_days=365 ", " ) ", " AS ", "SELECT * ", "FROM first.rdds AS rd ", "WHERE rd.actime = ", "'", CAST(T.actime AS STRING), "'", " AND ", "EXISTS ( ", "SELECT 1 ", "FROM

问题 I searched for Python API to interact with google bigquery. And I found two packages provides similar APIs: Google BigQuery Client(Part of Google API Client Package googleapiclient ) and Gcloud package gcloud . Here are the documentation about using these two APIs for Bigquery: Google API Client: googleapiclient https://developers.google.com/resources/api-libraries/documentation/bigquery/v2/python/latest/index.html https://cloud.google.com/bigquery/docs/reference/v2/ Google Cloud package: