google-bigquery

问题 I need some suggestions for one of the use cases I am working on. Use Case: We have data in Cloud SQL around 5-10 tables, some are treated as lookup and others transactional. We need to get this to BigQuery in a way to make 3-4 tables(Flattened, Nested or Denormalized) out of these which will be used for reporting in Data Studio, Looker, etc. Data should be processed incrementally and changes in Cloud SQL could happen every 5 min, which means that data should be available to BigQuery

问题 I need to capitalize a string: john doe -> John Doe How to do this? I suppose need to use NORMALIZE_AND_CASEFOLD, but it returns a lower case. https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#normalize_and_casefold NORMALIZE_AND_CASEFOLD NORMALIZE_AND_CASEFOLD(value[, normalization_mode]) Description Takes a STRING, value, and performs the same actions as NORMALIZE, as well as casefolding for case-insensitive operations. NORMALIZE_AND_CASEFOLD supports

问题 I need to capitalize a string: john doe -> John Doe How to do this? I suppose need to use NORMALIZE_AND_CASEFOLD, but it returns a lower case. https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#normalize_and_casefold NORMALIZE_AND_CASEFOLD NORMALIZE_AND_CASEFOLD(value[, normalization_mode]) Description Takes a STRING, value, and performs the same actions as NORMALIZE, as well as casefolding for case-insensitive operations. NORMALIZE_AND_CASEFOLD supports

问题 I need to capitalize a string: john doe -> John Doe How to do this? I suppose need to use NORMALIZE_AND_CASEFOLD, but it returns a lower case. https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#normalize_and_casefold NORMALIZE_AND_CASEFOLD NORMALIZE_AND_CASEFOLD(value[, normalization_mode]) Description Takes a STRING, value, and performs the same actions as NORMALIZE, as well as casefolding for case-insensitive operations. NORMALIZE_AND_CASEFOLD supports

问题 As anyone who has ever had the misfortune of having to interact with the panoply of Google CLI binaries programmatically will have realised, authenticating with the likes of gcloud , gsutil , bq , etc. is far from intuitive or trivial, especially when you need to work across different projects. I am running various cron jobs that interact with Google Cloud Storage and BigQuery for different projects. Since the cron jobs may overlap, renaming config files is clearly not an option, and nor

问题 I'm playing with BigQuery's Standard SQL Wildcard Table functionality. I have a table with the following schema: _PARTITIONTIME TIMESTAMP NULLABLE stamp TIMESTAMP NULLABLE value FLOAT NULLABLE source STRING NULLABLE ... Because the tables in my dataset have mixed schemas and their names are not commonly prefixed, I intend on making a wide Wildcard match - then, narrowing _TABLE_SUFFIX in the WHERE : SELECT *, _TABLE_SUFFIX AS table_name FROM `my-project.my-dataset.*` WHERE REGEXP_CONTAINS(

问题 I'm playing with BigQuery's Standard SQL Wildcard Table functionality. I have a table with the following schema: _PARTITIONTIME TIMESTAMP NULLABLE stamp TIMESTAMP NULLABLE value FLOAT NULLABLE source STRING NULLABLE ... Because the tables in my dataset have mixed schemas and their names are not commonly prefixed, I intend on making a wide Wildcard match - then, narrowing _TABLE_SUFFIX in the WHERE : SELECT *, _TABLE_SUFFIX AS table_name FROM `my-project.my-dataset.*` WHERE REGEXP_CONTAINS(

问题 I have thousands of script that include loop over dataset and use variables for accumulation. Eg: // assuming that 'ids' is populated from some BQ table ids = [1, 2, 3, 4, 5] var1 = "v1" //initialize variable var2 = "v2" //initialize variable for id in ids var1 = var2 if (id > 2) var2 = var1 + "-" + item else var2 = id print(id, var1, var2) This would produce below output: 1,v2,1 2,1,2 3,2,2-3 4,2-3,2-3-4 5,2-3-4,2-3-4-5 Few other things to consider: Script can involve n number of variables.

问题 I have thousands of script that include loop over dataset and use variables for accumulation. Eg: // assuming that 'ids' is populated from some BQ table ids = [1, 2, 3, 4, 5] var1 = "v1" //initialize variable var2 = "v2" //initialize variable for id in ids var1 = var2 if (id > 2) var2 = var1 + "-" + item else var2 = id print(id, var1, var2) This would produce below output: 1,v2,1 2,1,2 3,2,2-3 4,2-3,2-3-4 5,2-3-4,2-3-4-5 Few other things to consider: Script can involve n number of variables.

问题 We are trying to find a syntax to generate the DAY|WEEK|MONTH options from the 3rd param of date functions. DECLARE var_date_option STRING DEFAULT 'DAY'; select GENERATE_DATE_ARRAY('2019-01-01','2020-01-01',INTERVAL 1 WEEK) dynamic param here -^^^ Do you know what's the proper syntax to use in DECLARE and that should be converted to valid SQL. 回答1: Below is for BigQuery Standard SQL Those DAY|WEEK|MONTH are LITERALs and cannot be parametrized And, as you know - dynamic SQL is also not