google-bigquery

问题 In Google BigQuery, can a query joins two tables in different datasets? say If I want to join these two tables, what should I do? dataset1:table1 and dataset2:table2 Thank you 回答1: #standardSQL SELECT <fields list> FROM `project.dataset1.table1` t1 JOIN `project.dataset2.table2` t2 ON t1.id = t2.id see more about JOINs in documentation 来源： https://stackoverflow.com/questions/49327613/can-query-tables-from-different-datasets-in-bigquery

问题 I'm trying to range-join two tables, like so SELECT * FROM main_table h INNER JOIN test.delay_pairs d ON d.interval_start_time_utc < h.visitStartTime AND h.visitStartTime < d.interval_end_time_utc where h.visitStartTime is an INT64 epoch and d.interval_start_time_utc and d.interval_end_time_utc are proper TIMESTAMP s. The above fails with No matching signature for operator < for argument types: TIMESTAMP, INT64. Supported signature: ANY < ANY Neither wrapping h.visitStartTime in TIMESTAMP()

问题 My table looks like this: |--------|--------|--------|--------|---------|---------| | num_1 | num_2 | num_3 |level_1 | level_2 | level_3 | |--------|--------|--------|--------|---------|---------| | 1111 | 3333 | 4444 | false | false | false | |--------|--------|--------|--------|---------|---------| | 1111 | 3333 | 5555 | false | false | false | |--------|--------|--------|--------|---------|---------| | 1111 | 6666 | null | false | false | false | |--------|--------|--------|--------|------

问题 My table looks like this: |--------|--------|--------|--------|---------|---------| | num_1 | num_2 | num_3 |level_1 | level_2 | level_3 | |--------|--------|--------|--------|---------|---------| | 1111 | 3333 | 4444 | false | false | false | |--------|--------|--------|--------|---------|---------| | 1111 | 3333 | 5555 | false | false | false | |--------|--------|--------|--------|---------|---------| | 1111 | 6666 | null | false | false | false | |--------|--------|--------|--------|------

问题 My table looks like this: |--------|--------|--------|--------|---------|---------| | num_1 | num_2 | num_3 |level_1 | level_2 | level_3 | |--------|--------|--------|--------|---------|---------| | 1111 | 3333 | 4444 | false | false | false | |--------|--------|--------|--------|---------|---------| | 1111 | 3333 | 5555 | false | false | false | |--------|--------|--------|--------|---------|---------| | 1111 | 6666 | null | false | false | false | |--------|--------|--------|--------|------

问题 I have read that it's a bad idea to parse XML/HTML using regular expressions. The alternative suggestion is to use an XML parser. Does one exist in the BigQuery Standard SQL library? 回答1: Here is the documentation to how to use Javascript UDFs in BigQuery like Elliot has mentioned. https://cloud.google.com/bigquery/docs/reference/standard-sql/user-defined-functions I imagine the UDF might look something like CREATE TEMPORARY FUNCTION XML(x STRING) RETURNS STRING LANGUAGE js AS """ var data =

问题 I have read that it's a bad idea to parse XML/HTML using regular expressions. The alternative suggestion is to use an XML parser. Does one exist in the BigQuery Standard SQL library? 回答1: Here is the documentation to how to use Javascript UDFs in BigQuery like Elliot has mentioned. https://cloud.google.com/bigquery/docs/reference/standard-sql/user-defined-functions I imagine the UDF might look something like CREATE TEMPORARY FUNCTION XML(x STRING) RETURNS STRING LANGUAGE js AS """ var data =

问题 I have some custom sql to read the data from BigQuery. How can I execute that? I tried using option as query but it is not working. It is ignoring the query option and reading the full table. Dataset<Row> testDS = session.read().format("bigquery") //.option("table", <TABLE>) .option("query",<QUERY>) .option("project", <PROJECT_ID>) .option("parentProject", <PROJECT_ID>) .load(); 回答1: That's because the query option is not available in the connector. See https://github.com/GoogleCloudDataproc

问题 I've got a table in Google BigQuery that consists of a few fields, then a REPEATED record which may contain one or more objects. I want to create a new table with an extra field in the REPEATED data, and copy my original data into the new table, populating the new field with the output of GENERATE_UUID() so there is one unique identifier per REPEATED line of data. I had a similar question at How do I copy from one BigQuery Table to another when the target contains REPEATED fields? but I don't

问题 I have some custom sql to read the data from BigQuery. How can I execute that? I tried using option as query but it is not working. It is ignoring the query option and reading the full table. Dataset<Row> testDS = session.read().format("bigquery") //.option("table", <TABLE>) .option("query",<QUERY>) .option("project", <PROJECT_ID>) .option("parentProject", <PROJECT_ID>) .load(); 回答1: That's because the query option is not available in the connector. See https://github.com/GoogleCloudDataproc