google-bigquery

问题 I am quite new to Google BigQuery and definitely struggling. My table has the following content: +----------+----------------------------------------+ | order_id | line_items | +----------+----------------------------------------+ | 123 | id:1|qy:1|sum:1.00;id:2|qy:6|sum:4.50; | +----------+----------------------------------------+ | 456 | id:1|qy:3|sum:3.00;id:3|qy:4|sum:3.20; | +----------+----------------------------------------+ I would need to look it like this: +----------+----+----+---

问题 I have written an app script that will interact with the bigQuery API and fetch the bigQuery results. API is been hit around close to 60-70 times. In this process sometimes I'm getting the following error message: "JavaScript runtime exited unexpectedly" could someone help me here? Let me know if you need any additional information function fetchTables() { var timezone = "GMT+" + new Date().getTimezoneOffset()/60; var date = Utilities.formatDate(new Date(), timezone, "yyyy-MM-dd HH:mm");

问题 I created an export sink in Stackdriver to load audit logs into BigQuery. I want to be able to see audit logs from the past 3 months. However, when I queried the tables in BigQuery, I am only seeing logs from today and no earlier. I applied the following filters to my export sink. I also tried removing the timestamp filter but still only seeing logs from today and no prior. resource.type="bigquery_dataset" timestamp > "2019-05-01T23:59:09.739Z" 回答1: Exports only work for new entries. Per the

问题 I have a 100 GB table that I'm trying to load into google bigquery. It is stored as a single 100GB avro file on GCS. Currently my bq load job is failing with an unhelpful error message: UDF worker timed out during execution.; Unexpected abort triggered for worker avro-worker-156907: request_timeout I'm thinking of trying a different format. I understand that bigquery supports several formats (AVRO, JSON, CSV, Parquet, etc) and that in principle one can load large datasets in any of these

问题 I have a 100 GB table that I'm trying to load into google bigquery. It is stored as a single 100GB avro file on GCS. Currently my bq load job is failing with an unhelpful error message: UDF worker timed out during execution.; Unexpected abort triggered for worker avro-worker-156907: request_timeout I'm thinking of trying a different format. I understand that bigquery supports several formats (AVRO, JSON, CSV, Parquet, etc) and that in principle one can load large datasets in any of these

问题 I have a 100 GB table that I'm trying to load into google bigquery. It is stored as a single 100GB avro file on GCS. Currently my bq load job is failing with an unhelpful error message: UDF worker timed out during execution.; Unexpected abort triggered for worker avro-worker-156907: request_timeout I'm thinking of trying a different format. I understand that bigquery supports several formats (AVRO, JSON, CSV, Parquet, etc) and that in principle one can load large datasets in any of these

问题 I have a 100 GB table that I'm trying to load into google bigquery. It is stored as a single 100GB avro file on GCS. Currently my bq load job is failing with an unhelpful error message: UDF worker timed out during execution.; Unexpected abort triggered for worker avro-worker-156907: request_timeout I'm thinking of trying a different format. I understand that bigquery supports several formats (AVRO, JSON, CSV, Parquet, etc) and that in principle one can load large datasets in any of these

问题 When writing correlated sub-queries in BigQuery using a WHERE clause, it is resulting in an Error. LEFT OUTER JOIN cannot be used without a condition that is an equality of fields from both sides of the join. However it works in other SQL engines, I am not understanding the error or how to rewrite the queries. Different queries of the same type causing the same error. SELECT out.* FROM `august-monolith-205810.Assignment.students` AS out WHERE 2 >= ( SELECT COUNT(*) FROM `august-monolith

问题 When writing correlated sub-queries in BigQuery using a WHERE clause, it is resulting in an Error. LEFT OUTER JOIN cannot be used without a condition that is an equality of fields from both sides of the join. However it works in other SQL engines, I am not understanding the error or how to rewrite the queries. Different queries of the same type causing the same error. SELECT out.* FROM `august-monolith-205810.Assignment.students` AS out WHERE 2 >= ( SELECT COUNT(*) FROM `august-monolith

问题 I have tried installing it in the terminal and in juypter lab and it says that it has been successfully installed but when I run df = query_job.to_dataframe() I keep getting the error " ValueError: The pyarrow library is not installed, please install pyarrow to use the to_arrow() function.". I have no idea how to fix this. Any advice? I am trying to access data from google data studio ultimately with the code, from google.cloud import bigquery import pandas import numpy import pyarrow