google-bigquery

Turning JSON blob into BQ friendly format with JQ

我只是一个虾纸丫 提交于 2021-02-16 21:24:15
问题 To be upfront, I have next to no experience with JSON, JQ, or much of anything on the Java side. I've been spending a lot of time trying to use the jq command line function to properly format a test blob of data in a way that I can easily feed into Google BigQuery. { "total_items": 848, "page_count": 34, "items": [ { "landing_id": "708d9e3eb106820f98162d879198774b", "token": "708d9e3eb106820f98162d879198774b", "response_id": "708d9e3eb106820f98162d879198774b", "landed_at": "2019-02-12T01:58

How to export data from Firebase Crashlytics to another application?

徘徊边缘 提交于 2021-02-16 20:20:46
问题 I am interested in fetching the Crash-free users statistics and the user affected by Crashes trend from the Google firebase Crashlytics. Is there any API using which I can call using my application to export the data from Firebase Crashlytics? I looked at a few other similar questions like theses: How long does Firebase store my crash data? / Export data from Crashlytics? Export data from Firebase Crashlytics console? How to export crash-free users from firebase? , but none of them actually

How to export data from Firebase Crashlytics to another application?

时光总嘲笑我的痴心妄想 提交于 2021-02-16 20:19:01
问题 I am interested in fetching the Crash-free users statistics and the user affected by Crashes trend from the Google firebase Crashlytics. Is there any API using which I can call using my application to export the data from Firebase Crashlytics? I looked at a few other similar questions like theses: How long does Firebase store my crash data? / Export data from Crashlytics? Export data from Firebase Crashlytics console? How to export crash-free users from firebase? , but none of them actually

how clustering works in BigQuery

孤街浪徒 提交于 2021-02-16 15:32:07
问题 I have a a table UNITARCHIVE partitionned by date, and clustered by UNIT, DUID. the total size of the table 892 Mb. when I try this query SELECT * FROM `test-187010.ReportingDataset.UNITARCHIVE` WHERE duid="RRSF1" and unit="DUNIT" Bigquery tell me, it will process 892 mb, I thought clustering is supposed to reduce the scanned size, I understand when I filter per date, the size is reduced dramatically, but i need the whole date range. is it by design or am I doing something wrong 回答1: To get

Join by nearest date for the table with duplicate records in BigQuery

Deadly 提交于 2021-02-15 07:48:19
问题 I have installs table with installs that have the same user_id but different install_date . I want to get all revenue records joined with nearest install record by install_date that is less then revenue_date because I need it's source field value for next processing. That means that output rows count should be equal to revenue table records. How can it be achieved in BigQuery? Here is the data: installs install_date user_id source -------------------------------- 2020-01-10 user_a source_I

It's possible to create a rule in preceding rows in sum?

人盡茶涼 提交于 2021-02-13 05:46:45
问题 I have a historical stock variation values and have to calculate the stock value. But I also have a rule that the stock is max(stock, 0), or in other words, if stock is negative, I consider it as zero. To make this in Big Query, I think the way is like this: with stock as ( SELECT -7.34 variation, UNION ALL SELECT -1.81 UNION ALL SELECT -0.51 UNION ALL SELECT 17.19 UNION ALL SELECT -1.63 UNION ALL SELECT 2.82 UNION ALL SELECT -1.00 UNION ALL SELECT 0.56 UNION ALL SELECT -17.92 UNION ALL

How to get combination of value from single column?

寵の児 提交于 2021-02-11 15:23:44
问题 I'm trying to get distinct possible combination value from single column in BigQuery. Suppose i have this table: +---------------------------------------------+ | date |type |payment |customer_no|status| +---------------------------------------------+ |2019-01-02|Shirt |Cashless| 101|Cancel| |2019-01-02|Jeans |Cashless| 133|OK | |2019-01-02|Jeans |Cash | 102|OK | |2019-01-02|Cap |Cash | 144|OK | |2019-01-02|Shirt |Cash | 132|OK | |2019-01-01|Jeans |Cash | 111|Cancel| |2019-01-01|Cap |Cash |

Get a Big Query script to output a table

两盒软妹~` 提交于 2021-02-11 15:14:05
问题 I am using the new scripting feature of Big Query to declare a variable and then am using that variable in a standard SQL query. The structure of the query is : DECLARE { name of variable } { data type }; SET { name of variable } = { Value }' (A SQL QUERY THEN FOLLOWS USING THE ABOVE VARIABLE) I understand that this is now a script a no longer a typical query, and thus when I run it, it runs as a sequence of executable tasks. But is there anyway in the script to explicitly state that I only

Fill with zeros in a frequency of dates group by month and year in big query

こ雲淡風輕ζ 提交于 2021-02-11 13:58:54
问题 I have a table with hiredate (Date) ,First Name (String) and Sur Name (string) like this: hireDate First Name Surname 13-oct-14 Cintia Roxana Padilla Julca 28-oct-14 Conor McAteer 28-oct-14 Paolo Mesia Macher 28-oct-14 William Anthony Whelan 15-nov-14 Peter Michael Coates 13-feb-15 Natalie Conche 15-mar-15 Beatriz Vargas Huanca 01-may-15 Walter Calle Chenccnes 04-may-15 Sarah Louise Price And I made a view of a frequency of hire_dates(DATE) and the cumulative frequency in the other column

Dynamic Column Names in BigQuery SQL Query

…衆ロ難τιáo~ 提交于 2021-02-11 13:58:10
问题 I have a BigQuery table in which every row is a visit of a user in a country. The schema is something like this: UserID | Place | StartDate | EndDate | etc ... --------------------------------------------------------------- 134 | Paris | 234687432 | 23648949 | etc ... 153 | Bangkok | 289374897 | 2348709 | etc ... 134 | Paris | 9287324892 | 3435438 | etc ... The values of the "Place" columns can be no more than tens of options, but I don't know them all in advance. I want to query this table