group-by

Grouping records and getting standard deviation intervals for grouped records in BigQuery, getting wrong value

試著忘記壹切 提交于 2020-01-19 18:04:28
问题 I have a SQL below which is able to get the interval average of timestamp column grouped by icao_address, flight_number, flight_date. I'm trying to do the same for standard deviation and although I get a figure, it is wrong. The standard deviation that I get back is 14.06 (look at image below to see) while it should be around 1.8. Below is what I'm using for stddev calculation. STDDEV_POP(UNIX_SECONDS(timestamp))as standard_deviation Below is my SQL #standardSQL select DATE(timestamp) as

Pandas merge and grouby

 ̄綄美尐妖づ 提交于 2020-01-17 16:41:03
问题 I have 2 pandas dataframes which looks like below. Data Frame 1: Section Chainage Frame R125R002 10.133 1 R125R002 10.138 2 R125R002 10.143 3 R125R002 10.148 4 R125R002 10.153 5 Data Frame 2: Section Chainage 1 2 3 4 5 6 7 8 R125R002 10.133 0 0 1 0 0 0 0 0 R125R002 10.134 0 0 1 0 0 0 0 0 R125R002 10.135 0 0 1 0 0 0 0 0 R125R002 10.136 0 0 1 0 0 0 0 0 R125R002 10.137 0 0 1 0 0 0 0 0 R125R002 10.138 0 0 1 0 0 0 0 0 R125R002 10.139 0 0 1 0 0 0 0 0 R125R002 10.14 0 0 1 0 0 0 0 0 R125R002 10.141 0

Pandas merge and grouby

梦想的初衷 提交于 2020-01-17 16:40:08
问题 I have 2 pandas dataframes which looks like below. Data Frame 1: Section Chainage Frame R125R002 10.133 1 R125R002 10.138 2 R125R002 10.143 3 R125R002 10.148 4 R125R002 10.153 5 Data Frame 2: Section Chainage 1 2 3 4 5 6 7 8 R125R002 10.133 0 0 1 0 0 0 0 0 R125R002 10.134 0 0 1 0 0 0 0 0 R125R002 10.135 0 0 1 0 0 0 0 0 R125R002 10.136 0 0 1 0 0 0 0 0 R125R002 10.137 0 0 1 0 0 0 0 0 R125R002 10.138 0 0 1 0 0 0 0 0 R125R002 10.139 0 0 1 0 0 0 0 0 R125R002 10.14 0 0 1 0 0 0 0 0 R125R002 10.141 0

How to add GROUP_CONCAT to LEFT JOIN query?

*爱你&永不变心* 提交于 2020-01-17 09:13:16
问题 I have this query (and results): select articles.article_id, articles.article_text, article_photos.photo_filename from articles left join article_photos on article_photos.article_id=articles.article_id >>> results 1,some_text,photo1.jpg 1,some_text,photo2.jpg 1,some_text,photo3.jpg How do I incorporate GROUP_CONCAT to this so that I get: >>> results 1,some_text,photo1.jpg NULL,NULL,photo2.jpg NULL,NULL,photo3.jpg Basically, I have a table with articles, and related table with images. An

How to add GROUP_CONCAT to LEFT JOIN query?

家住魔仙堡 提交于 2020-01-17 09:12:08
问题 I have this query (and results): select articles.article_id, articles.article_text, article_photos.photo_filename from articles left join article_photos on article_photos.article_id=articles.article_id >>> results 1,some_text,photo1.jpg 1,some_text,photo2.jpg 1,some_text,photo3.jpg How do I incorporate GROUP_CONCAT to this so that I get: >>> results 1,some_text,photo1.jpg NULL,NULL,photo2.jpg NULL,NULL,photo3.jpg Basically, I have a table with articles, and related table with images. An

Issue combining two SQL queries

余生颓废 提交于 2020-01-17 05:18:25
问题 Hi everybody I'm trying to combine two MySQL queries into only one. I have two blocks, the FROM and the WHERE , I think they are the only ones needed in my case. FROM: FROM table1, table2, table3, table4, table5 WHERE: WHERE field1 = 1 AND (field2 = 3 OR field2 = 4) AND (table5.field3 <= 100000) AND (table5.field4 >= 1) AND (table5.field5 >= 1) AND (table5.field6 >= 60) AND table5.field2 = table4.id AND table5.field7 = table3.field8 AND table3.field9 = table1.field10 AND table3.field11 =

Group pages from one column

陌路散爱 提交于 2020-01-17 01:13:26
问题 I have these information in a page table in my db, page_id page_title 1 xy {a} 2 kp {a} 3 xyz {b} 4 mno {b} 5 abc {c} 6 mno {c} 7 qwe {c} I use curly brackets to group pages, such as page xy and kp belong to the group of a . So I want to return the result in this information, group_title page_title a xy a kp b xyz b mno c abc c mno But maximum of two pages in each group. Can I use SQL query to achieve this or PHP? This is what I am working on by using REGEXP... SELECT p.page_title REGEXP '{[a

Data Analysis in Time Interval

做~自己de王妃 提交于 2020-01-16 19:16:31
问题 I am working huge amount of data which I have recently collected for my research. I would like to get to know what time of the day user activity happened. Here is the small amount of my dataset snapshot: I applied to following code to get to know time interval login activity. =SUMPRODUCT(--(HOUR(I2:I185143)>=K4*24),--(HOUR(I2:I185143)<L4*24)) The result is written in the Column N. Everything so far, works I believe. However my question is what if I want to analyze in every 10 min rather than

Duplicates removal using Group By, Rank, Row_Number

丶灬走出姿态 提交于 2020-01-16 09:49:28
问题 I have two tables. One is CustomerOrders and the other is OrderCustomerRef - lookup table. Both tables have one-to-many relationship - one customer may be associated with multiple orders. CustomerOrders table has duplicate Customers (same LName, FName, Email). But they have different Cust_IDs. I need to merge all duplicate contacts in the base Customer table (one-to-one). (this table is not shown here). Step 1: Need to find out which Cust_ID should be merged into which corresponding duplicate

How to find minimum for each unique value for every column grouped by ID in pandas data frame

纵饮孤独 提交于 2020-01-16 08:08:10
问题 I have a pandas dataframe and would like to find minimum value for each column grouped by ID. #Input data df=pd.DataFrame({ 'id':[1,1,1,1,2,2,2,2], 'a':range(8), 'b':range(8,0,-1) }) #expected output is the minimum value for each id and column (a, b) id a b 1 0 5 2 4 1 回答1: df.groupby('id', as_index=False).agg(min) will do just that. id a b 1 0 5 2 4 1 回答2: You can use df.groupby('id').min() Result: a b id 1 0 5 2 4 1 来源: https://stackoverflow.com/questions/59217144/how-to-find-minimum-for