Grouping records hour by hour or day by day and filling gaps with zero or null in mysql

Question

I have written a query that counts records hour by hour:

select TO_CHAR(copied_timestamp, \'YYYY-MM-DD HH24\'),count(*) from req group by
TO_CHAR(copied_time         


        

Answer 1

Generate a single-column dates_hours table which contains all dates and hours within a reasonable range (e.g. from 1900 to 2200). Then do a LEFT JOIN from this table to your current query.

For this technique to perform correctly, you will probably need to add an indexed column to your table which contains a converted time stamp (your copied_timestamp converted to DATETIME, rounded to the hour)

SELECT date_hour, count(req.converted_timestamp)
FROM
    dates_hours 
    LEFT JOIN req ON req.converted_timestamp = dates_hours.date_hour
WHERE date_hour
    BETWEEN (SELECT MIN(req.converted_timestamp) FROM req)
    AND (SELECT MAX(req.converted_timestamp) FROM req)
GROUP BY date_hour

To generate the dates_hours table:

CREATE TABLE dates_hours (date_hour DATETIME PRIMARY KEY);

DELIMITER $$$
CREATE PROCEDURE generate_dates_hours (to_date DATETIME)
BEGIN

    DECLARE start_date DATETIME;
    DECLARE inc INT;

    SELECT MAX(date_hour) INTO start_date FROM dates_hours;
    IF start_date IS NULL THEN
        SET start_date = '1900-01-01';
    END IF;
    SET inc = 1;
    WHILE start_date + INTERVAL inc HOUR  <= to_date DO
        INSERT INTO dates_hours VALUE (start_date + INTERVAL inc HOUR);
        SET inc = inc +1;
    END WHILE;

END $$$
DELIMITER ;

CALL generate_dates_hours('2200-01-01');

Okay, now that I am proof-reading myself, I realise this is quite far-fetched a solution. I hope someone comes up with a more elegant one.

Answer 2

I created a table called TBL_NUMBERS

CREATE TABLE `TBL_NUMBER` (`n` int(11) NOT NULL)

and inserted records from 1 to 1000. Now I can generate any kind of date range using this query:

SELECT '2012-06-21' + INTERVAL n-1 [DAY | HOUR | MINUTE] or as dateRange
 FROM TBL_NUMBER
WHERE '2012-06-21' + INTERVAL n-1 [DAY | HOUR | MINUTE] <= '2012-06-23';

Then I can join this table with my results to fill the date gap. If i need more than 1000 date range I can insert more records in TBL_NUMBER

If you have any better idea, I'm eager to know that ;)