Filling Gaps in Dates Returned from Database - pure SQL solution possible?

Question

I have this query:

SELECT COUNT(*) as clicks, DATE_FORMAT(FROM_UNIXTIME(click_date), \'%w %M %Y\') as point 
FROM tracking 
WHERE click_date < $end_date A         


        

Answer 1

I'd stick to PHP logic, looping between the lowest date and the highest date, and incrementing one day at a time.

You could probably do it in SQL with some fancy joins, but I won't even start to consider that nastiness!

BTW, %w is the day of the week, starting 0=Sunday. You probably wanted %d.

Answer 2

In my opinion you are better off doing this type of logic in your code. But if you wanted to do it in pure SQL you could construct a query to give you the results of all the days between one day and the next ... either by inserting into a temp table or an in memory table... then left join that into your results so that you get all the days regardless of if there were results for that day.

Answer 3

Can this possibly be done via a pure SQL query or do I have to use some php logic?

Yes, it is better to create a Numbers table (single column N) that contains nothing but the numbers 0 to 999. It can be used for many things, not least a query like the below:

SELECT COUNT(t.click_date) as clicks,
    DATE_FORMAT(affffdate($start_date, interval N day), '%d %M %Y') as point 
FROM Numbers
LEFT JOIN tracking t
    ON t.click_date >= affffdate($start_date, interval N day)
    and t.click_date < affffdate($start_date, interval (N+1) day)
WHERE N between 0 and datediff($start_date, $end_date)
GROUP BY N

Btw, why do I have 0 February 2011 as my first date

You're using the wrong format. It's UPPER case W not lower for day-of-week, so '%W %M %Y' or '%d %M %Y' for day-of-month. http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_date-format

maybe my GROUP BY isn't working correctly?

You are using GROUP BY DAY(FROM_UNIXTIME(click_date)) note "day" not weekday, but you are displaying (or trying to) "%W" (weekday) - pick one, don't mix them.

---

EDIT: If you prefer not to materialize (create as a real table) a Numbers sequence table, you can construct one on the fly. It won't be pretty.

Note: N1, N2 and N3 below combine to give a possible range of 0-999

SELECT COUNT(t.click_date) as clicks,
    DATE_FORMAT(affffdate($start_date, interval N day), '%d %M %Y') as point 
FROM (
    select N1 * 100 + N2 * 10 + N3 as N
    from (
    select 0 N1 union all select 1 union all select 2 union all
    select 3 union all select 4 union all select 5 union all
    select 6 union all select 7 union all
    select 8 union all select 9) N1
    cross join (
    select 0 N2 union all select 1 union all select 2 union all
    select 3 union all select 4 union all select 5 union all
    select 6 union all select 7 union all
    select 8 union all select 9) N2
    cross join (
    select 0 N3 union all select 1 union all select 2 union all
    select 3 union all select 4 union all select 5 union all
    select 6 union all select 7 union all
    select 8 union all select 9) N3
    ) Numbers
LEFT JOIN tracking t
    ON t.click_date >= affffdate($start_date, interval N day)
    and t.click_date < affffdate($start_date, interval (N+1) day)
WHERE N between 0 and datediff($start_date, $end_date)
GROUP BY N

EDIT #2: A straight Dates table

Put this in a new window in phpMyAdmin or run it as a batch. It creates a table named Dates, with every single date from day 1900-01-01 (or change in the script) to 2300-01-01 (or change).

DROP PROCEDURE IF EXISTS FillDateTable;

delimiter //
CREATE PROCEDURE FillDateTable()
    LANGUAGE SQL
    NOT DETERMINISTIC
    CONTAINS SQL
    SQL SECURITY DEFINER
    COMMENT ''
BEGIN
  drop table if exists datetable;
  create table datetable (thedate datetime primary key, isweekday smallint);

  SET @x := date('1900-01-01');
  REPEAT 
    insert into datetable (thedate, isweekday) SELECT @x, case when dayofweek(@x) in (1,7) then 0 else 1 end;
    SET @x := date_add(@x, interval 1 day);
    UNTIL @x > date('2300-01-01') END REPEAT;
END//
delimiter ;

CALL FillDateTable;

With such a utility table, your query can be just

SELECT COUNT(t.click_date) as clicks,
    DATE_FORMAT(thedate, '%d %M %Y') as point 
FROM Dates
LEFT JOIN tracking t
    ON t.click_date >= thedate
    and t.click_date < affffdate(thedate, interval 1 day)
WHERE thedate between $start_date and $end_date
GROUP BY thedate