Aggregating data by date in a date range without date gaps in result set

Question

I have a table with sell orders and I want to list the COUNT of sell orders per day, between two dates, without leaving date gaps.

This is what I have

Answer 1

Creating a range of dates on the fly and joining that against you orders table:-

SELECT sub1.sdate, COUNT(ORDERS.id) as Norders
FROM
(
    SELECT DATE_FORMAT(DATE_SUB(NOW(), INTERVAL units.i + tens.i * 10 + hundreds.i * 100 DAY), "%M %e") as sdate 
    FROM (SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9)units
    CROSS JOIN (SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9)tens
    CROSS JOIN (SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9)hundreds
    WHERE DATE_SUB(NOW(), INTERVAL units.i + tens.i * 10 + hundreds.i * 100 DAY) BETWEEN DATE_SUB(NOW(), INTERVAL 1 MONTH) AND NOW()
) sub1
LEFT OUTER JOIN ORDERS
ON sub1.sdate = DATE_FORMAT(ORDERS.date, "%M %e")
GROUP BY sub1.sdate

This copes with date ranges of up to 1000 days.

Note that it could be made more efficient easily depending on the type of field you are using for your dates.

EDIT - as requested, to get the count of orders per month:-

SELECT aMonth, COUNT(ORDERS.id) as Norders
FROM
(
    SELECT DATE_FORMAT(DATE_SUB(NOW(), INTERVAL months.i MONTH), "%Y%m") as sdate, DATE_FORMAT(DATE_SUB(NOW(), INTERVAL months.i MONTH), "%M") as aMonth 
    FROM (SELECT 0 i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9 UNION SELECT 10 UNION SELECT 11)months
    WHERE DATE_SUB(NOW(), INTERVAL months.i MONTH) BETWEEN DATE_SUB(NOW(), INTERVAL 12 MONTH) AND NOW()
) sub1
LEFT OUTER JOIN ORDERS
ON sub1.sdate = DATE_FORMAT(ORDERS.date, "%Y%m")
GROUP BY aMonth

Answer 2

You are going to need to generate a virtual (or physical) table, containing every date in the range.

That can be done as follows, using a sequence table.

SELECT mintime + INTERVAL seq.seq DAY AS orderdate
  FROM (
        SELECT CURDATE() - INTERVAL 1 MONTH AS mintime,
               CURDATE() AS maxtime
          FROM obs
       ) AS minmax
  JOIN seq_0_to_999999 AS seq ON seq.seq < TIMESTAMPDIFF(DAY,mintime,maxtime)

Then, you join this virtual table to your query, as follows.

SELECT IFNULL(orders.Norders,0) AS Norders, /* show zero instead of null*/
       DATE_FORMAT(alldates.orderdate, "%M %e") as sdate 
  FROM (
        SELECT mintime + INTERVAL seq.seq DAY AS orderdate
          FROM (
                SELECT CURDATE() - INTERVAL 1 MONTH AS mintime,
                       CURDATE() AS maxtime
                  FROM obs
               ) AS minmax
          JOIN seq_0_to_999999 AS seq 
                        ON seq.seq < TIMESTAMPDIFF(DAY,mintime,maxtime)
       ) AS alldates
  LEFT JOIN (
    SELECT COUNT(*) as Norders, DATE(date) AS orderdate
      FROM ORDERS 
    WHERE date <= NOW() 
      AND date >= NOW() - INTERVAL 1 MONTH 
    GROUP BY DAY(date) 
       ) AS orders ON alldates.orderdate = orders.orderdate
ORDER BY alldates.orderdate ASC

Notice that you need the LEFT JOIN so the rows in your output result set will be preserved even if there's no data in your ORDERS table.

Where do you get this sequence table seq_0_to_999999? You can make it like this.

DROP TABLE IF EXISTS seq_0_to_9;
CREATE TABLE seq_0_to_9 AS
   SELECT 0 AS seq UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4
    UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9;

DROP VIEW IF EXISTS seq_0_to_999;
CREATE VIEW seq_0_to_999 AS (
SELECT (a.seq + 10 * (b.seq + 10 * c.seq)) AS seq
  FROM seq_0_to_9 a
  JOIN seq_0_to_9 b
  JOIN seq_0_to_9 c
);

DROP VIEW IF EXISTS seq_0_to_999999;
CREATE VIEW seq_0_to_999999 AS (
SELECT (a.seq + (1000 * b.seq)) AS seq
  FROM seq_0_to_999 a
  JOIN seq_0_to_999 b
);

You can find an explanation of all this in more detail at http://www.plumislandmedia.net/mysql/filling-missing-data-sequences-cardinal-integers/

If you're using MariaDB version 10+, these sequence tables are built in.

Answer 3

First create a Calendar Table

SELECT coalesce(COUNT(O.*),0) as Norders, DATE_FORMAT(C.date, "%M %e") as sdate 
FROM Calendar C 
  LEFT JOIN ORDERS O ON C.date=O.date
WHERE O.date <= NOW() AND O.date >= NOW() - INTERVAL 1 MONTH 
GROUP BY DAY(date) 
ORDER BY date ASC;