Optimize MySQL UPDATE query that contains WHERE and ORDER BY?

问题

How can I optimize this query? If I run it without the ORDER BY clause, it executes in <100ms. With the ORDER BY clause it takes many seconds, and crushes the server when more than one system is trying to make this query at once.

UPDATE companies
SET
    crawling = 1
WHERE
    crawling = 0
    AND url_host IS NOT NULL
ORDER BY
    last_crawled ASC
LIMIT 1;

If I run this query as a SELECT, it's also fast ( <100ms ).

SELECT id
FROM companies
WHERE
    crawling = 0
    AND url_host IS NOT NULL
ORDER BY
    last_crawled ASC
LIMIT 1;

Here is my table schema:

CREATE TABLE `companies` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `url` varchar(255) DEFAULT NULL,
  `url_scheme` varchar(10) DEFAULT NULL,
  `url_host` varchar(255) DEFAULT NULL,
  `name` varchar(255) DEFAULT NULL,
  `crawl` tinyint(1) unsigned NOT NULL DEFAULT '1',
  `crawling` tinyint(1) unsigned NOT NULL DEFAULT '0',
  `last_crawled` datetime NOT NULL,
  PRIMARY KEY (`id`),
  KEY `name` (`name`),
  KEY `url_host` (`url_host`),
  KEY `crawl` (`crawl`),
  KEY `crawling` (`crawling`),
  KEY `last_crawled` (`last_crawled`),
  KEY `url_scheme` (`url_scheme`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

UPDATE ONE

This query gives me the following error: You can't specify target table 'companies' for update in FROM clause

UPDATE companies
SET crawling = 1
WHERE id = (
    SELECT id
    FROM companies
    WHERE
        crawling = 0
        AND url_host IS NOT NULL
    ORDER BY
        last_crawled ASC
    LIMIT 1
);

This query gives me the following error: This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery'

UPDATE companies
SET crawling = 1
WHERE id in (
    SELECT id
    FROM companies
    WHERE
        crawling = 0
        AND url_host IS NOT NULL
    ORDER BY
        last_crawled ASC
    LIMIT 1
);

回答1:

try not to use ORDER-BY and LIMIT for such small number of updates.

    UPDATE companies t1
    join
    (
        SELECT c.id,@RowNum:=@RowNum+1 AS RowID
        FROM companies c, (SELECT @RowNum := 0)r
        WHERE c.crawling = 0 AND c.url_host IS NOT NULL
        ORDER BY c.last_crawled ASC
    )t2
    ON t2.RowID=1 AND t1.id=t2.id
    SET t1.crawling = 1

EDIT:1

make sure you have the index on (last_crawled ASC , id ASC)

    UPDATE companies t1
    join
    (
        Select ID,RowID
        From
        (
            SELECT c.id,@RowNum:=@RowNum+1 AS RowID
            FROM companies c, (SELECT @RowNum := 0)r
            WHERE c.crawling = 0 AND c.url_host IS NOT NULL
            ORDER BY c.last_crawled ASC
        )t2
        WHERE ROWID=1
    )t3
    ON t1.id=t3.id
    SET t1.crawling = 1

来源：https://stackoverflow.com/questions/19940797/optimize-mysql-update-query-that-contains-where-and-order-by

标签

mysql

indexing

query-optimization