Ignoring locked row in a MySQL query

Question

I have one table that is read at the same time by different threads.

Each thread must select 100 rows, execute some tasks on each row (unrelated to the database) then they must delete the selected row from the table.

rows are selected using this query:

SELECT id FROM table_name FOR UPDATE;

My question is: How can I ignore (or skip) rows that were previously locked using a select statement in MySQL ?

Answer 1

I typically create a process_id column that is default NULL and then have each thread use a unique identifier to do the following:

UPDATE table_name SET process_id = #{process.id} WHERE process_id IS NULL LIMIT 100;

SELECT id FROM table_name WHERE process_id = #{process.id} FOR UPDATE;

That ensures that each thread selects a unique set of rows from the table.

Hope this helps.

Answer 2

Even though it is not the best solution, as there is no way that I know to ignore locked rows, I select a random one and try to obtain a lock.

START TRANSACTION;
SET @v1 =(SELECT myId FROM tests.table WHERE status is NULL LIMIT 1);
SELECT * FROM tests.table WHERE myId=@v1 FOR UPDATE; #<- lock

Setting a small timeout for the transaction, if that row is locked the transaction is aborted and I try another one. If I obtain the lock, I process it. If (bad luck) that row was locked, it is processed and the lock is released before my timeout, I then select a row that has already been 'processed'! However, I check a field that my processes set (e.g. status): if the other process transaction ended OK, that field tells me that work has already been done and I do not process that row again.

Every other possible solution without transactions (e.g. setting another field if the row has no status and ... etc.) can easily provide race conditions and missed processes (e.g. one thread abruptly dies, the allocated data is still tagged, while a transaction expires; ref. comment here

Hope it helps