Job queue as SQL table with multiple consumers (PostgreSQL)

后端 未结 7 838
醉梦人生
醉梦人生 2020-12-04 09:34

I have a typical producer-consumer problem:

Multiple producer applications write job requests to a job-table on a PostgreSQL database.

The job requests have

7条回答
  •  萌比男神i
    2020-12-04 10:30

    Okay, here is the solution that is working for me, based on the link from jordani. As some of my problems were in the way Qt-SQL works, I've included the Qt code:

    QSqlDatabase db = GetDatabase();
    db.transaction();
    QSqlQuery lockQuery(db);
    bool lockResult = lockQuery.exec("LOCK TABLE serverjobrecord IN ACCESS EXCLUSIVE MODE; ");
    QSqlQuery query(db);
    query.prepare(    
    "UPDATE jobrecord "
    "  SET \"owner\"= :owner, state = :reserved "
    "  WHERE id = ( "
    "    SELECT id from jobrecord WHERE state = :queued ORDER BY id LIMIT 1 "
    "  ) RETURNING id;"
    );
    query.bindValue(":owner", pid);
    query.bindValue(":reserved", JobRESERVED);
    query.bindValue(":queued", JobQUEUED); 
    bool result = query.exec();
    

    To check, if multiple consumers process the same job, I added a rule and a log-table:

    CREATE TABLE serverjobrecord_log
    (
      serverjobrecord_id integer,
      oldowner text,
      newowner text
    ) WITH ( OIDS=FALSE );
    
    
    CREATE OR REPLACE RULE ownerrule AS ON UPDATE TO jobrecord
    WHERE old.owner IS NOT NULL AND new.state = 1 
    DO INSERT INTO jobrecord_log     (id, oldowner, newowner) 
        VALUES (new.id, old.owner, new.owner);
    

    Without the LOCK TABLE serverjobrecord IN ACCESS EXCLUSIVE MODE; statement, the log-table fills occasionaly with entries, were one consumers has overwritten the values of another, but using the LOCK statement, the log-table remains empty :-)

提交回复
热议问题