Unable to create MarkLogic scheduled tasks from within CPF action module

Question

I have a MarkLogic database with Content Processing Framework (CPF) installed and the CPF pipeline is such that:

1. Whenever a document is inserted then it grabs the value of execution-date from the document and schedule a task for that time.

Example:

Sample document:

<sample>
  <execution-date>2014-10-20T12:29:10</execution-date>
</sample>

when inserted triggers the CPF action module which reads the value of execution-date field and creates a scheduled task to be executed on the time read from execution-date field.

Following is the XQuery code snippet from the CPF action module that creates the scheduled task:

let $doc := fn:doc( $cpf:document-uri )
let $releasedon := xs:string($doc/sample/execution-date/text())

let $config := admin:get-configuration()
let $group := admin:group-get-id($config, "Default")

let $new-task :=
     admin:group-one-time-scheduled-task(
        "/tasks/task.xqy",
        "/",
        xs:dateTime($releasedon),
        xdmp:database("SampleDB"),
        xdmp:database("Modules"),
        xdmp:user("admin"), 
        (),
        "normal")

let $addTask := admin:group-add-scheduled-task($config,$group, $new-task)

return

    admin:save-configuration($addTask),
    xdmp:log(fn:concat("Task for document Uri: ", $cpf:document-uri, " created"))

Now, when I insert single document then everything works as expected, that is:

1. Document inserted successfully
2. the CPF action module is triggered successfully
3. Scheduled task created successfully.

But, when I try to insert multiple documents using:

xdmp:document-insert("/1.xml", 
    <sample>
      <execution-date>2014-10-21T10:00:00</execution-date>
    </sample>, 
    xdmp:default-permissions(),
    ("documents"))
,
xdmp:document-insert("/2.xml", 
    <sample>
      <execution-date>2014-10-20T11:00:00</execution-date>
    </sample>, 
    xdmp:default-permissions(),
    ("documents"))

CPF action module gets triggered successfully (log message can be seen in logs) BUT ONLY one scheduled task gets created.

When looking in MarkLogic Admin Interface I can only find a single scheduled task which is scheduled to run at 2014-10-20T11:00:00

Please let me know what am I doing wrong or is there any configuration I am missing. Any suggestions are welcomed.

Thanks!

Answer 1

The fundamental issue here is that the admin configuration manipulation APIs are not transactionally protected operations, so when you run two in parallel each one sees the initial state of the configuration files, then writes their bit to add the scheduled task, and then saves it, and only one of them wins. You can force this to behave in a transactionally protected way by forcing a lock on some URI It doesn't matter what it is. It doesn't even have to be in the database. As long as everything that is doing this is locking on the same URI you are fine. xdmp:lock-for-update("my.example.uri") will do this.

Answer 2

The following CPF action module is now working as expected:

xquery version "1.0-ml";
import module namespace cpf = "http://marklogic.com/cpf" at "/MarkLogic/cpf/cpf.xqy";
import module namespace admin = "http://marklogic.com/xdmp/admin" at "/MarkLogic/admin.xqy";

declare variable $cpf:document-uri as xs:string external;
declare variable $cpf:transition as node() external;

declare function local:scheduleTask()
{

xdmp:lock-for-update("/sample.xml"),
if (cpf:check-transition($cpf:document-uri,$cpf:transition)) then try
{

    let $doc := fn:doc( $cpf:document-uri )
    let $releasedon := xs:string($doc/sample/execution-date/text())

    let $config := admin:get-configuration()
    let $group := admin:group-get-id($config, "Default")

    let $new-task :=
         admin:group-one-time-scheduled-task(
            "/tasks/task.xqy",
            "/",
            xs:dateTime($releasedon),
            xdmp:database("SampleDB"),
            xdmp:database("Modules"),
            xdmp:user("admin"), 
            (),
            "normal")

    let $addTask := admin:group-add-scheduled-task($config,$group, $new-task)

    return

        admin:save-configuration($addTask),
        xdmp:log(fn:concat("Task for document Uri: ", $cpf:document-uri, " created")) 

}
catch ($e) {
    cpf:failure( $cpf:document-uri, $cpf:transition, $e, () )
}
else ( )
};

local:scheduleTask()