Rundeck - Fail a job only when all nodes fail

北城以北 提交于 2021-02-11 14:19:49

问题


Is it possible to have an overall job status set to OK if at least one node reports ok? Currently my job runs tasks on docker and will only run succeed on the leader and will fail on the others. I would like to have it so the job is OK so long as it has run successfully on at least one node. Is this possible?


回答1:


You can do that in this way:

First, you need two jobs. The first one: points to your nodes, but this job needs an option to pass the node name in the node filter textbox. And the second one: to call the first one via API with an inline script.

You just need to check the second job.

First job:

    <joblist>
      <job>
        <context>
          <options preserveOrder='true'>
            <option name='thenode' />
          </options>
        </context>
        <defaultTab>nodes</defaultTab>
        <description></description>
        <dispatch>
          <excludePrecedence>true</excludePrecedence>
          <keepgoing>false</keepgoing>
          <rankOrder>ascending</rankOrder>
          <successOnEmptyNodeFilter>false</successOnEmptyNodeFilter>
          <threadcount>1</threadcount>
        </dispatch>
        <executionEnabled>true</executionEnabled>
        <id>9fcc183b-b4df-4554-b75b-c660eda706b3</id>
        <loglevel>INFO</loglevel>
        <name>TestFWISE</name>
        <nodeFilterEditable>false</nodeFilterEditable>
        <nodefilters>
          <filter>${option.thenode}</filter>
        </nodefilters>
        <nodesSelectedByDefault>true</nodesSelectedByDefault>
        <scheduleEnabled>true</scheduleEnabled>
        <sequence keepgoing='true' strategy='node-first'>
          <command>
            <exec>sh hello.sh</exec>
          </command>
        </sequence>
        <uuid>9fcc183b-b4df-4554-b75b-c660eda706b3</uuid>
      </job>
    </joblist>

Second job:

    <joblist>
      <job>
        <defaultTab>nodes</defaultTab>
        <description></description>
        <executionEnabled>true</executionEnabled>
        <id>18bbd45e-5301-4498-8b92-0c4828194b61</id>
        <loglevel>INFO</loglevel>
        <name>Runner</name>
        <nodeFilterEditable>false</nodeFilterEditable>
        <scheduleEnabled>true</scheduleEnabled>
        <sequence keepgoing='false' strategy='node-first'>
          <command>
            <fileExtension>sh</fileExtension>
            <script><![CDATA[#!/bin/bash

    # script that detect only when all nodes fail
    # https://stackoverflow.com/questions/58798856/rundeck-fail-a-job-only-when-all-nodes-fail

    #####################################################
    # rundeck instance values
    server="localhost"
    port="4440"
    api="32"
    jobid="9fcc183b-b4df-4554-b75b-c660eda706b3"
    token="fb4ra5Rc1d71rOYhFxXK9a1vtMXtwVZ1"

    #####################################################
    # 1 - succeeded at least in one node
    # 0 - failed in all nodes
    #####################################################
    flag="0"

    #####################################################
    # here put all nodes to pass via options, like a list
    mynodes='node01'

    #####################################################
    # "prudential" time between actions
    pt="2"

    #####################################################
    # 1) run the job via API and store the execution ID.
    # 2) takes the execution ID and store job status via API
    # 3) with job status decides if runs at least on one node or not.
    for currentnode in $mynodes
        do
            sleep $pt

            execid=$(curl -H accept:application/json --data-urlencode "argString=-thenode $currentnode" http://$server:$port/api/$api/job/$jobid/run?authtoken=$token | jq -r '.id')

            # only for debug
            echo "execution id: $execid"

            sleep  $pt

            status=$(curl --location --request GET "http://$server:$port/api/$api/execution/$execid/state?authtoken=$token" | jq -r '.executionState')

            # only for debug
            echo "status is: $status"

            # if job runs OK, then assign the value 1 to flag
            if [ "$status" = "SUCCEEDED" ]
            then
                flag="1"
            fi
    done

    sleep  $pt

    # only for debug
    echo "flag value: $flag"

    #####################################################
    # now is time to check the flag
    if [ "$flag" -eq "1" ]
        then
            echo "the job has been succeeded at least in one node."
            exit 0 # rundeck job ends normally :3
        else
            echo "the job has been failed in all nodes."
            exit 1 # rundeck job ends with error :(
    fi]]></script>
            <scriptargs />
            <scriptinterpreter>/bin/bash</scriptinterpreter>
          </command>
        </sequence>
        <uuid>18bbd45e-5301-4498-8b92-0c4828194b61</uuid>
      </job>
    </joblist>

The second job fails only if your first job fails in all nodes. If the first job runs at least in one node the job shows "OK".

If you need the script separated I leave it here:

    #!/bin/bash

    # script that detects only when all nodes fail
    # https://stackoverflow.com/questions/58798856/rundeck-fail-a-job-only-when-all-nodes-fail

    #####################################################
    # rundeck instance values
    server="localhost"
    port="4440"
    api="32"
    jobid="9fcc183b-b4df-4554-b75b-c660eda706b3"
    token="fb4ra5Rc1d71rOYhFxXK9a1vtMXtwVZ1"

    #####################################################
    # 1 - succeeded at least in one node
    # 0 - failed in all nodes
    #####################################################
    flag="0"

    #####################################################
    # here put all nodes to pass via options, like a list
    mynodes='node00 node01'

    #####################################################
    # "prudential" time between actions
    pt="2"

    #####################################################
    # 1) run the job via API and store the execution ID.
    # 2) takes the execution ID and store job status via API
    # 3) with job status decides if runs at least on one node or not.
    for currentnode in $mynodes
        do
            sleep $pt

            execid=$(curl -H accept:application/json --data-urlencode "argString=-thenode $currentnode" http://$server:$port/api/$api/job/$jobid/run?authtoken=$token | jq -r '.id')

            # only for debug
            echo "execution id: $execid"

            sleep  $pt

            status=$(curl --location --request GET "http://$server:$port/api/$api/execution/$execid/state?authtoken=$token" | jq -r '.executionState')

            # only for debug
            echo "status is: $status"

            # if job runs OK, then assign the value 1 to flag
            if [ "$status" = "SUCCEEDED" ]
            then
                flag="1"
            fi
    done

    sleep  $pt

    # only for debug
    echo "flag value: $flag"

    #####################################################
    # now is time to check the flag
    if [ "$flag" -eq "1" ]
        then
            echo "the job has been succeeded at least in one node."
            exit 0 # rundeck job ends normally :3
        else
            echo "the job has been failed in all nodes."
            exit 1 # rundeck job ends with error :(
    fi

NOTE: The script needs JQ to work.



来源:https://stackoverflow.com/questions/58798856/rundeck-fail-a-job-only-when-all-nodes-fail

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!