Track failure of command on a Salt minion

问题

I am using salt for last one month. Whenever I run a command say sudo salt '*' test.ping, then the master pings all the minions and the response being the list of all the minions which are up and running. Output looks something like:

{
"minion_1": true
}
{
"minion_2": true
}
{
"minion_3": true
}

In the master's conf file, return type is configured to JSON. But if I execute an incorrect command through salt master say sudo salt '*' test1.ping, then the master returns something like this

{
"minion_1": "'test1.ping' is not available."
}
{
"minion_2": "'test1.ping' is not available."
}
{
"minion_3": "'test1.ping' is not available."
}

In both the outputs displayed above, the command has given a success exit code on the master's shell/terminal. How do we track which minions were not able to execute the commands. I am not interested in what type of error it is, I just need some or the other way to track the minions which failed to execute the command.

The last solution is to write a parser which will read the complete output and decide for itself. Hope that there is a better solution.

回答1:

Reasons to despair

I would not rely on Salt's CLI exit code at the moment (version 2014.7.5) - there are still many issues opened to solve this.

Get valid JSON output

There is --static option which fixes JSON output:

If using --out=json, you will probably want --static as well. Without the static option, you will get a JSON string for each minion.

Otherwise the output given by Salt above contains multiple objects (one per minion) which is not a valid JSON (JSON requires single object, array or value per document) and simple way of loading entire output by a standard JSON parser will fail. It is even mentioned in documentation (as of 5188d6c):

Some JSON parsers can guess when an object ends and a new one begins but many can not.

In addition to that, some Salt options (like show_jid) also send strings to STDOUT which mixes it with execution report and invalidates JSON output format. Option --static also solves this problem.

UPDATE: Parser to detect failure in Salt execution

This problem squeezed me so much so I gave quick birth to this Python script @ 75e42af with example how it is used @ b819961d.

NOTE: This won't address output of arbitrary Salt command (including test.ping above), but issues related to the output of state execution are covered. There is still a solution to test.ping problem above - it can be run from state, then the output can be analysed by the script. See how to call an execution module from within a state or *.sls file in this answer.

Features (details in the code itself):

Handle output from both highstate and orchestrate runners.
Handle output of multiple minions and any number of commands.
Report summary "? of N" and overall result.
Standalone file usable as script and module.

The only limitation is that it requires JSON output (Salt option --out json) simply because it is easy to fix the discussed issues before feeding it to parser.

回答2:

The above parser will only work for test.ping command. If multiple commands have to be executed we will have to write a more robust parser.

来源：https://stackoverflow.com/questions/29960947/track-failure-of-command-on-a-salt-minion

标签

salt-stack