Restarting process on -XX:OnOutOfMemoryError using shell script gives: Unrecognized option: -9

问题

I'm trying to restart process when OOME happens. Java binary is launched using two shell scripts, one of them imports other. I don't have any control of the first one but can modify the second one as I want. This is a prototype what I'm trying to do:

First shell script test.sh:

#!/bin/sh
JAVA_OPTS="$JAVA_OPTS -Xmx10m"
. test1.sh

echo $JAVA_OPTS
java $JAVA_OPTS $es_params TestMemory

Second shell script test1.sh:

#!/bin/sh

pidfile="test.pid"
touch $pidfile
params="$parms -Dpidfile=$pidfile"

kill_command="kill -9 \$(cat $pidfile)"

dir=$( cd $(dirname $0) ; pwd -P )
path="$dir/$(basename $0)"

start_command="$path $@"

restart_command="$kill_command;sleep 2;$start_command"

JAVA_OPTS="$JAVA_OPTS -XX:OnOutOfMemoryError=\"$restart_command\""

Generally what it does is JAVA_OPTS is constructed inside test1.sh and then used to run Java binary, which just writes PID in pidfile and then creates OOME.

Problem happens during execution, java can't understand what is a parameter and what is a class to run. I think it might be a problem of quoting, I tried different ways to escape JAVA_OPTS, but without any result. I'm either getting:

Unrecognized option: -9
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Error: Could not find or load main class "-XX:OnOutOfMemoryError=kill

If I just take a value of JAVA_OPTS and put it manually in test.sh it runs perfectly. Any ideas how can I change test1.sh to make it work? I think I tried almost every possible way of putting double and single quotes, but without any success. Also if I put restart_command in restart.sh file and use it instead of the variable, it works fine.

After running set -x I saw that shell modifies every single space character to ' ' - adds ' on both sides. Escaping doesn't gives any result. Any idea how to avoid this? So final commend is:

+ java -Xmx10m '"-XX:OnOutOfMemoryError=kill' '$(cat' 'test.pid);sleep' '2;/Users/davidt/test/TestMemory/bin/test.sh' '")' -Des.pidfile=test.pid TestMemory

Update

I can run simplified command successfully

java "-XX:OnOutOfMemoryError=echo 'Ups'" $es_params TestMemory

But it seems a general problem, shell just hates spaces into variables I guess:

JAVA_OPTS="\"-XX:OnOutOfMemoryError=echo 'Ups'\""
set -x
java $JAVA_OPTS TestMemory

This script fails and the last line is interpreted as:

java '"-XX:OnOutOfMemoryError=echo' ''\''Ups'\''"' TestMemory

I tried different options to escape

回答1:

This is a shell problem. Based on the evidence, I'd say that one of the ; characters ... and possibly some why space ... is being interpretted by the shell when you don't want / need this to happen.

If you run set -x in the shell before running the command that is trying to start the JVM, you will see the actual command that is being used.

It seems shell translates every single space to ' ',

Not exactly. The single quotes are inserted by the shell into the output you are getting from set -x. They simply indicating where the argument boundaries are. They are not really there ... and they are certainly NOT being passed to the java command.

Any idea how to [a]void it?

What you need to do is start from the (final) command that you are trying execute ...

java -Xmx10m -XX:OnOutOfMemoryError="kill NNNN;sleep 2;/Users/davidt/test/TestMemory/bin/test.sh" -Des.pidfile=test.pid TestMemory

... and work backwards, so that the shell variables, expansions and escaping give you what you need.

The other thing to note is that this:

java -Xmx10m -XX:OnOutOfMemoryError="kill $(cat test.pid); ..."

probably won't work. The kill $(cat test.pid) command is using shell syntax and requires shell functionality to interpolate the contents of the PID file. I doubt that the JVM is going to know what to do with that. (Or more accurately. It will do what you have literally told it to do, but that will not be what you want ...)

If you really need to interpolate the pid file content when the restart command is run as you appear to be trying to do, then suggest that turn the restart command into a free-standing shell script, and set the file mode so that it is executable. It will be simpler and a lot easier to get working.

As a general piece of advice, is is a bad idea to be too clever with shell scripts. The exact semantics of variable expansion and command parsing are rather tricky, and it is easy to get yourself really confused ... if you are trying to do this at multiple levels.

回答2:

I ended up put the script I wanted to execute in a separate file and gave it as a parameter to JVM to execute when OOME happens. echo "echo 'UPS'" >> oome_happened.sh JAVA_OPTS="\"-XX:OnOutOfMemoryError='oome_happened.sh'\"" set -x java $JAVA_OPTS TestMemory