I have a Maven project, which builds 6 separate Maven projects in Jenkins. The problem I face is that over the time the project build fails giving the 137 error code:
<I ran into the same error code. This error code does indeed seem related to JVM resource constraints in the Jenkins environment. I would suggest re-running the build a second time after the error occurs to see if you can get additional/different output, though this will certainly depend on which part of the build caused the resource issues (in my cause it was a Maven downloads).
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000fb7cb000, 7331840, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 7331840 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /vagrant/args4java/hs_err_pid10470.log
ERROR: Maven JVM terminated unexpectedly with exit code 1
Finished: FAILURE
I was running into the same behavior on our build server. The error is IMHO not associated with the maven memory settings (i.e. MAVEN_OPTS
) but rather with the memory of the underlying (Linux) machine itself (which Jenkins runs on).
The (rejected) Jenkins issue https://jenkins-ci.org/issue/12035 gives more detail on this matter:
For reference the status code 137 (128 + 9) typically means (can differ between flavours of unix). That the process was terminated by receipt of a signal. In this case signal 9 which is SIGKILL and unblockable kill.
If this is the case the underlying machine/OS needs more virtual memory. This can be added by either adding physical memory or swap space as appropriate.
You should try to increase the virtual memory of your machine.
Note:
This also explains why a Jenkins restart (temporarily) fixes the issue.
When running Maven via Jenkins I got this error:
ERROR: Maven JVM terminated unexpectedly with exit code 137
I accidently put an "and" in the MAVEN_OPTS parameter:
-Xmx1024m and -Xms1024m
Then, I got this error:
Error: Could not find or load main class and
ERROR: Failed to launch Maven. Exit code - 1
After, removing the 'and', I reran Jenkins and received this error:
java.lang.OutOfMemoryError: Java heap space
Finally, I increased the memory using Global MAVEN_OPTS:
-Xmx4096m -Xms4096m
This fixed the problem. So, this seems to be related to memory. However, it could be a machine/VM related issue (as @boskoop stated above) or a container issue (if JVM is run through Jenkins/Docker/etc).
I believe you should increase the values of the memory settings - in MAVEN_OPTS
on the Jenkins machine, e.g.
MAVEN_OPTS=-Xmx1.5G -XX:MaxPermSize=0.7G
If you machine has at least 2 processors and 4GB memory, your JVM will not only grab 1GB at startup but will turn -server mode, meaning memory will be retained for performance sake (source). If you have few JVMs running at the same time (several application components, maven builds etc.) you can easily get into low memory. And one of you JVM may be killed by Linux OOM Killer because you are low on resources on the machine.
Reduce memory footprint of your process which is directly impacted by jvm default Xmx, which most probably is far from what jvm actually need.
Give it additional java command line options
-Xmx256m -XX:MaxPermSize=512m
or configure system variable
MAVEN_OPTS=-Xmx256m -XX:MaxPermSize=512m
MaxPermSize have no use for java 8+