SQLRecoverableException: I/O Exception: Connection reset

Question

Yesterday evening I left the office with a running Java program written by me. It should insert a lot of records into our company database (Oracle) using a JDBC connection.

Answer 1

We experienced these errors intermittently after upgraded from 11g to 12c and our java was on 1.6.

The fix for us was to upgrade java and jdbc from 6 to 7

export JAVA_HOME='/usr/java1.7'

export CLASSPATH=/u01/app/oracle/product/12.1.0/dbhome_1/jdbc/libojdbc7.jar:$CLASSPATH 

Several days later, still intermittent connection resets.

We ended up removing all the java 7 above. Java 6 was fine. The problem was fixed by adding this to our user bash_profile.

Our groovy scripts that were experiencing the error were using /dev/random on our batch VM server. Below forced java and groovy to use /dev/urandom.

export JAVA_OPTS=" $JAVA_OPTS -Djava.security.egd=file:///dev/urandom "

Answer 2

Your exception says it all "Connection reset". The connection between your java process and the db server was lost, which could have happened for almost any reason(like network issues). The SQLRecoverableException just means that its recoverable, but the root cause is connection reset.

Answer 3

This simply means that something in the backend ( DBMS ) decided to stop working due to unavailability of resources etc. It has nothing to do with your code or the number of inserts. You can read more about similar problems here:

• http://kr.forums.oracle.com/forums/thread.jspa?threadID=941911
• http://forums.oracle.com/forums/thread.jspa?messageID=3800354

This may not answer your question, but you will get an idea of why it might be happening. You could further discuss with your DBA and see if there is something specific in your case.

Answer 4

I had a similar situation when reading from Oracle in a Spark job. This connection reset error was caused by an incompatibility between the Oracle server and the JDBC driver used. Worth checking it.

Answer 5

add java security in your run command

java -jar -Djava.security.egd="file:///dev/urandom" yourjarfilename.jar

Answer 6

I want to produce a complementary answer of nacho-soriano's solution ...

I recently search to solve a problem where a Java written application (a Talend ELT job in fact) want to connect to an Oracle database (11g and over) then randomly fail. OS is both RedHat Enterprise and CentOS. Job run very quily in time (no more than half a minute) and occur very often (approximately one run each 5 minutes).

Some times, during night-time as work-time, during database intensive-work usage as lazy work usage, in just a word randomly, connection fail with this message:

Exception in component tOracleConnection_1
java.sql.SQLRecoverableException: Io exception: Connection reset
        at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:101)
        at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:112)
        at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:173)
        at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:229)
        at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:458)
        at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:411)
        at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:490)
        at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:202)
        at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:33)
        at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:465)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:208)
    and StackTrace follow ...

Problem explanation:

As detailed here

Oracle connection needs some random numbers to assume a good level of security. Linux random number generator produce some numbers bases keyboard and mouse activity (among others) and place them in a stack. You will grant me, on a server, there is not a big amount of such activity. So it can occur that softwares use more random number than generator can produce.

When the pool is empty, reads from /dev/random will block until additional environmental noise is gathered. And Oracle connection fall in timeout (60 seconds by default).

Solution 1 - Specific for one app solution

The solution is to give add two parameters given to the JVM while starting:

-Djava.security.egd=file:/dev/./urandom
-Dsecurerandom.source=file:/dev/./urandom

Note: the '/./' is important, do not drop it !

So the launch command line could be:

java -Djava.security.egd=file:/dev/./urandom -Dsecurerandom.source=file:/dev/./urandom -cp <classpath directives> appMainClass <app options and parameters>

One drawback of this solution is that numbers generated are a little less secure as randomness is impacted. If you don't work in a military or secret related industry this solution can be your.

Solution 2 - General Java JVM solution

As explained here

Both directives given in solution 1 can be put in Java security setting file.

Take a look at $JAVA_HOME/jre/lib/security/java.security

Change the line

securerandom.source=file:/dev/random

to

securerandom.source=file:/dev/urandom

Change is effective immediately for new running applications.

As for solution #1, one drawback of this solution is that numbers generated are a little less secure as randomness is impacted. This time, it's a global JVM impact. As for solution #1, if you don't work in a military or secret related industry this solution can be your.

We ideally should use "file:/dev/./urandom" after Java 5 as previous path will again point to /dev/random.

Reported Bug : https://bugs.openjdk.java.net/browse/JDK-6202721

Solution 3 - Hardware solution

Disclamer: I'm not linked to any of hardware vendor or product ...

If your need is to reach a high quality randomness level, you can replace your Linux random number generator software by a piece of hardware.

Some information are available here.

Regards

Thomas