Is there a mysql JDBC that will respect fetchSize?

Question

I am using MySQL and want to utilize the setFetchSize property. The default MySQL JDBC implementation does not really respect it. If you set fetchsize to

Answer 1

This is not really an answer to the above question. As I could not fit it in comment, I went to provide it as an answer. It may prove helpful to some facing a similar issue.

For a batch job, I needed to switch on the streaming mode as my result set was too large. At first, as seen in the MySQL doc, I set my connection up this way:

Statement extrapackStreamingQuery = dbExtrapackConnection.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY, java.sql.ResultSet.CONCUR_READ_ONLY); 
extrapackStreamingQuery.setFetchSize(Integer.MIN_VALUE);

But it would systematically give me the error:

Application was streaming results when the connection failed. Consider raising value of 'net_write_timeout' on the server.

I did try a few configuration options, like: max_allowed_packet = 128M, max_connect_errors = 9999 and net_write_timeout = 180. But none of them helped.

Wrongly thinking the TCP connection could be closed for being idle too long, I even tried changing the TCP ping time frame with a: net.ipv4.tcp_keepalive_time=60 in the /proc/sys/net/ipv4/tcp_keepalive_time and /etc/sysctl.conf files.

Indeed, if a database connection is opened but no TCP packets are sent for long enough, then the database connection will be lost as the TCP connection is closed. Sending TCP packets more often to keep the TCP connection alive may solve the issue.

But this didn't help either.

Then, after reading this piece I change my connection setup to:

protected static final int DB_STREAMING_FETCH_AMOUNT = 50;
...
Statement extrapackStreamingQuery = dbExtrapackConnection.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY, java.sql.ResultSet.CONCUR_READ_ONLY);    
extrapackStreamingQuery.setFetchSize(DB_STREAMING_FETCH_AMOUNT);

with my url using a trailing option:

String fullUrl = url + host + ":" + port + "/" + dbName;
if (streaming) {
    fullUrl += "?useCursorFetch=true";
}

My batch job is now working fine, it completes and even runs faster.