Processing on MySQL query taking too long and throwing “Communications link failure”

Question

I issued a query using a JDBC connection:

Connection conn = null
Class.forName(\"com.mysql.jdbc.Driver\")
conn = DriverManager.getConnection(dbHost, dbUser,          


        

Answer 1

If it happens because you have a lot of data in the result set, I would suggest to paginate the response with "LIMIT s, m" (where s - start, and m - max records - both integers). Then process this data in a loop with portions of 1000 records at a time:

boolean finished = false;
int start = 1;
int max = 1000;
do {
   r = s.executeQuery (MY_SELECT_QUERY + " LIMIT " + start + ", " + max);
   finished = // if r returned less than M records
   while(...) { 
     processResultSet(r);
   } 
   start += // number of returned records;
} while (!finished);

Of course, 1000 is an arbitrary number, you can play around and find what is the optimal max size for you query.

UPDATE: If processing of each record takes a long time, then store records in a list and iterate through them when all data is fetched from the database.