Why does Microsoft SQL Server 2012 query take minutes over JDBC 4.0 but second(s) in Management Studio?

Question

I am dealing with what is apparently a performance issue while retrieving a relatively large

Answer 1

I'm simply going to toss out this suggestion, and leave it for you to test.

The JDBC driver may well be FETCHING all of the rows before it returns, whereas the other system is simply returning the open cursor.

I have seen this behavior on other databases with JDBC, but had not direct experience with SQL Server.

In the examples where I have seen it, setting the auto commit to false for the connection prevents it from loading the entire result set. There are other settings to have it load only portions, etc.

But that could well be the underlying issue you are facing.

Answer 2

Perhaps, this link from Microsoft documentation can help you to solve your issue: http://msdn.microsoft.com/en-us/library/bb879937(v=sql.110).aspx

Especially in the "Guidelines for using adaptive buffering" part :

There are some cases where using selectMethod=cursor instead of responseBuffering=adaptive would be more beneficial, such as:

If your application processes a forward-only, read-only result set slowly, such as reading each row after some user input, using selectMethod=cursor instead of responseBuffering=adaptive might help reduce resource usage by SQL Server.

If your application processes two or more forward-only, read-only result sets at the same time on the same connection, using selectMethod=cursor instead of responseBuffering=adaptive might help reduce the memory required by the driver while processing these result sets.

In both cases, you need to consider the overhead of creating, reading, and closing the server cursors.

Answer 3

The adaptive buffering is a good answer. I would also recommend checking the connections' SET options via SQL Server Profiler.

When you start a trace, make sure ExistingConnections is selected. Compare a SPID from a JDBC connection and a SSMS connection. ARITHABORT comes to mind as one that I have seen cause a difference in performance between SSMS and JDBC driver. Microsoft briefly mentions it here: http://msdn.microsoft.com/en-us/library/ms190306.aspx. Stack Exchange information here: https://dba.stackexchange.com/questions/9840/why-would-set-arithabort-on-dramatically-speed-up-a-query

On Oracle, I have seen huge impacts by playing with the setFetchSize method on the Statement / PreparedStatement object. Apparently, the SQL Server driver does not support that method. However, there is an internal method in the driver for it. See Set a default row prefetch in SQL Server using JDBC driver for details.

Also, what are you doing in your while (rs.next()) loop? Try doing nothing other than reading a column, like rs.getInt(1). See what happens. If it flies, that suggests the bottleneck is in your former processing of the result set. If it is still slow, then the problem must be in the driver or database.

You could use SQL Server Profiler to compare the executions as they come in via JDBC and as you run it via SSMS. Compare the CPU, reads, writes and duration. If they are different, then the execution plan is probably different, which points me back to the first thing I mentioned: the SET options.

Answer 4

We had a similar problem that turned out to be due to caching. We consulted a very good read about query plans.

SQL Server caches execution plans, you can see it using:

select * from sys.dm_exec_cached_plans

What worked for us was to ignore the cached execution plan for the slow queries.

What happens is probably that for different queries, the optimization stage takes into account the queries' parameter values. Since in some cases it makes more sense using a different execution plan.

If the execution plan has been cached, and there is a cache hit (using a prepared statement, which ignores parameters), the execution plan may be sub-optimal for the same query with different parameters.

To verify this you can try and restore some queries and see if you get a different execution plan for the same query with different parameter.

If that turns out to be the case, there are several things you can do:

• For an immediate result add a recompile hint OPTION (RECOMPILE) to your query. This will recompile the query every time but may already be a lot faster than what you currently have. From the documentation:

RECOMPILE - Instructs the SQL Server Database Engine to discard the plan generated for the query after it executes, forcing the query optimizer to recompile a query plan the next time the same query is executed. Without specifying RECOMPILE, the Database Engine caches query plans and reuses them. When compiling query plans, the RECOMPILE query hint uses the current values of any local variables in the query and, if the query is inside a stored procedure, the current values passed to any parameters.

• Invalidate cache (see Recompiling Execution Plans). The ways to invalidate cache are. Note: some of these are not supported in azure:

1. Changes made to a table or view referenced by the query (ALTER TABLE and ALTER VIEW).

2. Changes made to a single procedure, which would drop all plans for that procedure from the cache (ALTER PROCEDURE).

3. Changes to any indexes used by the execution plan.

4. Updates on statistics used by the execution plan, generated either explicitly from a statement, such as UPDATE STATISTICS, or generated automatically.

5. Dropping an index used by the execution plan.

6. An explicit call to sp_recompile.

7. Large numbers of changes to keys (generated by INSERT or DELETE statements from other users that modify a table referenced by the query).

8. For tables with triggers, if the number of rows in the inserted or deleted tables grows significantly.

9. Executing a stored procedure using the WITH RECOMPILE option.

• Optimize the plan manually using OPTIMIZE FOR hints (did not try it).

• Use separate queries after classifying parameters.