SqlException timeout expired without being reached

Question

From time to time our server throw this well-known exception:

Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

This happens under pressure when the server is working on big requests. I did some research and found out that I could change connection string connection timeout setting and/or SqlCommand.Timeout data reader properties.

By default, sql command timeout is set to 30 seconds and connection timeout to 15, and we never override them.

I reproduced the context and executed the failling requests by hand in management studio. Their duration are around 1 second and always far beyond 30.

But strangely when I take a look at the server logs, this exception is thrown right away the request call. I mean, the request is executing and one millisecond later the exception is raised. Excuse me but let me do my geek look about this 8-o.

To be complete, our sql instance is mirrored with another one in synchronous mode. We use Ado.Net through table adapters.

Answer 1

In fact we still experienced these random timeouts even after READ_COMMITED_SNAPSHOT was set.

Setting the mirror in asynchronous mode didn't helped, queries done in multiple threads still randomly timeouted after about 1ms, always on busy periods. On the other hand the specific query that triggered the timeout (an INSERT statement) executed itself really fast (less than 1ms CPU and about 10 reads in average).

The call stack was the following:

at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject)

at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)

at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)

at System.Data.SqlClient.SqlConnection.Open()

So the timeout didn't seem related to the query itself.

According to this other post : Multiple Simultaneous SQL Connection Timeouts In Multithreaded Windows Service and the linked MSDN blog post speaking about an ADO.NET bug, we tried to set the connection timeout to 150 in the connection string.

Can't be certain we were experiencing this bug, but no more timeouts have been thrown since this change.

Answer 2

In the end, after hours of tracking and profiling, the issue was the correlation of two things:

1. Read committed default Sql Server isolation level leading to blocking situations
2. Some really badly tuned requests and stored procedures mixed with indexless tables

The first cause was fixed with

ALTER DATABASE <dbName> SET READ_COMMITTED_SNAPSHOT ON

The second with lucid requests rewriting and table indexing.

Answer 3

I would run SQL Profiler at this point and see what queries are being executed.