“Between” operator generating bad query plan when using parameters

Question

I have a simple dates table (Date, DateID) which contains a list of dates between 1-Jan-1900 and 31-Dec-2100.

When selecting from the table using the between operator and hard-coded parameter values, I get a correct query plan with 3 estimated rows compared to 2 actual rows:

select v.Date from Dates v
where v.Date between '20130128' and '20130129';

However when replacing the hard-coded values with parameters, the query plan changes to a very poor plan, with over 6000 estimated rows and only 2 actual rows:

select v.Date from Dates v
where v.Date between @startdate and @enddate;

The query plans themselves are identical, it's just the difference in estimated rows that is causing the parameterised query to run about 4 times slower than the hard-coded query. Is there anything I'm missing as to why the parameterised version runs so much slower, and what indexes/hints can I give SQL Server to help it use the correct query plan?

Some additional info:

• The problem does not arise when using simple equality = criteria, it seems specific to the between operator.
• If I add option(recompile) to the end of the parameterised query, I get a perfect query plan, identical to the hard-coded query.
• The dates table has only two columns, Date and DateID, with a clustered index on the primary key DateID column, and a unique non-clustered index on the Date column. All have updated statistics.
• The query plan performs automatic parameterisation for the hard-coded query, replacing the hard-coded values with @1 and @2, and displaying the query as uppercase. It doesn't appear to perform any transformation for the parameterised query.
• Using SQL Server 2008 R2.

I know enough to realise suspect that this is some sort of parameter sniffing problem. Why not add option(recompile) to the query? This is being used as part of a much larger complex query and I understand good practice is to let SQL Server do its thing and re-use query plans from the cache where possible.

Edit and update: thanks for the thoughtful responses so far. To refine the question further, the query plan uses a perfectly good index for both of the above queries, but why does it not recognise that the date range is only two days wide for the parameterised query, why instead does it think the range is 6000 rows wide? Especially when, looking at the query plan, SQL Server is performing automatic parameterisation for the hard-coded query anyway?? In the underlying query plan, both plans look identical in that they both are parameterised!

Answer 1

The query plan is based on the parameter values when you first run the query. This is called parameter sniffing. When you add option (recompile), a new plan is generated for each execution.

A query plan is cached based on the hash of the SQL query. So there's a different cache slot for both versions of your query.

Adding option (recompile) is a good solution. You could also use:

option (optimize for (@startdate = '20130128', @enddate = '20130129'));

To generate a query plan as if those values had been passed in.

For testing, you can remove all plans from the cache with:

DBCC FREEPROCCACHE

Answer 2

The problem is likely due to parameter sniffing, a technique designed to optimise the query plan based on the parameters that you pass in first time.

I've had bad experiences with parameter sniffing with date parameters in the past - There is clearly a chance with parameter sniffing that the values initially entered for the query are not representative of typical use of the query (resulting in a query plan optimised for non-typical values), however with date parameters in particular I've often found that the resulting query plan is in fact very inefficient for all values supplied. I have no idea why this is however disabling parameter sniffing generally fixes it.

You can prevent SQL Server from parameter sniffing by using OPTIMIZE FOR UNKNOWN (I've never used this so can't guarentee that it works), or alternatively copying parameters into local variables seems to prevent SQL Server from being able to perform parameter sniffing

-- Perform the query using @StartDateLocal and @EndDateLocal
DECLARE @StartDateLocal DATETIME;
SET @StartDateLocal = @StartDate;

Disabling parameter sniffing in this way is better than forcing a recompile of the query plan each time as compiling a query plan is typically relatively expensive compared to the cost of executing the query unless the query is fairly slow.