Why does the SqlServer optimizer get so confused with parameters?

Question

I know this has something to do with parameter sniffing, but I\'m just perplexed at how something like the following example is even possible with a piece of technology that

Answer 1

I have had this problem repeatedly on moving my code from a test server to production - on two different builds of SQL Server 2005. I think there are some big problems with the parameter sniffing in some builds of SQL Server 2005. I never had this problem on the dev server, or on two local developer edition boxes. I've never seen it it be such a big problem on SQL Server 2000 or any version going back to 6.5 either.

The cases where I found it, the only workaround was to use parameter masking, and I'm still hoping the DBAs will patch up the production server to SP3 so it will maybe go away. Things which did not work:

• using the WITH RECOMPILE hint on EXEC or in the SP itself.
• dropping and recreating the SP
• using sp_recompile

Note that in the case I was working on, the data was not changing since an earlier invocation - I had simply scripted the code onto the production box which already had data loaded. All the invocations came with no changes to the data since before the SPs existed.

Oh, and if SQL Server can't handle this without masking, they need to add a parameter modifier NOSNIFF or something. What happens if you mask all your parameters, so you have @Something_parm and @Something_var and someone changes the code to use the wrong one and all of a sudden you have a sniffing problem again? Plus you are polluting the namespace within the SP. All these SPs I am "fixing" drive me nuts because I know they are going to be a maintenance nightmare for the less experienced satff I will be handing this project off to one day.

Answer 2

It's probably caused by the fact that SQL Server compiles stored procedures and caches execution plans for them and the cached execution plan is probably unsuitable for this new set of parameters. You can try WITH RECOMPILE option to see if it's the cause.

EXECUTE MyProcedure [parameters] WITH RECOMPILE

WITH RECOMPILE option will force SQL Server to ignore the cached plan.

Answer 3

Could you check on the SQL Profiler how many reads and execution time when it is quick and when it is slow? It could be related to the number of rows fetched depending on the parameter value. It doesn't sound like a cache plan issue.

Answer 4

The 1 in 10 gives the wrong plan that is cached.

RECOMPILE adds an overhead, masking allows each parameter to be evaluated on it's own merits (very simply).

By wrong plan, what if the 1 in 10 generates an scan on index 1 but the other 9 produce a seek on index 2? eg, the 1 in 10 is, say, 50% of the rows?

Edit: other questions

• Known issue?: SQL Server 2005 stored procedure fails to complete with a parameter
• Stored Procedure failing on a specific user

Edit 2:

Recompile does not work because the parameters are sniffed at compile time.
From other links (pasted in):

This article explains...

...parameter values are sniffed during compilation or recompilation...

Finally (edit 3):

Parameter sniffing was probably a good idea at the time and probably works well mostly. We use it across the board for any parameter that will end up in a WHERE clause. We don't need to use it because we know that only a few (more complex eg reports or many parameters) could cause issues but we use it for consistency.

And the fact that it will come back and bite us when the users complain and we should have used masking...

Answer 5

Is every query reference to the parameter comparing it with int values, without functions and without casting?

Can you increase the specificity of any expressions using the parameter to make the use of multifield indexes more likely?

Answer 6

It is a problem with plan caching, and it isn't always related to parameters, as it was in your scenario.

(Parameter Sniffing problems occur when a proc is called with unusual parameters the FIRST time it runs, and so the cached plan works great for those odd values, but lousy for most other times the proc is called.)

We had a similar situation when the app team deleted all old records from a highly-used log table on a production server. Removing records improves performance, right? Nope, performance immediately tanked.

Turns out that a frequently-used stored proc was recompiled right when the table was nearly empty, and it cached an extremely poor execution plan ("hey, there's only 50 records here, might as well do a Table Scan!"). Would have happened no matter what the initial parameters.

Our fix was to force a recompile with sp_recompile.