One 400GB table, One query - Need Tuning Ideas (SQL2005)

Question

I have a single large table which I would like to optimize. I\'m using MS-SQL 2005 server. I\'ll try to describe how it is used and if anyone has any suggestions I would appreci

Answer 1

It looks like you are not using your clustered index to its full potential, and have a LOT of duplicated data.

Your clustered index seems to be constructed something like:

create clustered index IX_Clustered on Table(k1 ASC, k2 ASC)

However, your other k* columns represent only 40,000 possible permutations.

10 (k1) * 10 (k3) * 100 (k4) * 2 (k5) * 2 (k6) = 40,000

You should pull unique combinations of these 4 keys out into a separate table and give each of these a unique int (primary key "newPK").

Excuse the pseudocode please:

create table SurrogateKey(
  newPK int -- /*primary key*/
, k1, k3, k4, k5, k6
)

constraint: newPK is primary key, clustered
constraint: k1, k3, k4, k5, k6 is unique

This table will only have 40,000 rows and be very fast to lookup the primary key, newPK. Then, you can lookup a single integer in your large table.

Your existing table should be altered to have the following columns:

• newPK
• k2 (which really is not a key, probably just a sequence number)
• d1
• k7 the datetime

Given the above, you can change your clustered index to:

create clustered index IX_Clustered on Table(newPK ASC)

And you can seek along this. It is guaranteed to be faster than what your query is doing now (equivalent performance to an index scan + key lookup).

declare @pk int
select @pk = newPK 
from SurrogateKey
where
      k1 = @k1
  and k3 = @k3
  and k4 = @k4
  and k5 = @k5
  and k6 = @k6

select top(g1) d1, k2, k7
from Table with(read uncommitted)
where newPK = @pk
order by k7

Your insert statement will need to be modified to query/insert the SurrogateKey table as well.