PL/SQL Performance Tuning for LIKE '%…%' Wildcard Queries

Question

We're using Oracle 11g database.
As you may or may not know, if you use wildcard query with "%" in front of the string, the column index is not being used and a full table scan is happening.

It looks like there isn't a definitive suggestion on how to improve this kind of query, but perhaps you could share some valuable information from your experience on how to optimize the following query:

SELECT * 
  FROM myTable 
 WHERE UPPER(CustomerName) like '%ABC%' 
    OR UPPER(IndemnifierOneName) like '%ABC%' 
    OR UPPER(IndemnifierTwoName) like '%ABC%';

...where all 3 columns are of type varchar2(100) and ABC is a value of variable input parameter.

@All suggesting CONTEX index, please note my data gets updated any time of the day every day and this index requires re-syncing, hence it's not a good option for a table of 1.5 million rows, sorry.

P.S. I'll upvote every answer, so please do keep them coming.

Answer 1

As already mentioned you could add a ctx context index to the name columns.

assuming a small number of records get updated, 1 option is to refresh your index daily. (and record when it happened)

then add a lastupdate date column & index to your table being searched.

It should be possible to scan your ctx index for the majority of the old unchanged data and select from the small percentage of updated data using the traditonal LIKE e.g:

WHERE (lastupdated<lastrefresh AND contains(name,'%ABC%')) 
   OR (lastupdated>lastrefresh AND name like '%ABC%')

NOTE: you may find your query plan goes a little mental (lots of bitmap conversions to row ids) in that case split the 2 parts of the OR into a UNION ALL query. e.g

SELECT id FROM mytable   
    WHERE 
    (lastupdate>lastrefresh and name LIKE '%ABC%')
    UNION ALL
    SELECT id FROM mytable   
    WHERE lastupdate<lastrefresh and CONTAINS(name, '%ABC%', 1) > 0

Answer 2

The only optimization is to not use that type of query and instead use the native capabilities of the database platform:

See Oracle Text: http://www.oracle.com/technetwork/database/enterprise-edition/index-098492.html

The common answer for SQL Server related questions would be Full Text Search.. nice to see Oracle has something as good or better.

Answer 3

The UPPER() is killing your indexes before anything, consider using a regexp. The initial % may avoid a normal index scan, but not always results in a Full Table Scan but into a Full Index Scan, which is faster than the FTS.

I suppose that 'ABC' is variable. If not, a function index is the way to go.

Answer 4

Sometimes this kind of query is unavoidable -- extracting the domain from a URL, or perhaps the root from a word with a prefix and a suffix.

You can resort to a full text index with or without a custom tokenizer.

Or if the strings you're searching for are finite in number and known in advance (e.g. you're working with a limited set of domain names that need to be extracted from a URL) you can use a deterministic function which can be indexed.

http://www.akadia.com/services/ora_function_based_index_2.html

Answer 5

Use Oracle text BUT the slightly newer CTXCAT variation - this domain index is updated as part of the transaction that inserts/updates the row in question and is therefore always up-to-date - See Oracle's own Oracle Text documentation for details.