Get word frequencies from SQL Server Full Text Search

Question

SQL Server Full-Text Search computes rank based on the frequency of the word in the document and in the set of all documents (TFIDF). Is it possible to access these values d

Answer 1

I don't know of an easy way to do this on SQL Server 2005, but on 2008 there are two Transact-SQL commands do exactly what you need.

• sys.dm_fts_index_keywords_by_document
  • http://technet.microsoft.com/en-us/library/cc280607.aspx
• sys.dm_fts_index_keywords
  • http://technet.microsoft.com/en-us/library/cc280900.aspx

Answer 2

I've not worked with Full-Text Search, but I have before accomplished a similar result by using a numbers table-based routine to split the input text into individual words (many examples of how to do this available elsewhere) and running aggregate functions against the results. Faster than you'd expect actually.