Algorithm wanted: Find all words of a dictionary that are similar to words in a free text

Question

We have a list of about 150,000 words, and when the user enters a free text, the system should present a list of words from the dictionary, that are very close to words in t

Answer 1

You would likely want to use an algorithm which calculates the Levenshtein distance.

However, since your data set is quite large, and you'll be comparing lots of words against it, a direct implementation of typical algorithms that do this won't be practical.

In order to find words in a reasonable amount of time, you will have to index your set of words in some way that facilitates fuzzy string matching.

One of these indexing methods would be to use a suffix tree. Another approach would be to use n-grams.

I would lean towards using a suffix tree since I find it easier to wrap my head around it and I find it more suited to the problem.