Example using WikipediaTokenizer in Lucene

Question

I want to use WikipediaTokenizer in lucene project - http://lucene.apache.org/java/3_0_2/api/contrib-wikipedia/org/apache/lucene/wikipedia/analysis/WikipediaTokenizer.html B

Answer 1

WikipediaTokenizer tf = new WikipediaTokenizer(new StringReader(test));

Token token = new Token();

token = tf.next(token);

http://www.javadocexamples.com/java_source/org/apache/lucene/wikipedia/analysis/WikipediaTokenizerTest.java.html

Regards