Simplest method for text lemmatization in Scala and Spark

Question

I want to use lemmatization on a text file:

surprise heard thump opened door small seedy man clasping package wrapped.

upgrading system found review spring          


        

Answer 1

I would suggest using the Stanford CoreNLP wrapper for Apache Spark as it gives the official API for the basic core nlp function such as Lemmatization, tokenization, etc.

I have used the same for lemmatization on a spark dataframe.

Link to use :https://github.com/databricks/spark-corenlp