Stanford coreNLP - split words ignoring apostrophe

Question

I\'m trying to split a sentence into words using Stanford coreNLP . I\'m having problem with words that contains apostrophe.

For example, the sentence: I\'m 24 years

Answer 1

There are possessives and contractions. Your example is a contraction. Just looking for an apostrophe won't find you the difference between the two. "This is Pete's answer. I'm sure you knew that." In these two sentences we have one of each case.

With the part of speech tags we can tell the difference. With the tree surgeon syntax you can assemble those, change them and so forth. The syntax is listed here: http://nlp.stanford.edu/nlp/javadoc/javanlp/edu/stanford/nlp/trees/tregex/tsurgeon/package-summary.html. I've found tree surgeon to be really useful in pulling apart NP groups as I like to break them up over conjunctions.

Alternatively, does 'm stem to "am"? You might want to look for those and look for it's stem tag and simply revert it to that value. Stemming is extremely useful in many other aspects of machine learning and analysis.