Best Algorithmic Approach to Sentiment Analysis [closed]

Answer 1

Machine-learning techniques are probably better.

Whitelaw, Garg, and Argamon have a technique that achieves 92% accuracy, using a technique similar to yours for dealing with negation, and support vector machines for text classification.

Answer 2

I believe the best answer to all of the questions that you mentioned is reading the book under the title of "Sentiment Analysis and opinion mining" by Professor Bing Liu. This book is the best of its own in the field of sentiment analysis. it is amazing. Just take a look at it and you will find the answer to all your 'why' and 'how' questions!

Answer 3

You may find the OpinionFinder system and the papers describing it useful. It is available at http://www.cs.pitt.edu/mpqa/ with other resources for opinion analysis.

It goes beyond polarity classification at the document level, but try to find individual opinions at the sentence level.

Answer 4

Why don't you try something similar to how SpamAsassin spam filter works? There really not much difference between intension mining and opinion mining.

Answer 5

my favorite example is "just read the book". it contains no explicit sentiment word and it is highly depending on the context. If it apears in a movie review it means that the-movie-sucks-it's-a-waste-of-your-time-but-the-book-is-good. However, if it is in a book review it delivers a positive sentiment.

And what about - "this is the smallest [mobile] phone in the market". back in the '90, it was a great praise. Today it may indicate that it is a way too small.

I think this is the place to start in order to get the complexity of sentiment analysis: http://www.cs.cornell.edu/home/llee/opinion-mining-sentiment-analysis-survey.html (by Lillian Lee of Cornell).

Answer 6

I don't think there's anything particularly wrong with your algorithm, it's a fairly straightforward and practical way to go, but there are a lot of situations where it will get make mistakes.

1. Ambiguous sentiment words - "This product works terribly" vs. "This product is terribly good"

2. Missed negations - "I would never in a millions years say that this product is worth buying"

3. Quoted/Indirect text - "My dad says this product is terrible, but I disagree"

4. Comparisons - "This product is about as useful as a hole in the head"

5. Anything subtle - "This product is ugly, slow and uninspiring, but it's the only thing on the market that does the job"

I'm using product reviews for examples instead of news stories, but you get the idea. In fact, news articles are probably harder because they will often try to show both sides of an argument and tend to use a certain style to convey a point. The final example is quite common in opinion pieces, for example.

As far as NLP helping you with any of this, word sense disambiguation (or even just part-of-speech tagging) may help with (1), syntactic parsing might help with the long range dependencies in (2), some kind of chunking might help with (3). It's all research level work though, there's nothing that I know of that you can directly use. Issues (4) and (5) are a lot harder, I throw up my hands and give up at this point.

I'd stick with the approach you have and look at the output carefully to see if it is doing what you want. Of course that then raises the issue of what you want you understand the definition of "sentiment" to be in the first place...