implementing a perceptron classifier

Question

Hi I\'m pretty new to Python and to NLP. I need to implement a perceptron classifier. I searched through some websites but didn\'t find enough information. For now I have a numb

Answer 1

I had a try at implementing something similar the other day. I made some code to recognize english looking text vs non-english. I hadn't done AI or statistics in many years, so it was a bit of a shotgun attempt.

My code is here (don't want to bloat the post): http://cnippit.com/content/perceptron-statistically-recognizing-english

Inputs:

• I take a text file, split it up into tri-grams (eg "abcdef" => ["abc", "bcd", "cde", "def"])
• I calculate the relative frequencies of each, and feed that as the inputs to the perceptron (so there are 26^3 inputs)

Despite me not really knowing what I was doing, it seems to work fairly well. The success depends quite heavily on the training data though. I was getting poor results until I trained it on more french/spanish/german text etc.

It's a very small example though, with lots of "lucky guesses" at values (eg. initial weights, bias, threshold, etc.).

Multiple classes: If you have multiple classes you want to distinquish between (ie. not as simple as "is A or NOT-A"), then one approach is to use a perceptron for each class. Eg. one for sport, one for news, etc.

Train the sport-perceptron on data grouped as either sport or NOT-sport. Similar for news or Not-news, etc.

When classifying new data, you pass your input to all perceptrons, and whichever one returns true (or "fires"), then that's the class the data belongs to.

I used this approach way back in university, where we used a set of perceptrons for recognizing handwritten characters. It's simple and worked pretty effectively (>98% accuracy if I recall correctly).