text-processing

I have a lot of compound strings that are a combination of two or three English words. e.g. "Spicejet" is a combination of the words "spice" and "jet" I need to separate these individual English words from such compound strings. My dictionary is going to consist of around 100000 words. What would be the most efficient by which I can separate individual English words from such compound strings. I'm not sure how much time or frequency you have to do this (is it a one-time operation? daily? weekly?) but you're obviously going to want a quick, weighted dictionary lookup. You'll also want to have a

Is there an open source Java library/algorithm for finding if a particular piece of text is a question or not? I am working on a question answering system that needs to analyze if the text input by user is a question. I think the problem can probably be solved by using opensource NLP libraries but its obviously more complicated than simple part of speech tagging. So if someone can instead tell the algorithm for it by using an existing opensource NLP library, that would be good too. Also let me know if you know a library/toolkit that uses data mining to solve this problem. Although it will be

Here's a website I found that will produce upside down versions of any English text. how does it work? does unicode have upside down chars? Or what? How can I write my own text flipping function? Patrick Hendricks does unicode have upside down chars? Yup! Or at least characters that look like they are upside down. Also, regular English-alphabetical characters can appear to be upside down. Like u could be an upside-down n . To code it up, you just have to take an array of characters, display them in reverse order and replace those characters with the upside down version of them. This will get

The English language has a couple of contractions . For instance: you've -> you have he's -> he is These can sometimes cause headache when you are doing natural language processing. Is there a Python library, which can expand these contractions? I made that wikipedia contraction-to-expansion page into a python dictionary (see below) Note, as you might expect, that you definitely want to use double quotes when querying the dictionary: Also, I've left multiple options in as in the wikipedia page. Feel free to modify it as you wish. Note that disambiguation to the right expansion would be a

Is there any library, preferably in python but at least open source, that can summarize and or simplify natural-language text? Rion Williams I'm not sure if there is currently any libraries that do this, as text summarization, or at least understandable text summarization isn't something that will be easily accomplished by a simple plug & play library. Here are a few links that I managed to find regarding projects / resources that are related to text summarization to get you started: The Lemur Project Python Natural Language Toolkit O'Reilly's Book on Natural Language Processing in Python

How can I remove duplicate characters from a string using Python? For example, let's say I have a string: foo = "SSYYNNOOPPSSIISS" How can I make the string: foo = SYNOPSIS I'm new to python and What I have tired and it's working. I knew there is smart and best way to do this.. and only experience can show this.. def RemoveDupliChar(Word): NewWord = " " index = 0 for char in Word: if char != NewWord[index]: NewWord += char index += 1 print(NewWord.strip()) NOTE: Order is important and this question is not similar to this one. Using itertools.groupby : >>> foo = "SSYYNNOOPPSSIISS" >>> import