How to grab streaming data from twitter connect with pycurl using nltk - regular expression

Question

I am newbie in Python and given a task from my boss to do this :

1. Grab streaming data from twitter connect with pycurl and output in JSON
2. Parsing using NLTK and Regular Expression
3. Save it to database file(mySQL) or file base(txt)

Note : this is the url that i want to grab ('http://search.twitter.com/search.json?geocode=-0.789275%2C113.921327%2C1.0km&q=+near%3Aindonesia+within%3A1km&result_type=recent&rpp=10')

Is there anyone know how to grab a streaming data from twitter using the step above ?

Your help would be very grateful :)

Answer 1

I would look at pattern: it's a very nice web mining library, and it comes with a Twitter mining api as well. The documentation is pretty good too.

Otherwise, look at https://dev.twitter.com/docs/twitter-libraries for twitter libraries, and getting the stream should be pretty straightforward too.