how to submit query to .aspx page in python

Question

I need to scrape query results from an .aspx web page.

http://legistar.council.nyc.gov/Legislation.aspx

The url is static, so how do I submit a query to thi

Answer 1

The code in the other answers was useful; I never would have been able to write my crawler without it.

One problem I did come across was cookies. The site I was crawling was using cookies to log session id/security stuff, so I had to add code to get my crawler to work:

Add this import:

    import cookielib            

Init the cookie stuff:

    COOKIEFILE = 'cookies.lwp'          # the path and filename that you want to use to save your cookies in
    cj = cookielib.LWPCookieJar()       # This is a subclass of FileCookieJar that has useful load and save methods

Install CookieJar so that it is used as the default CookieProcessor in the default opener handler:

    cj.load(COOKIEFILE)
    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
    urllib2.install_opener(opener)

To see what cookies the site is using:

    print 'These are the cookies we have received so far :'

    for index, cookie in enumerate(cj):
        print index, '  :  ', cookie        

This saves the cookies:

    cj.save(COOKIEFILE)                     # save the cookies