Programmatic Form Submit

Question

I want to scrape the contents of a webpage. The contents are produced after a form on that site has been filled in and submitted.

I\'ve read on how to scrape the en

Answer 1

Using python, I think it takes the following steps:

1. parse the web page that contains the form, find out the form submit address, and the submit method ("post" or "get").

this explains form elements in html file

2. Use urllib2 to submit the form. You may need some functions like "urlencode", "quote" from urllib to generate the url and data for post method. Read the library doc for details.