Program to automate form field entry and result checking of webpage

Question

I am trying to create a program in C# (maybe using WinForms) which will enter a licence number into a form field of a specific website and validate whether or not the licence number is a currently valid licence.

I am unsure as to where to start, as I can't even find the form field id in the source code of the website, and am unsure what technologies the website uses.

Additionally, the purpose of this program will be to enter a list of license numbers and return the names and validation status of each license. Datasource being the website.

Any information on how to go about this would be much appreciated, I am an intermediate C# developer - having mostly worked in ASP.Net, though feel Winforms may be better suited for this project.

Kind Regards

Answer 1

You can use a WebBrowser control:

1. You can load the page using webBrowser1.Navigate("url of site")
2. Find elements in page using webBrowser1.Document.GetElementById("buttonid") also you can iterate over HtmlElement of webBrowser1.Document.Body.All and check for example element.GetAttribute("value") == "some vaule" to find it.
3. Set value for element using element.InnerText ="some value" or element.SetAttribute("value", "some value")
4. Submit your form by invoking the submit of form or click of its submit button using element.InvokeMember("method")

Example

For example, if you browse google and look at page source, you will see name of search text box is "q" and name of the form that contains the search box is "f", so you can write this codes to automate search.

1. Create a form with name BrowserSample.
2. From toolbox, drag a WebBrowser and drop on form.
3. Hanfdle Load event of form and navigate to google.
4. Handle DocumentCompleted event of webBrowser1 and find f and find q and set InnerText of q and invoke submit of f. This event fires after the navigation and document load completed.
5. In a real application add required null checking.

Code:

private void BrowserSample_Load(object sender, EventArgs e)
{
    this.webBrowser1.Navigate("https://www.google.com/");
}

void webBrowser1_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
    //Because submitting f causes navigation
    //to pervent a loop, we check the url of navigation 
    //and if it's different from google url, return
    if (e.Url.AbsoluteUri != "https://www.google.com/")
        return;
    var f = this.webBrowser1.Document.Body.All.GetElementsByName("f")
                .Cast<HtmlElement>()
                .FirstOrDefault();

    var q = f.All.GetElementsByName("q")
                .Cast<HtmlElement>()
                .FirstOrDefault();

    q.InnerText = "C# Webbrowser Control";
    f.InvokeMember("submit");
}

If you execute the program, it first navigate to google and then shows search result:

In your special case

Since the site loads content using ajax, then you should make a delay in DocumentCompleted:

async void webBrowser1_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
    if (e.Url.AbsoluteUri != "https://www.onegov.nsw.gov.au/PublicRegister/#/publicregister/search/Security")
        return;

    await Task.Delay(5000);
    var f = this.webBrowser1.Document.Body.All.GetElementsByName("searchForm")
                .Cast<HtmlElement>()
                .FirstOrDefault();

    var q = f.All.GetElementsByName("searchText")
                .Cast<HtmlElement>()
                .FirstOrDefault();

    q.InnerText = "123456789";
    f.InvokeMember("submit");
}

Don't forget to add using System.Threading.Tasks; or if you use .Net 4.0 you simple can use System.Threading.Thread.Sleep(5000) and remove async/await.

Answer 2

It looks like the website uses JSON POSTs. If you have FireFox open Developer -> Network and look at the "PerformSearch" entry. That will tell you everything you need to know as far as what the website is expecting in a POST request so you can read the response.