download csv from google insight for search

Question

Need help writing a script downloads data from google insight using c#

this is the download url and requires a login

http://www.google.com/insights/search/ov

Answer 1

To make this work you need to first authenticate in order to obtain a valid SID for a given google site which can be used to access data. Here's how you could achieve this:

class Program
{
    static void Main(string[] args)
    {
        using (var client = new WebClient())
        {
            // TODO: put your real email and password in the request string
            var response = client.DownloadString("https://www.google.com/accounts/ClientLogin?accountType=GOOGLE&Email=youraccount@gmail.com&Passwd=secret&service=trendspro&source=test-test-v1");
            // The SID is the first line in the response
            var sid = response.Split('\n')[0];
            client.Headers.Add("Cookie", sid);
            byte[] csv = client.DownloadData("http://www.google.com/insights/search/overviewReport?q=test&cmpt=q&content=1&export=2");

            // TODO: do something with the downloaded csv file:
            Console.WriteLine(Encoding.UTF8.GetString(csv));
            File.WriteAllBytes("report.csv", csv);
        }
    }
}

Answer 2

Ok it changed since few days.

Now you have to pass auth and not SID.

So code is now :

class Program
{
    static void Main(string[] args)
    {
        using (var client = new WebClient())
        {
            // TODO: put your real email and password in the request string
            var response = client.DownloadString("https://www.google.com/accounts/ClientLogin?accountType=GOOGLE&Email=youraccount@gmail.com&Passwd=secret&service=trendspro&source=test-test-v1");
            // The Auth line
            var auth = response.Split('\n')[2];
            client.Headers.Add("Authorization", "GoogleLogin " + auth);
            byte[] csv = client.DownloadData("http://www.google.com/insights/search/overviewReport?q=test&cmpt=q&content=1&export=2");

            // TODO: do something with the downloaded csv file:
            Console.WriteLine(Encoding.UTF8.GetString(csv));
            File.WriteAllBytes("report.csv", csv);
        }
    }
}

And now it work again for me.