Chromium/Chrome headless - file download not working?

Question

I downloaded the latest version of chromium, to test out the headless feature.

When I run (as root, because I'm still testing things):

./chrome --no-sandbox http://cp7.awardspace.com/speed-test/awardspace-data1mb.zip

In the GUI terminal, it opens Chromium and downloads the file.

If I'm trying to run it headless, I enter the following:

./chrome --no-sandbox --headless http://cp7.awardspace.com/speed-test/awardspace-data1mb.zip

The terminal outputs some information, no window gets opened, but also: I don't have the file downloaded anywhere.

I have been scouting the internet and discussion groups for more information, but cannot find anything.

Is file downloading not working in headless mode for Chromium?

Answer 1

That's a reported bug in headless implementation: https://bugs.chromium.org/p/chromium/issues/detail?id=696481

Answer 2

In Java use following code :

System.setProperty("webdriver.chrome.driver", "/usr/local/bin/chromedriver");
         ChromeOptions options = new ChromeOptions();
                options.addArguments("--test-type");
                options.addArguments("--headless");
                options.addArguments("--disable-extensions"); //to disable browser extension popup

                ChromeDriverService driverService = ChromeDriverService.createDefaultService();
                ChromeDriver driver = new ChromeDriver(driverService, options);

                Map<String, Object> commandParams = new HashMap<>();
                commandParams.put("cmd", "Page.setDownloadBehavior");
                Map<String, String> params = new HashMap<>();
                params.put("behavior", "allow");
                params.put("downloadPath", "//home//vaibhav//Desktop");
                commandParams.put("params", params);
                ObjectMapper objectMapper = new ObjectMapper();
                HttpClient httpClient = HttpClientBuilder.create().build();
                String command = objectMapper.writeValueAsString(commandParams);
                String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";
                HttpPost request = new HttpPost(u);
                request.addHeader("content-type", "application/json");
                request.setEntity(new StringEntity(command));
                httpClient.execute(request);
        driver.get("http://www.seleniumhq.org/download/");
        driver.findElement(By.linkText("32 bit Windows IE")).click();

Answer 3

Use ChromeDriverService and POST session/{sessionId}/chromium/send_command

JSON for POST:

{
    "cmd": "Page.setDownloadBehavior",
    "params": {
        "behavior": "allow",
        "downloadPath": "C:\\Download\\Path"
    }
}

C# Solution

Add reference to System.Net.Http and use NuGet to install Newtonsoft.JSON.

public static IWebDriver Driver { get; private set; }

public void SetDriver()
{
    var chromeOptions = new ChromeOptions();
    chromeOptions.AddArguments("--headless", "--window-size=1920,1080");

    var driverService = ChromeDriverService.CreateDefaultService();
    Driver = new ChromeDriver(driverService, chromeOptions);

    Task.Run(() => AllowHeadlessDownload(driverService));
}

static async Task AllowHeadlessDownload(ChromeDriverService driverService )
{
    var jsonContent = new JObject(
        new JProperty("cmd", "Page.setDownloadBehavior"),
        new JProperty("params",
        new JObject(new JObject(
            new JProperty("behavior", "allow"),
            new JProperty("downloadPath", @"C:\Download\Path")))));
    var content = new StringContent(jsonContent.ToString(), Encoding.UTF8, "application/json");
    var sessionIdProperty = typeof(ChromeDriver).GetProperty("SessionId");
    var sessionId = sessionIdProperty.GetValue(Driver, null) as SessionId;

    using (var client = new HttpClient())
    {
        client.BaseAddress = driverService.ServiceUrl;
        var result = await client.PostAsync("session/" + sessionId.ToString() + "/chromium/send_command", content);
        var resultContent = await result.Content.ReadAsStringAsync();
    }
}

Answer 4

Note: Not exactly answer to the question, but solves the problem

I researched a lot on making headless chrome download with different parameters/options/preferences, but nothing worked. Then I used standard Java way of downloading file using Apache Commons's FileUtils

FileUtils.copyURLToFile(URI, FILE);

Answer 5

I was able to download files with chrome headless thanks to Chrome Remote Interface

public void TryEnableFileDownloading(string downloadPath)
{
    TrySendCommand("Page.setDownloadBehavior", new Dictionary<string, object>()
    {
        ["behavior"] = "allow",
        ["downloadPath"] = downloadPath
    });
}

Full code for integration with selenium could be found here https://github.com/cezarypiatek/Tellurium/blob/master/Src/MvcPages/SeleniumUtils/ChromeRemoteInterface/ChromeRemoteInterface.cs

More info about setDownloadBehavior and Chrome Remote interface https://chromedevtools.github.io/devtools-protocol/tot/Page/#method-setDownloadBehavior

Answer 6

The following code works in C# using ChromeDriver 2.46

    private ChromeDriver GetDriver()
    {
        var options = new ChromeOptions();

        options.AddArguments("headless");
        options.AddUserProfilePreference("download.prompt_for_download", "false");
        options.AddUserProfilePreference("download.directory_upgrade", "true");
        options.AddUserProfilePreference("download.prompt_for_download", "false");
        options.AddUserProfilePreference("safebrowsing.enabled", "false");
        options.AddUserProfilePreference("safebrowsing.disable_download_protection", "true");

        options.AddArguments("--disable-web-security");

        var curr = Directory.GetCurrentDirectory();
        options.AddUserProfilePreference("download.default_directory", curr);

        var driver = new ChromeDriver(options);
        Log.Info($"Started Chrome Driver with options: {options.ToJsonNoTypes()}");

        var param = new Dictionary<string, object>();
        param.Add("behavior", "allow");
        param.Add("downloadPath", curr);
        driver.ExecuteChromeCommand("Page.setDownloadBehavior", param);

        return driver;
    }