Cannot download file in R - status 503

Question

I\'m trying to download file:

> URL <- \"https://www.bitmarket.pl/graphs/BTCPLN/90m.json\"
> download.file(URL, destfile = \"res.json\", method = \"         


        

Answer 1

That's because the page is using a DDoS protection service. On the first load, the page itself does a JavaScript-initiated redirect after 5 seconds to fetch the final content so the process fails with tools like wget/curl which do not interpret JavaScript. If you think that it is justifiable to do so, then one option would be to use for example phantomjs and supply a custom script (say, save.js):

var system = require('system');
var page = require('webpage').create();

page.userAgent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/604.3.5 (KHTML, like Gecko) Version/11.0.1 Safari/604.3.5';

page.open(system.args[1], function(){
    setTimeout(function(){
        console.log(page.evaluate(function(){
            //gets the JSON from the first 
 element rendered on the page
            return document.getElementsByTagName('pre')[0].textContent;
        }));
        phantom.exit();
    }, 6000); //waits 6 seconds for the page to reload
});


and then use it instead of wget as:

phantomjs save.js https://www.bitmarket.pl/graphs/BTCPLN/90m.json