Is there a way to let cURL wait until the page's dynamic updates are done?

Question

I\'m fetching pages with cURL in PHP. Everything works fine, but I\'m fetching some parts of the page that are calculated with JavaScript a fraction after the page is loaded

Answer 1

There is one quite tricky way to achieve it using php. If you' really like it to work for php you could potentially use Codeception setup in junction with Selenium and use Chrome browser webdriver in headless mode.

Here are some general steps to have it working.

1. You make sure you have codeception in your PHP project https://codeception.com

2. Download chrome webdriver: https://chromedriver.chromium.org/downloads

3. Download selenium: https://www.seleniumhq.org/download/

4. Configure it accordingly looking into documentation of codeception framework.

5. Write codeception test where you can use expression like $I->wait(5) for waiting 5 seconds or $I->waitForJs('js expression here') for waiting for js script to complete on the page.

6. Run written in previous step test using command php vendor/bin/codecept path/to/test