Python Requests_html: giving me Timeout Error

问题

I'm trying to scrape headlines from medium.com by using this library called requests_html

The code I'm using works well on other's PC but not mine.

Here's what the original code looks like this:

from requests_html import HTMLSession

session = HTMLSession()
r = session.get('https://medium.com/@daranept27')
r.html.render()

x = r.html.find('a.eg.bv')
[print(elem.text) for elem in x]

It gives me pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 8000 ms exceeded.

Here's the full error:

Traceback (most recent call last):

  File "C:\Users\intel\Desktop\hackerrank.py", line 5, in <module>
    r.html.render()
  File "C:\Users\intel\AppData\Local\Programs\Python\Python38\lib\site-packages\requests_html.py", line 598, in render
    content, result, page = self.session.loop.run_until_complete(self._async_render(url=self.url, script=script, sleep=sleep, wait=wait, content=self.html, reload=reload, scrolldown=scrolldown, timeout=timeout, keep_page=keep_page))
  File "C:\Users\intel\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 616, in run_until_complete
    return future.result()
  File "C:\Users\intel\AppData\Local\Programs\Python\Python38\lib\site-packages\requests_html.py", line 512, in _async_render
    await page.goto(url, options={'timeout': int(timeout * 1000)})
  File "C:\Users\intel\AppData\Local\Programs\Python\Python38\lib\site-packages\pyppeteer\page.py", line 885, in goto

    raise error
pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 8000 ms exceeded.

[Finished in 13.0s with exit code 1]

[shell_cmd: python -u "C:\Users\intel\Desktop\hackerrank.py"]
[dir: C:\Users\intel\Desktop]
[path: C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\nodejs\;C:\Python38;C:\Users\intel\AppData\Local\Programs\Python\Python38\Scripts\;C:\Users\intel\AppData\Local\Programs\Python\Python38\;C:\MinGW\bin;C:\Users\intel\AppData\Local\Programs\Microsoft VS Code\bin]

I saw a comment on one of my posts and saw others' answers too to re-run it, then it will work. I don't understand why...

回答1:

The error you are getting suggests that you are not getting a response from the server in a timely manner.

I ran your code on my machine (Ubuntu 18.04) successfully and got the following results:

Seven Days -Between Life And Death
Have you ever encountered a fake friend? If So, Try These Simple Tips To Overcome it.
Does Anybody Ever Wonder Why He’s My Everything?
Ladies, Why Should You Treat Your Face Like The Coloring Books?
Listen, Girl, Aren’t You Curious How The Last Line Could Be This Hurtful?
The girl name “Rich”
She Lost Her Beloved Mother, But Why She Asserted that Loss Was Not Just A Loss sometimes?
You Used To Try This Lonely. Have You Ever Imagine The flavor You Tried To Eat it with Your Lover?
If You Have Siblings, You Won’t Comprehend this. Have You Ever Wonder How A Child Feels Like? This Is How It Perceives.
Is It Okay To Help A Stranger?
The Nightmare Was Always Considered A Bad Omen, But It Turned Incredible Differently.
If You’re A Woman Or Girl Who Loves To Wear Lipstick, Read This Poetry.
She Wants To Spread This Poetry For Every Girl Or Woman That Was Born Just Like The Way She Was.

You must check your internet connection.

Alternatively, I'd suggest you run your idle in administrator mode and re-run your code through idle.

来源：https://stackoverflow.com/questions/64648747/python-requests-html-giving-me-timeout-error