UnicodeEncodeError: 'charmap' codec can't encode characters

Question

I\'m trying to scrape a website, but it gives me an error.

I\'m using the following code:

import urllib.request
from bs4 import BeautifulSoup

get =          


        

Answer 1

set PYTHONIOENCODING=utf-8
set PYTHONLEGACYWINDOWSSTDIO=utf-8

You may or may not need to set that second environment variable PYTHONLEGACYWINDOWSSTDIO.

Alternatively, this can be done in code (although it seems that doing it through env vars is recommended):

sys.stdin.reconfigure(encoding='utf-8')
sys.stdout.reconfigure(encoding='utf-8')

---

Additionally: Reproducing this error was a bit of a pain, so leaving this here too in case you need to reproduce it on your machine:

set PYTHONIOENCODING=windows-1252
set PYTHONLEGACYWINDOWSSTDIO=windows-1252