Under Powershell v5, Windows 8.1, Python 3. Why these fails and how to fix?
[system.console]::InputEncoding = [System.Text.Encoding]::UTF8; [system.console]::OutputEncoding = [System.Text.Encoding]::UTF8; chcp; "import sys print(sys.stdout.encoding) print(sys.stdin.encoding) sys.stdout.write(sys.stdin.readline()) " | sc test.py -Encoding utf8; [char]0x0422+[char]0x0415+[char]0x0421+[char]0x0422+"`n" | py -3 test.py prints:
You are piping data into Python; at that point Python's stdin is no longer attached to a TTY (your console) and won't guess at what the encoding might be. Instead, the default system locale is used; on your system that's cp1251 (the Windows Latin-1-based codepage).
Set the PYTHONIOENCODING environment variable to override:
PYTHONIOENCODING
If this is set before running the interpreter, it overrides the encoding used for stdin/stdout/stderr, in the syntaxencodingname:errorhandler. Both theencodingnameand the:errorhandlerparts are optional and have the same meaning as instr.encode().
PowerShell doesn't appear to support per-command-line environment variables the way UNIX shells do; the easiest is to just set the variable first:
Set-Item Env:PYTHONIOENCODING "UTF-8" or even
Set-Item Env:PYTHONIOENCODING "cp65001" as the Windows UTF-8 codepage is apparently not quite UTF-8 really, depending on the Windows version and on wether or not pipe redirection is used.
Why not embed CPython in powershell?! CPython is so easy to embed, and powershell is very good REPL to play with .NET and COM objects. Here is a simple introduction to using pythonnet from PowerShell. Note how encoding is automatically propagated from powershell to python.
Windows PowerShell Copyright (C) 2015 Microsoft Corporation. All rights reserved. PS C:\Users\denfromufa> [system.console]::InputEncoding = [System.Text.Encoding]::UTF8; PS C:\Users\denfromufa> [system.console]::OutputEncoding = [System.Text.Encoding]::UTF8; PS C:\Users\denfromufa> [Reflection.Assembly]::LoadFile("C:\Python\Miniconda3_64b\Lib\site-packages\Python.Runtime.dll") GAC Version Location --- ------- -------- False v4.0.30319 C:\Python\Miniconda3_64b\Lib\site-packages\Python.Runtime.dll PS C:\Users\denfromufa> $gil = [Python.Runtime.Py]::GIL() PS C:\Users\denfromufa> $sys=[Python.Runtime.Py]::Import("sys") PS C:\Users\denfromufa> $sys.stdin.encoding.ToString() cp65001 PS C:\Users\denfromufa> $sys.stdout.encoding.ToString() cp65001 PS C:\Users\denfromufa> $gil.Dispose() PS C:\Users\denfromufa> [Python.Runtime.PythonEngine]::Shutdown() PS C:\Users\denfromufa> [EDIT]
Here is snek package that was released by one of powershell developers for embedding Python in powershell: