Why does PowerShell redirection >> change the formatting of the text content?

Question

I want to use the redirect append >> or write > to write to a txt file, but when I do, I receive a weird format "\\x00a\\x00p..

Answer 1

>> or out-file -append will append unicode text by default, even if the file isn't unicode in the first place. Add-content will check the encoding of the file first, and match it. Add-content or set-content defaults to ansi encoding as well. I would never use >, >>, or out-file.

Seeing something with spaces in between is a giveaway that it's unicode. Unicode has $nulls between each letter usually. If you dump the hex, like in emacs esc-x hexl-mode, you can see it. Boms are 2 or 3 hex characters in the beginning of a file.

a p p e n d e d   u s i n g   r e d i r e c t

This is a correctly constructed unicode text file, copied and pasted from emacs hexl-mode. fffe is the bom. After each character is 00. At the end is 0d and 0a, carriage return and linefeed. Stuff like this interests me. It's possible for some windows utilities to make a unicode text file with no bom (icacls /save). Then if you type the file, the letters will appear to have spaces in-between.

00000000: fffe 6100 7000 7000 6500 6e00 6400 6500  ..a.p.p.e.n.d.e.
00000010: 6400 2000 7500 7300 6900 6e00 6700 2000  d. .u.s.i.n.g. .
00000020: 7200 6500 6400 6900 7200 6500 6300 7400  r.e.d.i.r.e.c.t.
00000030: 0d00 0a00                                ....