Getting “sed error - illegal byte sequence” (in bash) [duplicate]

Answer 1

I managed to do it by running:

unset LANG

before the sed command.

Not sure what I've done or why it works but it did.

Answer 2

LANG=C alone didn't do the trick for me but adding LC_CTYPE=C as well solved it.

Answer 3

Try setting the LANG environment variable (LANG=C sed ...) or use one of the binary sed tools mentioned here: binary sed replacement

Why the error?

Without LANG=C sed assumes that files are encoded in whatever encoding is specified in LANG and the file (being binary) may contain bytes which are not valid characters in LANG's encoding (thus you could get 'illegal byte sequence').

Why does LANG=C work?

C just happens to treat all ASCII characters as themselves and non-ASCII characters as literals.

Answer 4

In addition to LANG=C and LC_CTYPE=C, I had to do LC_ALL=C to get this to work.

LC_ALL overrides all individual LC_* categories. Thus, the most robust approach is to use LC_ALL=C sed ... - no need to also deal with the other variables.