Capture a literal string on stderr in powershell

Question

My application should write it\'s errors as literal JSON objects on stderr. This is proving difficult with PowerShell (5, 6 or 7) since PowerShell seems to want to prevent y