problem with powershell and cmd with pipes

Question

I have this command that works ok on powershell

Compare-Object (Get-Content \"tex1.txt\") (Get-Content \"tex2.txt\") | Where-Object{$_.SideIndicator -eq \"&l         


        

Answer 1

tl;dr

When calling the PowerShell CLI (powershell.exe for Windows PowerShell, pwsh for the cross-platform PowerShell [Core] 6+ edition):

• Using embedded " in an overall "..." string comes with escaping challenges.

• If feasible for a given command, formulating it without embedded " is the easiest solution:

powershell -Command "Compare-Object (Get-Content tex1.txt) (Get-Content tex2.txt) | Where-Object {$_.SideIndicator -eq '<='} | select inputobject |  ft -hidetableheaders"

Read on, if you do need to use embedded ".

---

eryksun points out that your problem is your lack of escaping of embedded " chars. inside the overall "..." string, which causes cmd.exe to see multiple strings, including parts it considers unquoted, which causes problems with special characters such as | and < - they never reach PowerShell.

Nesting double-quote strings from cmd.exe is tricky business:

• To make cmd.exe happy, you need to double the embedded " chars. ("")
• Separately, to make powershell.exe happy, you need to \-escape " chars.
  • Note: PowerShell [Core] 6+, the cross-platform edition, on Windows now also accepts "" by itself, which is the most robust choice.

Generally, dealing with quoting and escaping arguments when calling from cmd.exe is a frustrating experience with no universal solutions, unlike in the Unix world. Sadly, PowerShell has its own challenges, even in the Unix world.^[1]

In short: Escape embedded " chars. when calling Windows PowerShell from cmd.exe as follows:

• Use \"" (sic) when using powershell.exe -Command if preserving whitespace as-is is not required.
  This saves you from additional escaping, but the whitespace between \""...\"" runs isn't preserved, so this method is not suitable for passing a JSON string, for instance.

  • Caveat: \"" will not work for calling other programs.
  • Example: powershell -command " Write-Output \""a & b\"" " yields a & b; that is, while the & didn't need escaping, the double spaces around it were folded into a single space each.

• Use \" in all other cases, but you then need to individually ^-escape the following cmd.exe metacharacters with ^ inside \"...\" runs: & | < > ^^{Thanks, LotPings.}

  • Example: powershell -command " Write-Output \"a ^& b\" " yields a & b; that is, the & needed escaping with ^, but the double spaces around it were correctly preserved.

  • Additionally, to treat % (and, with enabledelayedexpansion , !) literally, the escaping syntax unfortunately depends on whether you're calling from the command line or a batch file: use %^USERNAME% (!^USERNAME) from the former, and %%USERNAME%% (^!USERNAME^! / ^^!USERNAME^^! inside \"...\" runs) from the latter - see this answer for the gory details.

  • Needless to say, implementing the above is a cumbersome and error-prone task that is not trivial to automate.
  • \" is supported by virtually all programs (except batch files), and if it weren't for these extra escaping requirements, command lines that use it have the potential to work across different platforms and shells - with the notable exception of calling from PowerShell, where, sadly, an additional layer of escaping is needed and " inside "..." must be escaped as \`" (sic).

See the bottom section for ways to ease the escaping pain by avoiding use of ".

---

Other programs, including PowerShell Core:

• Use just "" for programs compiled with Microsoft compilers and, on Windows, also Python and Node.js as well as PowerShell Core (pwsh.exe).
  Regrettably, this robust option does not work with powershell.exe, i.e. Windows PowerShell.

• Use \" for programs with Unix heritage, such as Perl and Ruby - which comes with the escaping headaches discussed above.

---

Avoiding embedded ":

When you call PowerShell's CLI, you can often get away without needing to embed double quotes:

• There may be arguments in your string that don't require quoting at all, such as text1.txt and text2.txt

• You can alternatively use single-quoting ('...') inside the overall command string, which require no escaping; note that such strings, from PowerShell's perspective, are string literals.

To put it all together:

powershell -Command "Compare-Object (Get-Content tex1.txt) (Get-Content tex2.txt) | Where-Object {$_.SideIndicator -eq '<='} | select inputobject |  ft -hidetableheaders"

^{Note that I've also removed the & { ... } around your command, as it isn't necessary.}

---

^{[1] eryksun puts it as follows: "This is the inescapable frustration of the Windows command line. Every program parses its own command line, using whatever rules it wants. So the syntax of a command line has to work with not only the shell (CMD) but also all programs invoked in the pipeline. In the Unix world the shell parses the command line into argv arrays, so typically you only have to get the syntax right to make the shell happy."
The problems with PowerShell Core, even on Unix, stem from how it re-quotes arguments behind the scenes before passing them on - see this GitHub docs issue.}