Why and how are these two $null values different?

Question

Apparently, in PowerShell (ver. 3) not all $null\'s are the same:

    >function emptyArray() { @() }
    >$l_t = @() ; $l_t.Count
0
    &g         


        

Answer 1

To complement PetSerAl's great answer with a pragmatic summary:

• Commands that happen to produce no output do not return $null, but the [System.Management.Automation.Internal.AutomationNull]::Value singleton, which can be thought of as an "array-valued $null" or, to coin a term, null array.

  • Note that, due to PowerShell's unwrapping of collections, even a command that explicitly outputs an empty collection object such as @() has no output (unless enumeration is explicitly prevented, such as with Write-Output -NoEnumerate).

• In short, this special value behaves like $null in scalar contexts, and like an empty array in array / pipeline contexts, as the examples below demonstrate.

Caveats:

• Passing [System.Management.Automation.Internal.AutomationNull]::Value as a cmdlet / function parameter value invariably converts it to $null.

  • See this GitHub issue.

• In PSv3+, even an actual (scalar) $null is not enumerated in a foreach loop; it is enumerated in a pipeline, however - see bottom.

• In PSv2-, saving a null array in a variable quietly converted it to $null and $null was enumerated in a foreach loop as well (not just in a pipeline) - see bottom.

# A true $null value:
$trueNull = $null  

# An operation with no output returns
# the [System.Management.Automation.Internal.AutomationNull]::Value singleton,
# which is treated like $null in a scalar expression context, 
# but behaves like an empty array in a pipeline or array expression context.
$automationNull = & {}  # calling (&) an empty script block ({}) produces no output

# In a *scalar expression*, [System.Management.Automation.Internal.AutomationNull]::Value 
# is implicitly converted to $null, which is why all of the following commands
# return $true.
$null -eq $automationNull
$trueNull -eq $automationNull
$null -eq [System.Management.Automation.Internal.AutomationNull]::Value
& { param($param) $null -eq $param } $automationNull

# By contrast, in a *pipeline*, $null and
# [System.Management.Automation.Internal.AutomationNull]::Value
# are NOT the same:

# Actual $null *is* sent as data through the pipeline:
# The (implied) -Process block executes once.
$trueNull | % { 'input received' } # -> 'input received'

# [System.Management.Automation.Internal.AutomationNull]::Value is *not* sent 
# as data through the pipeline, it behaves like an empty array:
# The (implied) -Process block does *not* execute (but -Begin and -End blocks would).
$automationNull | % { 'input received' } # -> NO output; effectively like: @() | % { 'input received' }

# Similarly, in an *array expression* context
# [System.Management.Automation.Internal.AutomationNull]::Value also behaves
# like an empty array:
(@() + $automationNull).Count # -> 0 - contrast with (@() + $trueNull).Count, which returns 1.

# CAVEAT: Passing [System.Management.Automation.Internal.AutomationNull]::Value to 
# *any parameter* converts it to actual $null, whether that parameter is an
# array parameter or not.
# Passing [System.Management.Automation.Internal.AutomationNull]::Value is equivalent
# to passing true $null or omitting the parameter (by contrast,
# passing @() would result in an actual, empty array instance).
& { param([object[]] $param) 
    [Object].GetMethod('ReferenceEquals').Invoke($null, @($null, $param)) 
  } $automationNull  # -> $true; would be the same with $trueNull or no argument at all.

---

The [System.Management.Automation.Internal.AutomationNull]::Value documentation states:

Any operation that returns no actual value should return AutomationNull.Value.

Any component that evaluates a Windows PowerShell expression should be prepared to deal with receiving and discarding this result. When received in an evaluation where a value is required, it should be replaced with null.

---

PSv2 vs. PSv3+, and general inconsistencies:

PSv2 offered no distinction between [System.Management.Automation.Internal.AutomationNull]::Value and $null for values stored in variables:

• Using a no-output command directly in a foreach statement / pipeline did work as expected - nothing was sent through the pipeline / the foreach loop wasn't entered:

  Get-ChildItem nosuchfiles* | ForEach-Object { 'hi' }
  foreach ($f in (Get-ChildItem nosuchfiles*)) { 'hi' }
  

• By contrast, if a no-output commands was saved in a variable or an explicit $null was used, the behavior was different:

  # Store the output from a no-output command in a variable.
  $result = Get-ChildItem nosuchfiles* # PSv2-: quiet conversion to $null happens here
  
  # Enumerate the variable.
  $result | ForEach-Object { 'hi1' }
  foreach ($f in $result) { 'hi2' }
  
  # Enumerate a $null literal.
  $null | ForEach-Object { 'hi3' }
  foreach ($f in $null) { 'hi4' }
  

  • PSv2: all of the above commands output a string starting with hi, because $null is sent through the pipeline / being enumerated by foreach:
    Unlike in PSv3+, [System.Management.Automation.Internal.AutomationNull]::Value is converted to $null on assigning to a variable, and $null is always enumerated in PSv2.

  • PSv3+: The behavior changed in PSv3, both for better and worse:

    • Better: Nothing is sent through the pipeline for the commands that enumerate $result: The foreach loop is not entered, because the [System.Management.Automation.Internal.AutomationNull]::Value is preserved when assigning to a variable, unlike in PSv2.

    • Possibly Worse: foreach no longer enumerates $null (whether specified as a literal or stored in a variable), so that foreach ($f in $null) { 'hi4' } perhaps surprisingly produces no output.
      On the plus side, the new behavior no longer enumerates uninitialized variables, which evaluate to $null (unless prevented altogether with Set-StrictMode).
      Generally, however, not enumerating $null would have been more justified in PSv2, given its inability to store the null-collection value in a variable.

In summary, the PSv3+ behavior:

• takes away the ability to distinguish between $null and [System.Management.Automation.Internal.AutomationNull]::Value in the context of a foreach statement

• thereby introduces an inconsistency with pipeline behavior, where this distinction is respected.

For the sake of backward compatibility, the current behavior cannot be changed. This comment on GitHub proposes a way to resolve these inconsistencies for a (hypothetical) potential future PowerShell version that needn't be backward-compatible.