Merging hashtables in PowerShell: how?

Question

I am trying to merge two hashtables, overwriting key-value pairs in the first if the same key exists in the second.

To do this I wrote this function which first removes all key-value pairs in the first hastable if the same key exists in the second hashtable.

When I type this into PowerShell line by line it works. But when I run the entire function, PowerShell asks me to provide (what it considers) missing parameters to foreach-object.

function mergehashtables($htold, $htnew)
{
    $htold.getenumerator() | foreach-object
    {
        $key = $_.key
        if ($htnew.containskey($key))
        {
            $htold.remove($key)
        }
    }
    $htnew = $htold + $htnew
    return $htnew
}

Output:

PS C:\> mergehashtables $ht $ht2

cmdlet ForEach-Object at command pipeline position 1
Supply values for the following parameters:
Process[0]:

$ht and $ht2 are hashtables containing two key-value pairs each, one of them with the key "name" in both hashtables.

What am I doing wrong?

Answer 1

I see two problems:

1. The open brace should be on the same line as Foreach-object
2. You shouldn't modify a collection while enumerating through a collection

The example below illustrates how to fix both issues:

function mergehashtables($htold, $htnew)
{
    $keys = $htold.getenumerator() | foreach-object {$_.key}
    $keys | foreach-object {
        $key = $_
        if ($htnew.containskey($key))
        {
            $htold.remove($key)
        }
    }
    $htnew = $htold + $htnew
    return $htnew
}

Answer 2

Merge-Hashtables

Instead of removing keys you might consider to simply overwrite them:

$h1 = @{a = 9; b = 8; c = 7}
$h2 = @{b = 6; c = 5; d = 4}
$h3 = @{c = 3; d = 2; e = 1}


Function Merge-Hashtables {
    $Output = @{}
    ForEach ($Hashtable in ($Input + $Args)) {
        If ($Hashtable -is [Hashtable]) {
            ForEach ($Key in $Hashtable.Keys) {$Output.$Key = $Hashtable.$Key}
        }
    }
    $Output
}

For this cmdlet you can use several syntaxes and you are not limited to two input tables: Using the pipeline: $h1, $h2, $h3 | Merge-Hashtables
Using arguments: Merge-Hashtables $h1 $h2 $h3
Or a combination: $h1 | Merge-Hashtables $h2 $h3
All above examples return the same hash table:

Name                           Value
----                           -----
e                              1
d                              2
b                              6
c                              3
a                              9

If there are any duplicate keys in the supplied hash tables, the value of the last hash table is taken.

---

(Added 2017-07-09)

Merge-Hashtables version 2

In general, I prefer more global functions which can be customized with parameters to specific needs as in the original question: "overwriting key-value pairs in the first if the same key exists in the second". Why letting the last one overrule and not the first? Why removing anything at all? Maybe someone else want to merge or join the values or get the largest value or just the average...
The version below does no longer support supplying hash tables as arguments (you can only pipe hash tables to the function) but has a parameter that lets you decide how to treat the value array in duplicate entries by operating the value array assigned to the hash key presented in the current object ($_).

Function

Function Merge-Hashtables([ScriptBlock]$Operator) {
    $Output = @{}
    ForEach ($Hashtable in $Input) {
        If ($Hashtable -is [Hashtable]) {
            ForEach ($Key in $Hashtable.Keys) {$Output.$Key = If ($Output.ContainsKey($Key)) {@($Output.$Key) + $Hashtable.$Key} Else  {$Hashtable.$Key}}
        }
    }
    If ($Operator) {ForEach ($Key in @($Output.Keys)) {$_ = @($Output.$Key); $Output.$Key = Invoke-Command $Operator}}
    $Output
}

Syntax

HashTable[] <Hashtables> | Merge-Hashtables [-Operator <ScriptBlock>]

Default By default, all values from duplicated hash table entries will added to an array:

PS C:\> $h1, $h2, $h3 | Merge-Hashtables

Name                           Value
----                           -----
e                              1
d                              {4, 2}
b                              {8, 6}
c                              {7, 5, 3}
a                              9

Examples To get the same result as version 1 (using the last values) use the command: $h1, $h2, $h3 | Merge-Hashtables {$_[-1]}. If you would like to use the first values instead, the command is: $h1, $h2, $h3 | Merge-Hashtables {$_[0]} or the largest values: $h1, $h2, $h3 | Merge-Hashtables {($_ | Measure-Object -Maximum).Maximum}.

More examples:

PS C:\> $h1, $h2, $h3 | Merge-Hashtables {($_ | Measure-Object -Average).Average} # Take the average values"

Name                           Value
----                           -----
e                              1
d                              3
b                              7
c                              5
a                              9


PS C:\> $h1, $h2, $h3 | Merge-Hashtables {$_ -Join ""} # Join the values together

Name                           Value
----                           -----
e                              1
d                              42
b                              86
c                              753
a                              9


PS C:\> $h1, $h2, $h3 | Merge-Hashtables {$_ | Sort-Object} # Sort the values list

Name                           Value
----                           -----
e                              1
d                              {2, 4}
b                              {6, 8}
c                              {3, 5, 7}
a                              9

Answer 3

Not a new answer, this is functionally the same as @Josh-Petitt with improvements.

In this answer:

• Merge-HashTable uses the correct PowerShell syntax if you want to drop this into a module
• Wasn't idempotent. I added cloning of the HashTable input, otherwise your input was clobbered, not an intention
• added a proper example of usage

function Merge-HashTable {
    param(
        [hashtable] $default, # Your original set
        [hashtable] $uppend # The set you want to update/append to the original set
    )

    # Clone for idempotence
    $default1 = $default.Clone();

    # We need to remove any key-value pairs in $default1 that we will
    # be replacing with key-value pairs from $uppend
    foreach ($key in $uppend.Keys) {
        if ($default1.ContainsKey($key)) {
            $default1.Remove($key);
        }
    }

    # Union both sets
    return $default1 + $uppend;
}

# Real-life example of dealing with IIS AppPool parameters
$defaults = @{
    enable32BitAppOnWin64 = $false;
    runtime = "v4.0";
    pipeline = 1;
    idleTimeout = "1.00:00:00";
} ;
$options1 = @{ pipeline = 0; };
$options2 = @{ enable32BitAppOnWin64 = $true; pipeline = 0; };

$results1 = Merge-HashTable -default $defaults -uppend $options1;
# Name                           Value
# ----                           -----
# enable32BitAppOnWin64          False
# runtime                        v4.0
# idleTimeout                    1.00:00:00
# pipeline                       0

$results2 = Merge-HashTable -default $defaults -uppend $options2;
# Name                           Value
# ----                           -----
# idleTimeout                    1.00:00:00
# runtime                        v4.0
# enable32BitAppOnWin64          True
# pipeline                       0

Answer 4

The open brace has to be on the same line as ForEach-Object or you have to use the line continuation character (backtick).

This is the case because the code within { ... } is really the value for the -Process parameter of ForEach-Object cmdlet.

-Process <ScriptBlock[]> 
Specifies the script block that is applied to each incoming object.

This will get you past the current issue at hand.

Answer 5

In case you want to merge the whole hashtable tree

function Join-HashTableTree {
    param (
        [Parameter(Mandatory = $true, ValueFromPipeline = $true)]
        [hashtable]
        $SourceHashtable,

        [Parameter(Mandatory = $true, Position = 0)]
        [hashtable]
        $JoinedHashtable
    )

    $output = $SourceHashtable.Clone()

    foreach ($key in $JoinedHashtable.Keys) {
        $oldValue = $output[$key]
        $newValue = $JoinedHashtable[$key]

        $output[$key] =
        if ($oldValue -is [hashtable] -and $newValue -is [hashtable]) { $oldValue | ~+ $newValue }
        elseif ($oldValue -is [array] -and $newValue -is [array]) { $oldValue + $newValue }
        else { $newValue }
    }

    $output;
}

Then, it can be used like this:

Set-Alias -Name '~+' -Value Join-HashTableTree -Option AllScope

@{
    a = 1;
    b = @{
        ba = 2;
        bb = 3
    };
    c = @{
        val = 'value1';
        arr = @(
            'Foo'
        )
    }
} |

~+ @{
    b = @{
        bb = 33;
        bc = 'hello'
    };
    c = @{
        arr = @(
            'Bar'
        )
    };
    d = @(
        42
    )
} |

ConvertTo-Json

It will produce the following output:

{
  "a": 1,
  "d": 42,
  "c": {
    "val": "value1",
    "arr": [
      "Foo",
      "Bar"
    ]
  },
  "b": {
    "bb": 33,
    "ba": 2,
    "bc": "hello"
  }
}

Answer 6

I wanted to point out that one should not reference base properties of the hashtable indiscriminately in generic functions, as they may have been overridden (or overloaded) by items of the hashtable.

For instance, the hashtable $hash=@{'keys'='lots of them'} will have the base hashtable property, Keys overridden by the item keys, and thus doing a foreach ($key in $hash.Keys) will instead enumerate the hashed item keys's value, instead of the base property Keys.

Instead the method GetEnumerator or the keys property of the PSBase property, which cannot be overridden, should be used in functions that may have no idea if the base properties have been overridden.

Thus, Jon Z's answer is the best.

Answer 7

I just wanted to expand or simplify on jon Z's answer. There just seems to be too many lines and missed opportunities to use Where-Object. Here is my simplified version:

Function merge_hashtables($htold, $htnew) {
    $htold.Keys | ? { $htnew.ContainsKey($_) } | % {
        $htold.Remove($_)
    }
    $htold += $htnew
    return $htold
}

Answer 8

To 'inherit' key-values from parent hashtable ($htOld) to child hashtables($htNew), without modifying values of already existing keys in the child hashtables,

function MergeHashtable($htOld, $htNew)
{
    $htOld.Keys | %{
        if (!$htNew.ContainsKey($_)) {
            $htNew[$_] = $htOld[$_];
        }
    }
    return $htNew;
}

Please note that this will modify the $htNew object.

Answer 9

Here is a function version that doesn't use the pipeline (not that the pipeline is bad, just another way to do it). It also returns a merged hashtable and leaves the original unchanged.

function MergeHashtable($a, $b)
{
    foreach ($k in $b.keys)
    {
        if ($a.containskey($k))
        {
            $a.remove($k)
        }
    }

    return $a + $b
}

Answer 10

I think the most compact code would be this:

function Merge-Hashtables($htold, $htnew)
{
   $htnew.keys | where {$_ -notin $htold.keys} | foreach {$htold[$_] = $htnew[$_]}
}

I borrowed it from Union and Intersection of Hashtables in PowerShell

Answer 11

I just needed to do this and found this works:

$HT += $HT2

The contents of $HT2 get added to the contents of $HT.