Powershell - Export-CSV and Append

Question

I have a script such as the following:

$in_file = "C:\Data\Need-Info.csv"
$out_file = "C:\Data\Need-Info_Updated.csv"
$list = Import-Csv $in_file 
ForEach ( $user in $list ) {
$zID = $user.zID
ForEach-Object { Get-QADUser -Service 'domain.local' -SearchRoot 'OU=Users,DC=domain,DC=local' -SizeLimit 75000 -LdapFilter "(&(objectCategory=person)(objectClass=user)(PersonzID=$zID))" |
Select-Object DisplayName,samAccountName,@{Name="zID";expression={$zID}} | 
Export-Csv $out_file -NoTypeInformation -Force
}
}

However, I am not able to get it to output all of the results to the $out_file since it does not seem to append the data to the csv file. I have tried a number of things, ideas, and suggestions found online. None of them seem to work and I feel like I am missing something rather easy here. Is there a way to make this append the data to a file? If so, could you provide an example or a snippet of code to make it work in that manner? I am not asking for a rewrite of all of the code or anything along those lines, just a way to make the data that is being exported appended to the file that is output.

Answer 1

Convert the foreach loop into a foreach-object and move the export-csv to outside the outter foreach object so that you can pipe all the objects to the export-csv.

Something like this (untested):

$list | ForEach {
$zID = $_.zID
ForEach-Object { Get-QADUser -Service 'domain.local' -SearchRoot 'OU=Users,DC=domain,DC=local' -SizeLimit 75000 -LdapFilter "(&(objectCategory=person)(objectClass=user)(PersonzID=$zID))" |
Select-Object DisplayName,samAccountName,@{Name="zID";expression={$zID}} | 

}
} |Export-Csv $out_file -NoTypeInformation -Force

Answer 2

-append is broken in Powershell v2.0 Use Dmitry Sotikovs workaound: http://dmitrysotnikov.wordpress.com/2010/01/19/export-csv-append/

I would however recommend manojlds excellent solution!

Answer 3

As Sune mentioned, PowerShell v3's Export-Csv has an Append flag but no character encoding protection. manojlds is correct, since your code is writing all new data to a new CSV file.

Meanwhile, you can append data to a CSV by:

1. Convert the objects to CSV with ConvertTo-Csv
2. Strip the header —and type information if necessary— and collect the CSV data only
3. Append the new CSV data to the CSV file through Add-Content or Out-File, be sure to use same character encoding

Here is a sample:

1..3 | ForEach-Object {
 New-Object PSObject -Property @{Number = $_; Cubed = $_ * $_ * $_}
} | Export-Csv -Path .\NumTest.csv -NoTypeInformation -Encoding UTF8

# create new data
$newData = 4..5 | ForEach-Object {
 New-Object PSObject -Property @{Number = $_; Cubed = $_ * $_ * $_}
} | ConvertTo-Csv -NoTypeInformation

# strip header (1st element) by assigning it to Null and collect new data
$null, $justData = $newData

# append just the new data
Add-Content -Path .\NumTest.csv -Value $justData -Encoding UTF8

# create more new data, strip header and collect just data
$null, $data = 6..9 | ForEach-Object {
 New-Object PSObject -Property @{Number = $_; Cubed = $_ * $_ * $_}
} | ConvertTo-Csv -NoTypeInformation

# append the new data
Add-Content -Path .\NumTest.csv -Value $data -Encoding UTF8

# verify
Import-Csv .\NumTest.csv

# clean up
Remove-Item .\NumTest.csv

Answer 4

After version 3 you can use:

Export-Csv $out_file -append -notypeinformation -encoding "unicode"