Powershell - Removing all duplicate entries

跟風遠走 提交于 2019-12-08 19:21:23

PSv2:

$f = 'C:\Temp\OriginalFile.txt'

Get-Content $f | Group-Object | ? { $_.Count -eq 1 } | Select-Object -ExpandProperty Name

PSv3+ allows for a cleaner and more concise solution:

Get-Content $f | Group-Object | ? Count -eq 1 | % Name

For brevity, the commands use built-in aliases ? (for Where-Object) and % (for ForEach-Object).

Neither Select-Object -Unique nor Get-Unique seemingly allow restricting the output to singletons in the input (standard Unix utility uniq has such a feature built in: uniq -u), so a different approach is needed.

The above Group-Object-based solution may not be efficient, but it is convenient:

  • lines are grouped by identical content, yielding objects that represent each group.

  • ? { $_.Count -eq 1 } the filters the groups down to those that have just 1 member, in effect weeding out all duplicate lines.

  • Select-Object -ExpandProperty Name then transforms the filtered group objects back to the input line they represent.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!