Using Windows/DOS shell/batch commands, how do I take a file and only keep unique lines?

Question

Say I have a file like:

apple
pear
lemon
lemon
pear
orange
lemon

How do I make it so that I only keep the unique lines, so I get:

Answer 1

There's no easy way to do that from the command line without an additional program.

uniq will do what you want.

Or you can download CoreUtils for Windows to get GNU tools. Then you can just use sort -u to get what you want.

Either one of those should be callable from a batch file.

Personally though, if you need to do a lot text manipulation like that I think you'd be better off getting Cygwin. Then you'd have easy access to sort, sed, awk, vim, etc.

Answer 2

Run PowerShell from the command prompt.

Assuming the items are in a file call fruits.txt, the following will put the unique lines in uniques.txt:

type fruits.txt | Sort-Object -unique | Out-File uniques.txt