How to join two text files, removing duplicates, in Windows

Question

file 1

A
B
C

file 2

B
C
D

file1 + file2 =

A
B
C
D

Is it possible to do

Answer 1

You may also use the same approach of Unix or PowerShell with pure Batch, developing a simple uniq.bat filter program:

@echo off
setlocal EnableDelayedExpansion
set "prevLine="
for /F "delims=" %%a in ('findstr "^"') do (
   if "%%a" neq "!prevLine!" (
      echo %%a
      set "prevLine=%%a"
   )
)

EDIT: The program below is a Batch-JScript hybrid version of uniq program, more reliable and faster; copy this program in a file called uniq.bat:

@if (@CodeSection == @Batch) @then

@CScript //nologo //E:JScript "%~F0" & goto :EOF

@end

var line, prevLine = "";
while ( ! WScript.Stdin.AtEndOfStream ) {
   line = WScript.Stdin.ReadLine();
   if ( line != prevLine ) {
      WScript.Stdout.WriteLine(line);
      prevLine = line;
   }
}

This way, you may use this solution:

(type file1.txt & type file2.txt) | sort | uniq > result.txt

However, in this case the result lost the original order.