Quickly find differences between two large text files

Question

I have two 3GB text files, each file has around 80 million lines. And they share 99.9% identical lines (file A has 60,000 unique lines, file B has 80,000 unique lines).

Answer 1

If I understand correctly, you want the lines of these files without duplicates. This does the job:

uniqA = set(open('fileA', 'r'))