grep a large list against a large file

前端 未结 4 875
天命终不由人
天命终不由人 2020-12-07 17:09

I am currently trying to grep a large list of ids (~5000) against an even larger csv file (3.000.000 lines).

I want all the csv lines, that contain an i

4条回答
  •  谎友^
    谎友^ (楼主)
    2020-12-07 17:52

    You may get a significant search speedup with ugrep to match the strings in the_ids.txt in your large huge.csv file:

    ugrep -F -f the_ids.txt huge.csv
    

    This works with GNU grep too, but I expect ugrep to run several times faster.

提交回复
热议问题