Fastest Text search method in a large text file

前端 未结 4 1462
南旧
南旧 2021-01-19 04:35

I am doing a text search in a rather big txt file (100k lines, 7mo) Text is not that big but I need a lot of searches. I want to look for a target string and return the line

4条回答
  •  自闭症患者
    2021-01-19 05:04

    First, don't explicitly decode bytes.

    from io import open
    

    Second, consider things like this.

    with open(path,'r',encoding='UTF-8') as src:
        found= None
        for line in src:
            if len(line) == 0: break #happens at end of file, then stop loop
            if target in line:
                found= line
                break
        return found
    

    This can be simplified slightly to use return None or return line instead of break. It should run a hair faster, but it's slightly harder to make changes when there are multiple returns.

提交回复
热议问题