Golang: How do I determine the number of lines in a file efficiently?

Question

In Golang, I am looking for an efficient way to determine the number of lines a file has.

Of course, I can always loop through the entire file, but does not seem ver

Answer 1

The most efficient way I found is using IndexByte of the byte packet, it is at least four times faster than using bytes.Count and depending on the size of the buffer it uses much less memory.

func LineCounter(r io.Reader) (int, error) {

    var count int
    const lineBreak = '\n'

    buf := make([]byte, bufio.MaxScanTokenSize)

    for {
        bufferSize, err := r.Read(buf)
        if err != nil && err != io.EOF {
            return 0, err
        }

        var buffPosition int
        for {
            i := bytes.IndexByte(buf[buffPosition:], lineBreak)
            if i == -1 || bufferSize == buffPosition {
                break
            }
            buffPosition += i + 1
            count++
        }
        if err == io.EOF {
            break
        }
    }

    return count, nil
}

Benchmark

BenchmarkIndexByteWithBuffer  2000000          653 ns/op        1024 B/op          1 allocs/op
BenchmarkBytes32k             500000          3189 ns/op       32768 B/op          1 allocs/op