Passing a pointer to bufio.Scanner()

问题

Lest I provide an XY problem, my goal is to share a memory-mapped file between multiple goroutines as recommended. Each goroutine needs to iterate over the file line by line so I had hoped to store the complete contents in memory first to speed things up.

The method I tried is passing a pointer to a bufio.Scanner, but that is not working. I thought it might be related to needing to set the seek position back to the beginning of the file but it is not even working the very first time and I can find no such parameter in the documentation. My attempt was to create this function then pass the result by reference to the function I intend to run in a goroutine (for right now, I am not using goroutines just to make sure this works outright, which it does not).

Here is a MWE:

// ... package declaration; imports; yada yada

func main() {
    // ... validate path to file stored in filePath variable
    filePath := "/path/to/file.txt"

    // get word list scanner to be shared between goroutines
    scanner := getScannerPtr(&filePath)

    // pass to function (no goroutine for now, I try to solve one problem at a time)
    myfunc(scanner)
}

func getScannerPtr(filePath *string) *bufio.Scanner {
    f, err := os.Open(*filePath)
    if err != nil {
        fmt.Fprint(os.Stderr, "Error opening file\n")
        panic(err)
    }
    defer f.Close()
    scanner := bufio.NewScanner(f)
    scanner.Split(bufio.ScanLines)
    return scanner
}

func myfunc(scanner *bufio.Scanner) {
    for scanner.Scan() {
        line := strings.TrimSpace(scanner.Text())
        // ... do something with line
    }
}

I'm not receiving any errors, it just is not iterating over the file when I call Scan() so it never makes it inside that block to do anything with each line of the file. Keep in mind I am not even using concurrency yet, that is just my eventual goal which I want to point out in case that impacts the method I need to take.

Why is Scan() not working?
Is this is a viable approach if I intend to call go myfunc(scanner) in the future?

回答1:

You're closing the file before you ever use the Scanner:

func getScannerPtr(filePath *string) *bufio.Scanner {
    f, err := os.Open(*filePath)
    if err != nil {
        fmt.Fprint(os.Stderr, "Error opening file\n")
        panic(err)
    }
    defer f.Close() // <--- Here
    scanner := bufio.NewScanner(f)
    scanner.Split(bufio.ScanLines)
    return scanner // <-- File gets closed, then Scanner that tries to read it is returned for further use, which won't work
}

Because Scanner does not expose Close, you'll need to work around this; the quickest is probably to make a simple custom type with a couple of embedded fields:

type FileScanner struct {
    io.Closer
    *bufio.Scanner
}

func getScannerPtr(filePath *string) *FileScanner  {
    f, err := os.Open(*filePath)
    if err != nil {
        fmt.Fprint(os.Stderr, "Error opening file\n")
        panic(err)
    }
    scanner := bufio.NewScanner(f)
    return &FileScanner{f, scanner}
}

func myfunc(scanner *FileScanner) {
    defer scanner.Close()
    for scanner.Scan() {
        line := strings.TrimSpace(scanner.Text())
        // ... do something with line
    }
}

来源：https://stackoverflow.com/questions/48026422/passing-a-pointer-to-bufio-scanner

标签

concurrency

memory-mapped-files