Go all goroutines are asleep deadlock

问题

Can't seem to figure out why I'm getting the error message: fatal error: all goroutines are asleep - deadlock!.

I suspected a race condition was occurring in my block below which should only execute after the channel is closed.

I thought adding a sync WaitGroup would help, but it's only given me this deadlock. What I have looks close to the samples I've seen online, so I'm just not sure what's wrong here.

func S3UploadFolder(instance *confighelper.Instance, sess *session.Session, 
    srcFolder string, bucketName string) (err error) {

    log.Println("S3UploadFolder", srcFolder, bucketName)

    wg := &sync.WaitGroup{}

    // find files recursively
    walker := make(fileWalk)
    wg.Add(1)

    go func() {

        // Gather the files to upload by walking the path recursively
        if err := filepath.Walk(srcFolder, walker.Walk); err != nil {
            log.Fatalln("Walk failed:", err)
        }
        wg.Done()
        close(walker)

    }()
    wg.Wait()

    for path := range walker {
    // THE GO routine above needs to have finished by the time this for loop 
       // ranges over the channel
         fmt.Println(path)

 }



return
}


type fileWalk chan string

func (f fileWalk) Walk(path string, info os.FileInfo, err error) error {
    if err != nil {
        return err
    }
    if !info.IsDir() {
        f <- path
    }
    return nil
}

回答1:

The walker channel is unbuffered. Communication on an unbuffered channel does not proceed until until a sender and receiver are ready.

The deadlock is this: The main goroutine waits on the walker goroutine to complete by calling wg.Done(). The walker goroutine waits on the main goroutine to receive on the channel.

Fix the program by removing all code related to the wait group. The wait group is not needed. Range over the channel in the main goroutine does not complete until the channel is closed by the walker goroutine. The walker goroutine does not close the channel until the walk is complete. No other coordination is required.

You can also fix the code by removing the goroutines and channels:

err := filepath.Walk(srcFolder, func(path string, info os.FileInfo, err error) error {
    if err != nil {
        return err
    }
    if info.IsDir() {
        return nil
    }

    // Insert body of for path := range walker here ... 
    fmt.Println(path)

    return nil
})
if err != nil {
    log.Fatal(err)
}

Another option is to create a buffered channel with capacity greater than the number of files that will be walked, but this requires knowing the number of files in advance and provides no benefit over collecting the file names in a slice.

回答2:

As written (and shown), you definitely must not call wg.Wait() before running your for path := range walker loop. You may (but don't need to) call wg.Wait() when the loop terminates. You don't need the wg variable at all.

Your comment says:

// THE GO routine above needs to have finished by the time this for loop 
// ranges over the channel

but there is nothing in the for loop that requires the function to be finished, and there is something—the overall strategy here—that requires that the goroutine not be blocked in a send, as the for loop will only finish when the sender—the goroutine—closes the channel.

(See Cerise Limón's answer for why the goroutine becomes blocked in the send.)

来源：https://stackoverflow.com/questions/57403500/go-all-goroutines-are-asleep-deadlock

标签