What is the neatest idiom for producer/consumer in Go?

Question

What I would like to do is have a set of producer goroutines (of which some may or may not complete) and a consumer routine. The issue is with that caveat in parentheses - we do

Answer 1

You can use simple unbuffered channels without wait groups if you use the generator pattern with a fanIn function.

In the generator pattern, each producer returns a channel and is responsible for closing it. A fanIn function then iterates over these channels and forwards the values returned on them down a single channel that it returns.

The problem of course, is that the fanIn function forwards the zero value of the channel type (int) when each channel is closed.

You can work around it by using the zero value of your channel type as a sentinel value and only using the results from the fanIn channel if they are not the zero value.

Here's an example:

package main

import (
    "fmt"
    "math/rand"
)

const offset = 1

func producer() chan int {
    cout := make(chan int)
    go func() {
        defer close(cout)
        // May or may not produce.
        success := rand.Float32() > 0.5
        if success {
            cout <- rand.Int() + offset
        }
    }()
    return cout
}

func fanIn(cin []chan int) chan int {
    cout := make(chan int)
    go func() {
        defer close(cout)
        for _, c := range cin {
            cout <- <-c
        }
    }()
    return cout
}

func main() {
    chans := make([]chan int, 0)
    for i := 0; i < 10; i++ {
        chans = append(chans, producer())
    }

    for num := range fanIn(chans) {
        if num > offset {
            fmt.Printf("Producer produced: %d\n", num)
        }
    }
    fmt.Println("All done.")
}