How can I use parMap with a monadic function?

Question

I have a monadic function getRate:

getRate :: String -> IO Double

I'd like to map this function over a list of String's. Normally, I would just do:

mapM getRate ["foo", "bar"]

but since each call to getRate makes network calls, I'd like to parallelize the map so that each rate is fetched in a separate thread (or at least spread out among queues). I'm thinking of something like

parMapM getRate ["foo", "bar"]

but there is no parMapM function and parMap doesn't work with monadic functions.

What can I do?

Answer 1

You should use Control.Concurrent and synchronize around a Control.Concurrent.MVar; something like:

fork1 :: (a -> IO b) -> a -> IO (MVar b)
fork1 f x =
  do
    cell <- newEmptyMVar
    forkIO (do { result <- f x; putMVar cell result })
    return cell

fork :: (a -> IO b) -> [a] -> IO [MVar b]
fork f = mapM (fork1 f)

join :: [MVar b] -> IO [b]
join = mapM takeMVar

forkJoin :: (a -> IO b) -> [a] -> IO [b]
forkJoin f xs = (fork f xs) >>= join

Parts of this (fork, join) look sequential. What's happening in practice is the threads are fired off sequentially in fork and rendezvous walks through waiting for each thread in turn. But the IO happens concurrently.

Note that if you need to call foreign functions you should use forkOS instead of forkIO.

Answer 2

There is also a monad-parallel package which provides mapM :: MonadParallel m => (a -> m b) -> [a] -> m [b]. Looking at the IO instance for MonadParallel it does it the same way as in Dominic's answer.