Futures somehow slower then agents?

问题

The following code does essentially just let you execute something like (function (range n)) in parallel.

(experiment-with-agents 10000 10 #(filter prime? %))

This for example finds the prime numbers between 0 and 10000 with 10 agents.

(experiment-with-futures 10000 10 #(filter prime? %))

Same just with futures.

Now the problem is that the solution with futures doesn't run faster with more futures. Example:

; Futures
(time (experiment-with-futures 10000 1 #(filter prime? %)))
"Elapsed time: 33417.524634 msecs"

(time (experiment-with-futures 10000 10 #(filter prime? %)))
"Elapsed time: 33891.495702 msecs"

; Agents
(time (experiment-with-agents 10000 1 #(filter prime? %)))
"Elapsed time: 33048.80492 msecs"

(time (experiment-with-agents 10000 10 #(filter prime? %)))
"Elapsed time: 9211.864133 msecs"

Why? Did I do something wrong (probably, new to Clojure and just playing around with stuff^^)? Because I thought that futures are actually prefered in that scenario.

Source:

(defn setup-agents
  [coll-size num-agents]
  (let [step (/ coll-size num-agents)
        parts (partition step (range coll-size))
        agents (for [_ (range num-agents)] (agent []) )
        vect (map #(into [] [%1 %2]) agents parts)]
    (vec vect)))

(defn start-agents
  [coll f]
  (for [[agent part] coll] (send agent into (f part))))

(defn results
  [agents]
  (apply await agents)
  (vec (flatten (map deref agents))))

(defn experiment-with-agents
  [coll-size num-agents f]
  (-> (setup-agents coll-size num-agents)
      (start-agents f)
      (results)))

(defn experiment-with-futures
  [coll-size num-futures f]
  (let [step (/ coll-size num-futures)
        parts (partition step (range coll-size))
        futures (for [index (range num-futures)] (future (f (nth parts index))))]
    (vec (flatten (map deref futures)))))

回答1:

You're getting tripped up by the fact that for produces a lazy sequence inside of experiment-with-futures. In particular, this piece of code:

(for [index (range num-futures)] (future (f (nth parts index))))

does not immediately create all of the futures; it returns a lazy sequence that will not create the futures until the contents of the sequence are realized. The code that realizes the lazy sequence is:

(vec (flatten (map deref futures)))

Here, map returns a lazy sequence of the dereferenced future results, backed by the lazy sequence of futures. As vec consumes results from the sequence produced by map, each new future is not submitted for processing until the previous one completes.

To get parallel processing, you need to not create the futures lazily. Try wrapping the for loop where you create the futures inside a doall.

The reason you're seeing an improvement with agents is the call to (apply await agents) immediately before you gather the agent results. Your start-agents function also returns a lazy sequence and does not actually dispatch the agent actions. An implementation detail of apply is that it completely realizes small sequences (under 20 items or so) passed to it. A side effect of passing agents to apply is that the sequence is realized and all agent actions are dispatched before it is handed off to await.

来源：https://stackoverflow.com/questions/25874868/futures-somehow-slower-then-agents

标签

clojure