Why Spark 1.6 does not use Akka?

Question

When I read spark-1.6 source code of the Master class, the receiveAndReply method seems not to be using Akka. [Cf. here.]

Why is it not using Akka ? And What did the

Answer 1

The motivation behind making Spark independent from Akka are well described in SPARK-5293 which is an umbrella task for Akka related issues.

To quote original description:

Spark depends on Akka, [so] it is not possible for users to rely on different versions, and we have received many requests in the past asking for help about this specific issue. For example, Spark Streaming might be used as the receiver of Akka messages - but our dependency on Akka requires the upstream Akka actors to also use the identical version of Akka.

Since our usage of Akka is limited (mainly for RPC and single-threaded event loop), we can replace it with alternative RPC implementations and a common event loop in Spark.

As you can see, the main reason is simple - to give users more flexibility in creating their own applications.

Also removing complex dependency like Akka, which hasn't been used extensively by Spark anyway, which means lower cost of maintenance.