How to perform join on MySQL (JDBC) with Spark?

问题

I would like to read data from MySQL through Spark. The API which I saw is able to read data from specific table. something like,

val prop = new java.util.Properties
prop.setProperty("user", "<username>")
prop.setProperty("password", "<password>")

sparkSession.read.jdbc("jdbc:mysql://????:3306/???", "some-table", prop)

Now, I would like to perform a query for join tables. Does anyone know how to do it (on the database side, not with Spark SQL) ?

Thanks,

Eran

回答1:

You'll need to use the "table " argument as a query:

val table = "(SELECT foo JOIN bar ON foo.id = bar.id) as t"

spark.read.jdbc("jdbc:mysql://????:3306/???", table, prop)

You should note that giving an alias to your query is important or this won't work.

来源：https://stackoverflow.com/questions/40720619/how-to-perform-join-on-mysql-jdbc-with-spark