spark Athena connector

Question

I need to use Athena in spark but spark uses preparedStatement when using JDBC drivers and it gives me an exception \"com.amazonaws.athena.jdbc.NotImplementedException: Meth

Answer 1

You can't directly connect Spark to Athena. Athena is simply an implementation of Prestodb targeting s3. Unlike Presto, Athena cannot target data on HDFS.

However, if you want to use Spark to query data in s3, then you are in luck with HUE, which will let you query data in s3 from Spark on Elastic Map Reduce (EMR).

See Also: Developer Guide for Hadoop User Experience (HUE) on EMR.