Detected Guava issue #1635 which indicates that a version of Guava less than 16.01 is in use

Question

I am running spark job on emr and using datastax connector to connect to cassandra cluster. I am facing issues with the guava jar please find the details as below I am using

Answer 1

I was facing the the same issue while retrieving records from Cassandra table using Spark (java) on Spark submit.

Please check your guava jar version used by Hadoop and Spark in cluster using find command and change it accordingly.

find / -name "guav*.jar"

Otherwise add guava jar externally during spark-submit for driver and executer spark.driver.extraClassPath and spark.executor.extraClassPath respectively.

spark-submit --class com.my.spark.MySparkJob --master local --conf 'spark.yarn.executor.memoryOverhead=2048' --conf 'spark.cassandra.input.consistency.level=ONE' --conf 'spark.cassandra.output.consistency.level=ONE' --conf 'spark.dynamicAllocation.enabled=false' --conf "spark.driver.extraClassPath=lib/guava-19.0.jar" --conf "spark.executor.extraClassPath=lib/guava-19.0.jar" --total-executor-cores 15 --executor-memory 15g  --jars $(echo lib/*.jar | tr ' ' ',') target/my-sparkapp.jar

It's working for me. Hope you can try it.