How to run a Spark test from IntelliJ (or other IDE)

问题

I am trying to create a Test for some Spark code. The following code fails when getting a SparkSession object. NOTE: The test runs fine when running from the cli: gradle my_module:build

@Test
def myTest(): Unit = {
  val spark = SparkSession.builder().master("local[2]").getOrCreate()
  ...
}

Error:

java.lang.IllegalArgumentException: Can't get Kerberos realm
...
Caused by: java.lang.reflect.InvocationTargetException
...
Caused by: KrbException: Cannot locate default realm

My set-up: IntelliJ + Gradle + Mac OS

Questions:

• How do I run a Spark Test from within IntelliJ?
• Why is Spark looking for Kerberos at all when running 'local'

回答1:

By your code you need to run Spark from JUnit, not specifically from IntelliJ, you can try something like https://github.com/sleberknight/sparkjava-testing

来源：https://stackoverflow.com/questions/43398058/how-to-run-a-spark-test-from-intellij-or-other-ide