Trigger a spark job whenever an event occurs
问题 I have a spark application which should run whenever it receives a kafka message on a topic. I won't be receiving more than 5-6 messages a day so I don't want to take spark streaming approach. Instead I tried to submit the application using SparkLauncher but I don't like the approach as I have to set spark and Java classpath programmatically within my code along with all the necessary spark properties like executor cores, executor memory etc. How do I trigger the spark application to run from