spark ssc.textFileStream is not streamining any files from directory

Question

I am trying to execute below code using eclipse (with maven conf) with 2 worker and each have 2 core or also tried with spark-submit.

public class StreamingW         


        

Answer 1

had the same problem. Here is my code:

lines = jssc.textFileStream("file:///Users/projects/spark/test/data');

the TextFileSTream is very sensitive; what i ended up doing was:

1. Run Spark program
2. touch datafile
3. mv datafile datafile2
4. mv datafile2  /Users/projects/spark/test/data

and that did it.