(Hadoop) MapReduce - Chain jobs - JobControl doesn't stop

前端 未结 4 1175
情深已故
情深已故 2020-12-29 13:40

I need to chain two MapReduce jobs. I used JobControl to set job2 as dependent of job1. It works, output files are created!! But it doesn\'t stop! In the shell it remains in

4条回答
  •  我在风中等你
    2020-12-29 14:09

    Just a tweak to the code snippet what sinemetu1 had shared..

    You can drop call to the JobRunner as JobControl by itself implements Runnable

            Thread thread = new Thread(jobControl);
            thread.start();
    
            while (!jobControl.allFinished()) {
                System.out.println("Still running...");
                Thread.sleep(5000);
            }
    

    I also stumbled upon this link where the user confirms that JobControl can be run ONLY with new thread. https://www.mail-archive.com/common-user@hadoop.apache.org/msg00556.html

提交回复
热议问题