(Hadoop) MapReduce - Chain jobs - JobControl doesn't stop

前端 未结 4 1174
情深已故
情深已故 2020-12-29 13:40

I need to chain two MapReduce jobs. I used JobControl to set job2 as dependent of job1. It works, output files are created!! But it doesn\'t stop! In the shell it remains in

4条回答
  •  孤独总比滥情好
    2020-12-29 13:52

    I essentially did what Pietro alluded to above.

    public class JobRunner implements Runnable {
      private JobControl control;
    
      public JobRunner(JobControl _control) {
        this.control = _control;
      }
    
      public void run() {
        this.control.run();
      }
    }
    

    and in my map/reduce class I have:

    public void handleRun(JobControl control) throws InterruptedException {
        JobRunner runner = new JobRunner(control);
        Thread t = new Thread(runner);
        t.start();
    
        while (!control.allFinished()) {
            System.out.println("Still running...");
            Thread.sleep(5000);
        }
    }
    

    in which I just pass the jobControl object.

提交回复
热议问题