Writing to an appengine blob asynchronously and finalizing it when all tasks complete

后端 未结 3 745
自闭症患者
自闭症患者 2021-01-07 08:06

I have a difficult problem.

I am iterating through a set of URLs parameterized by date and fetching them. For example, here is an example of one:

somewebserv

3条回答
  •  予麋鹿
    予麋鹿 (楼主)
    2021-01-07 09:05

    All right, so here's what I did. I had to modify Mitch's solution just a bit, but he definitely got me in the right direction with the advice to return the future value instead of an immediate one.

    I had to create an intermidate DummyJob that takes the output of the recursion

       public static class DummyJob extends Job1> {
          @Override
          public Value run(List dummies) {
             return null;
          }
       }
    

    Then, I submit the output of the DummyJob to the Blob Finalizer in a waitFor

    List> dummies = new ArrayList>();
    for (Interval in : ins) {
       dummies.add(futureCall(new DataFetcher(), immediate(file), immediate(in.getStart()),
             immediate(in.getEnd())));
    }
    
    FutureValue fv = futureCall(new DummyJob(), futureList(dummies));
    
    return futureCall(new DataWriter(), immediate(file), waitFor(fv));
    

    Thank you Mitch and Nick!!

提交回复
热议问题