How to simplify complex parallel branch interdependencies for Step Functions

时间秒杀一切 提交于 2020-03-19 05:15:46

问题


I have the task of translating a list of dependent nodes into AWS Step Functions. The AWS Step Function definition allows for parallel branches or even branches nested to multiple levels deep. Unfortunately it does not support dependencies between tasks in the branches and therefore forces you to complete the parallel step before both results are available to subsequent tasks in the step function.

In my diagram delow a simple parallel branch like shown in Graph 1 is easily supported by Step Functions.

When it comes to Graph 2 and especially Graph 3 it becomes a problem.

As a simple approach we could introduce additional nodes to collect the results together for their dependent nodes as demonstrated in Graphs 2b and 3b but this now introduces dependencies that didn't exist before:

  • In Graph 2b these new dependencies have been introduced:
    • E -> A, F -> A
  • In Graph 3b these new dependencies have been introduced:
    • E -> A, F -> A, F-> B, C -> E

This is a problem because in the case of manual approval tasks the time for these tasks could be in the order of hours to days. This would cause later steps to be delayed unnecessarily by tasks that they do not have dependencies on.

Any suggestions on how to solve this? Maybe I could take a different approach? Maybe there is some fancy graph theory algorithm I can apply? I don't even know what words to use to explain this problem in graph theory.

Here is a url for playing with these graphs on draw.io if you need to.


回答1:


Fundamentally what you asked is a DAG while the Amazon States Language is state machine based. So I don't think there is a solution for your problem.



来源:https://stackoverflow.com/questions/57221589/how-to-simplify-complex-parallel-branch-interdependencies-for-step-functions

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!