akka-stream Zipping Flows with SubFlows

拥有回忆 提交于 2019-12-23 12:28:46

问题


I've a short question about akka-streams. Basically, I try to split a stream into two streams, one of these two streams will be split again in multiple subFlows using groupBy, each of these subFlows needs to be connected with the other stream (zip). I tried to illustrate this here:

Here is what I got so far

 val aggFlow = Flow.fromGraph(GraphDSL.create() { implicit builder =>

        val broadcast = builder.add(Broadcast[Event](2))
        val zip = builder
            .add(ZipWith[ChangedEvent, Long, (ChangedEvent, Long)]((value, key) => (value, key)))

        val source = broadcast.out(1) ~> identityFlow ~> maxFlow

        broadcast.out(0) ~> 
        identityFlow ~> 
        topicFlow
            .groupBy(MAX_SUB_STREAMS, _._1)
            .zipWith[ChangedEvent, (Long, ChangedEvent)](source)((max, ce) => (max, sm))
            .takeWhile(deciderFunction)
            .mergeSubstreams

        ???
      })

I'm facing the problem that broadcast.out(1) ~> identityFlow ~> maxFlow doesn't return a source but is there are way to get a source from that or use zipWith with two flows? Or even a better approach for getting the same result?

Further information: I have a stream (endless) with events (some Information and timestamp). I want to aggregate them based on the timestamp (I generate a "topic" from the timestamp). So Events are grouped based on their timestamp using the groupBy function. Then I want to fold them. The Problem is due to the fact that it is a (endless) stream the subStreams/subFlows created by the groupBy function will never close. Therefore, I want to split the stream. One Part of the stream will be used to generate Events that can be used by deciderFunction. The over part of will be used for creating the subStreams/SubFlows. I hope this helps to understand what I am trying to do.

来源:https://stackoverflow.com/questions/36862520/akka-stream-zipping-flows-with-subflows

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!