akka-stream

Send big file over reactive stream

廉价感情. 提交于 2019-12-12 16:34:17
问题 Part of application I am writing requires transferring arbitrarily big (for this question I will assume 100-200 GB) files from client to server. Important thing, is that receiver (server) is not storing this file - it just read/examine stream and sends it to next point. Because at no point I need whole file, but expect multiple transfers at same time, I would like to minimize RAM usage and eliminate disk usage. I would like to process files in chunks of 1 MB. Right now, server uses Spring

How to group incoming events from infinite stream?

本小妞迷上赌 提交于 2019-12-12 10:56:24
问题 I have an infinite stream of events: (timestamp, session_uid, traffic) i.e. ... (1448089943, session-1, 10) (1448089944, session-1, 20) (1448089945, session-2, 50) (1448089946, session-1, 30) (1448089947, session-2, 10) (1448089948, session-3, 10) ... These events I want to group by session_uid and calculate sum of traffic for each session. I wrote an akka-streams flow which works fine with finite stream use groupBy (my code base on this example from cookbook). But with infinite stream it

BroadcastHub filtering based on “resource” the connected client is working on?

有些话、适合烂在心里 提交于 2019-12-12 08:48:29
问题 I am writing a pure websocket web application, meaning that prior to the websocket upgrade there is no user/client step, more specifically: Authentication request goes over websockets as does the rest of the communication There is/are: Exactly ONE websocket endpoint on /api/ws Multiple clients connected to that endpoint Multiple projects for multiple clients Now, not each client has access to each project - the access control for that is implemented on the server side (ofc) and has nothing to

Akka Stream - Select Sink based on Element in Flow

时间秒杀一切 提交于 2019-12-12 07:36:52
问题 I'm creating a simple message delivery service using Akka stream. The service is just like mail delivery, where elements from source include destination and content like: case class Message(destination: String, content: String) and the service should deliver the messages to appropriate sink based on the destination field. I created a DeliverySink class to let it have a name: case class DeliverySink(name: String, sink: Sink[String, Future[Done]]) Now, I instantiated two DeliverySink , let me

Loop in flow makes stream never end

你说的曾经没有我的故事 提交于 2019-12-12 05:08:26
问题 I have properly working graph that has flow with loop. Items go through as expected and everything is working. But unfortunately with bounded source graph never ends. Ever. How can I fix it ? Here is schema of my flow. Here is simplified version of the flow with the same topology. val badFlow = Flow.fromGraph(GraphDSL.create() { implicit builder => import GraphDSL.Implicits._ val mergeEntrance = builder.add(MergePreferred[Int](1)) val mergePreExit = builder.add(Merge[Int](2)) val part1 =

Named event stream with Play 2.5 and server sent events

二次信任 提交于 2019-12-12 04:02:54
问题 Default event name/type in server sent event is "message". I am trying to change the event name but it is not working. I am using Play 2.5 and akka streams. (actorRef,sourcePublisher)= Source .actorRef[T](10, OverflowStrategy.fail) .toMat(Sink.asPublisher(true))(Keep.both) .run() backsource = Source.fromPublisher[T](sourcePublisher).named("test1") Ok.chunked(backsource via EventSource.flow) .as(ContentTypes.EVENT_STREAM) But it is not changing the event name/type. It is still listening to

Play Framework 2.5 Streaming content with delay

旧街凉风 提交于 2019-12-12 01:57:08
问题 I have been trying to stream content through HTTP with Play Framework 2.5 in Java with a delay. The problem is that I am not sure if the result is actually streaming, that is why I tried to delay each item from emitting which does not seem to work for some reason. The code public Result test(){ HttpEntity http = new HttpEntity.Streamed(Source.range(0, 99999) .map(i -> ByteString.fromString(i.toString())) .initialDelay(FiniteDuration.create(200, TimeUnit.MILLISECONDS)) , Optional.empty(),

Modify via to accept method returning a Future

核能气质少年 提交于 2019-12-11 18:34:47
问题 I am calling a method using via like below: myRawStr(id) .take(1) .via(myMethod("someString", someSource) .zip(Source.fromIterator(() => Iterator.from(1))) .collect { ... } myMethod returns type Flow[ByteString, MyValidated[MyClass], NotUsed] but now it will be returning Future[Flow[ByteString, MyValidated[MyClass], NotUsed]] ( Note : Future) but doing this gives me compilation error on via . The error states: [error] found : [as, mat, ec]scala.concurrent.Future[akka.stream.scaladsl.Flow[akka

Play framework How to use akka streams output to a websocket

前提是你 提交于 2019-12-11 15:24:14
问题 Whats the easiest way to have the output of a flow be sent to a web socket in Playframework with Scala. I want the WebSocket to act as the sink of the stream. For example the source generate a random number and then go to the sink (Websocket) so it is pushed to client. Thanks 回答1: If you are talking about the new Akka streams integration in Play 2.5.x (still M2), then I think this should do the job as a simple valid flow that represents an echo server (with a touch). def socket = WebSocket

Akka stream batching

折月煮酒 提交于 2019-12-11 15:00:02
问题 Learning Akka Streams. I have a stream of records, many per time unit, already ordered by time (from Slick), and I want to batch them into time groups for processing by detecting when the time step changes. Example case class Record(time: Int, payload: String) If the incoming stream is Record(1, "a") Record(1, "k") Record(1, "k") Record(1, "a") Record(2, "r") Record(2, "o") Record(2, "c") Record(2, "k") Record(2, "s") Record(3, "!") ... I would like to transform this into Batch(1, Seq("a","k"