akka-stream

TLDR: is it better to materialize a stream per request (i.e. use short-lived streams) or to use a single stream materialization across requests, when I have an outgoing http request as a part of the stream? Details: I have a typical service that takes an HTTP request, scatters it to several 3rd party downstream services (not controlled by me) and aggregates the results before sending them back. I'm using akka-http for client implementation and spray for server (legacy, will move to akka-http over time). Schematically: request -> map -1-*-> map -> 3rd party http -> map -*-1> aggregation ->

I'm trying to understand why the below code snippet is doing what it's doing. I would have thought that because the Sink cannot produce demand faster than the Source is producing content, then I would be getting dropped messages in response to some of the offers (overflow strategy is set to Drop Buffer) and also an error and queue closed message after the self destruct piece. The snippet: package playground import java.time.LocalDateTime import java.util.concurrent.atomic.AtomicInteger import akka.actor.{Actor, ActorLogging, ActorSystem, Props} import akka.stream.QueueOfferResult.{Dropped,

Let's assume we have two Akka Stream flows, each running on its own JVM. // A reactive streams publisher running on JVM 1: val stringPublisher: Publisher[String] = Source(() => "Lorem Ipsum".split("\\s").iterator).runWith(Sink.publisher[String]) // A reactive streams subscriber running on JVM 2: def subscriber: Subscriber[String] = Sink.foreach[String](println(_)).runWith(Source.subscriber[String]) // Subscribe the second stream to the first stream stringPublisher.subscribe(subscriber) This example runs fine on one JVM, but how can I subscribe to a publisher running on a different JVM? Do I

I'm trying out Akka Streams and here is a short snippet that I have: override def main(args: Array[String]) { val filePath = "/Users/joe/Softwares/data/FoodFacts.csv"//args(0) val file = new File(filePath) println(file.getAbsolutePath) // read 1MB of file as a stream val fileSource = SynchronousFileSource(file, 1 * 1024 * 1024) val shaFlow = fileSource.map(chunk => { println(s"the string obtained is ${chunk.toString}") }) shaFlow.to(Sink.foreach(println(_))).run // fails with a null pointer def sha256(s: String) = { val messageDigest = MessageDigest.getInstance("SHA-256") messageDigest.digest

How do you throttle Flow in the latest Akka (2.4.6) ? I'd like to throttle Http client flow to limit number of requests to 3 requests per second. I found following example online but it's for old Akka and akka-streams API changed so much that I can't figure out how to rewrite it. def throttled[T](rate: FiniteDuration): Flow[T, T] = { val tickSource: Source[Unit] = TickSource(rate, rate, () => ()) val zip = Zip[T, Unit] val in = UndefinedSource[T] val out = UndefinedSink[T] PartialFlowGraph { implicit builder => import FlowGraphImplicits._ in ~> zip.left ~> Flow[(T, Unit)].map { case (t, _) =>

I am using Akka Streams in Scala to poll from an AWS SQS queue using the AWS Java SDK . I created an ActorPublisher which dequeues messages on a two second interval: class SQSSubscriber(name: String) extends ActorPublisher[Message] { implicit val materializer = ActorMaterializer() val schedule = context.system.scheduler.schedule(0 seconds, 2 seconds, self, "dequeue") val client = new AmazonSQSClient() client.setRegion(RegionUtils.getRegion("us-east-1")) val url = client.getQueueUrl(name).getQueueUrl val MaxBufferSize = 100 var buf = Vector.empty[Message] override def receive: Receive = { case