apache-flink

I was trying to run the beam Python-SDK example, but I had problem in reading the input. https://cwiki.apache.org/confluence/display/BEAM/Usage+Guide#UsageGuide-RunaPython-SDKPipeline when I used gs://dataflow-samples/shakespeare/kinglear.txt as the input, the error was apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://dataflow-samples/shakespeare/kinglear.txt': TypeError("__init__() got an unexpected keyword argument 'response_encoding'",)} when I used my local file, it seemed it didn't actually read the file, and output nothing. The result didn't include

I'm having troubles using Apache Flink Scala API For example, even when I take the examples from the official documentation, the scala compiler gives me tons of compilation errors. Code: object TestFlink { def main(args: Array[String]) { val env = ExecutionEnvironment.getExecutionEnvironment val text = env.fromElements( "Who's there?", "I think I hear them. Stand, ho! Who's there?") val counts = text.flatMap { _.toLowerCase.split("\\W+") filter { _.nonEmpty } } .map { (_, 1) } .groupBy(0) .sum(1) counts.print() env.execute("Scala WordCount Example") } } Scala IDE outputs the following for the

The doc of kafka give an approach about with following describes: One Consumer Per Thread:A simple option is to give each thread its own consumer > instance. My code: public class KafkaConsumerRunner implements Runnable { private final AtomicBoolean closed = new AtomicBoolean(false); private final CloudKafkaConsumer consumer; private final String topicName; public KafkaConsumerRunner(CloudKafkaConsumer consumer, String topicName) { this.consumer = consumer; this.topicName = topicName; } @Override public void run() { try { this.consumer.subscribe(topicName); ConsumerRecords<String, String>

Flink has been compared to Spark , which, as I see it, is the wrong comparison because it compares a windowed event processing system against micro-batching; Similarly, it does not make that much sense to me to compare Flink to Samza. In both cases it compares a real-time vs. a batched event processing strategy, even if at a smaller "scale" in the case of Samza. But I would like to know how Flink compares to Storm, which seems conceptually much more similar to it. I have found this (Slide #4) documenting the main difference as "adjustable latency" for Flink. Another hint seems to be an article