apache-flink

问题 I am trying to use an implementation of the abstract RichAggregateFunction in Flink. I want it to be "rich" because I need to store some state as part of the aggregator, and I can do this since I have access to the runtime context. My code is something like below: stream.keyBy(...) .window(GlobalWindows.create()) .trigger(...) .aggregate(new MyRichAggregateFunction()); However, I get an UnsupportedOperationException saying This aggregation function cannot be a RichFunction. I'm clearly not

问题 I'm trying to do an exponentially decaying moving average over a hopping window in Flink SQL. I need the have access to one of the borders of the window, the HOP_START in the following: SELECT lb_index one_key, -- I have access to this one: HOP_START(proctime, INTERVAL '0.05' SECOND, INTERVAL '5' SECOND) start_time, -- Aggregation primitive: SUM( Y * EXP(TIMESTAMPDIFF( SECOND, proctime, -- This one throws: HOP_START(proctime, INTERVAL '0.05' SECOND, INTERVAL '5' SECOND) ))) FROM write

问题 I have a flink cep code that reads from socket and detects for a pattern. Lets say the pattern(word) is 'alert'. If the word alert occurs five times or more, an alert should be created. But I am getting an input mismatch error. Flink version is 1.3.0. Thanks in advance !! package pattern; import org.apache.flink.cep.CEP; import org.apache.flink.cep.PatternStream; import org.apache.flink.cep.pattern.Pattern; import org.apache.flink.cep.pattern.conditions.IterativeCondition; import org.apache

问题 I'm trying to access a key from a map using Flink's SQL API. It fails with the error Exception in thread "main" org.apache.flink.table.api.TableException: Type is not supported: ANY Please advise how i can fix it. Here is my event class public class EventHolder { private Map<String,String> event; public Map<String, String> getEvent() { return event; } public void setEvent(Map<String, String> event) { this.event = event; } } Here is the main class which submits the flink job public class

问题 I have an Apache Flink application that reads from a single Kafka topic. I would like to update the application from time to time without experiencing downtime. For now the Flink application executes some simple operators such as map and some synchronous IO to external systems via http rest APIs. I have tried to use the stop command, but i get "Job termination (STOP) failed: This job is not stoppable.", I understand that the Kafka connector does not support the the stop behavior - a link! A

问题 I have an interesting use case which I want to test with Flink. I have an incoming stream of Message which is either PASS or FAIL . Now if the message is of type FAIL , I have a downstream ProcessFunction which saves the Message state and then sends pause commands to everything that depends on this. When I receive a PASS message which is associated with the FAIL I had received earlier (keying by message id), I send resume commands to everything I had paused earlier. Now I plan on using State

问题 After upgrading to Apache Flink 1.4.2 we get following errors every few seconds on one TaskManager out of 3. 2018-06-27 17:33:46.632 [jobmanager-future-thread-2] DEBUG o.a.flink.runtime.rest.handler.legacy.metrics.MetricFetcher - Could not retrieve QueryServiceGateway. java.util.concurrent.CompletionException: akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://flink@tm03-dev:6124/), Path(/user/MetricQueryService_64bde0e9e6f3f0a906a30e88c261c9d7)] at java.util

问题 I've set up the example project and built it. I'm able to run the WordCount program as expected. But when I run the SocketTextWordCount, I'm not getting any results printed out. I send data in through nc (localhost:9999 on both sides) In the web console for the running job, I can see that there are messages being sent/received But I never see the counts.print() output printed out anywhere, even after killing the nc session. EDIT - when I change it around to print results to a text file, no

问题 I have two questions on Flink externalized checkpoints (Q1) I can set "state.checkpoints.dir" in flink-conf.yaml to get externalized checkpoints to work all right, but how do I achieve same thing when I run flink from IDE? I tried the GlobalConfiguration approach mentioned in (http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/state-checkpoints-dir-td17921.html) but no luck. This is how I did it: Configuration cfg = GlobalConfiguration.loadConfiguration(); cfg.setString(

问题 I've written a simple word count application using Flink 1.7.2 with Kafka 2.2 as both consumer and producer. I use Exactly-Once semantic for the Kafka producer, KeyedProcessFunction for stateful processing, MapState to keep my state and RocksDB with incremental checkpointing as my state-backend. The application works pretty fine when I run it from IntelliJ but when I submit it to my local Flink cluster I receive the AsynchronousException exception and the Flink application keeps retrying