complex-event-processing

Causes of CPU load on the machine increase when flink run more more than 12 hours

对着背影说爱祢 提交于 2021-01-29 20:41:43
问题 I have a flink job, with parallelism set to 6, few simple transformations and the issue is that when Flink is been running for more than 12 hours for example the Load on the machine start to increase, then I thought that was because of the traffic into flink during some hours of the day, but the issue is that when the traffic goes down, the load on the machine continue a bit higher, lower than before but still higher. Use cases: DataStream<Event> from_source = rabbitConsumer .flatMap(new

Esper/NEsper EPL event Statement

让人想犯罪 __ 提交于 2021-01-29 07:26:57
问题 I'm new in Esper. Can anyone help me define the EPL statement to catch the event when the following situation occurs: Assumming that there are events with 3 attributes - (string)Symbol, (boolean)Value, (datetime)Timestamp. For example event1 (Symbol-apple, Value-true, Timestamp- 2020.10.07 14:00:00), event2 (Symbol-orange, Value-true, Timestamp- 2020.10.07 14:00:00) and event3 (Symbol-banana, Value-false, Timestamp- 2020.10.07 14:00:00). If they have same (or almost the same) Timestamp only

Esper loss-less events processing

萝らか妹 提交于 2020-01-16 16:48:18
问题 I'm evaluating Esper as a system for loss-less processing of billing data. It is expected that system can handle ~20000 events per second and run ~400 statements with continuos aggregation (without storing events in memory). In order to get expected performance I've started to send events in multiple threads and found that esper often looses data. Simple example that shows data loss import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.List;

Esper window usage: Recalculation based on event leaving window

本小妞迷上赌 提交于 2020-01-14 04:06:07
问题 I need somewhat like this. //Create a named window (w1) Create window w1.... //then insert events into window insert into w1.... select amount,..... from.... where... //remove from window on RemoveEventArrived as r Delete from w1 where w1.id = r.id Now insert into another event from w1 named window //inserting into final output event insert into derivedevent select sum(w.amount) from w1 as w suppose event sequence: 1. Event with id=1 and amount= 100 arrived. o/p of sum(amount) triggered and

WSO2 CEP : Siddhi QL: Creating a unique stream with similar event records

*爱你&永不变心* 提交于 2020-01-07 03:11:30
问题 I am pretty new to WSO2 CEP Siddhi QL, I got a requirement to analyze the events coming to a stream. For ex: I have data coming in like this [id,value]: InputStream=[1001,90] InputStream=[1001,85] InputStream=[1002,70] InputStream=[1001,85] InputStream=[1003,70] InputStream=[1003,85] InputStream=[1002,70] InputStream=[1003,70] InputStream=[1003,85] InputStream=[1002,70] InputStream=[1001,95] InputStream=[1001,65] In this, I want to segregate each records and group based on the id 1001, 1002

Complext event structure in FIWARE Generic Enabler CEP - Proton

这一生的挚爱 提交于 2020-01-06 14:07:23
问题 I'd like to use Proton as CEP GE in my FI-WARE architecture. I ask your help because I have an event that has a complex structure and I would like to understand how can I define it and the EPA that should use it. I check in documents you provide in this wiki https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/CEP_GE_-_IBM_Proactive_Technology_Online_User_and_Programmer_Guide and then forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Complex_Event_Processing_Open_RESTful

Complext event structure in FIWARE Generic Enabler CEP - Proton

烂漫一生 提交于 2020-01-06 14:05:52
问题 I'd like to use Proton as CEP GE in my FI-WARE architecture. I ask your help because I have an event that has a complex structure and I would like to understand how can I define it and the EPA that should use it. I check in documents you provide in this wiki https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/CEP_GE_-_IBM_Proactive_Technology_Online_User_and_Programmer_Guide and then forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Complex_Event_Processing_Open_RESTful

Azure Stream Analytics: Multiple Windows + JOINS

流过昼夜 提交于 2019-12-25 03:08:12
问题 My architecture: 1 EventHub with 8 Partitions & 2 TPUs 1 Streaming Analytics Job 6 Windows based on the same input (from 1mn to 6mn) Sample Data: {side: 'BUY', ticker: 'MSFT', qty: 1, price: 123, tradeTimestamp: 10000000000} {side: 'SELL', ticker: 'MSFT', qty: 1, price: 124, tradeTimestamp:1000000000} The EventHub PartitionKey is ticker I would like to emit every second, the following data: (Total quantity bought / Total quantity sold) in the last minute, last 2mn, last 3mn and more What I

How Event stream Works in wso2cep3.0.0

社会主义新天地 提交于 2019-12-25 02:13:49
问题 I am working with wso2cep3.0.0, My input source and out put source is JMs.I written my Input event adapter and output event adapter like this Input adapter is <?xml version="1.0" encoding="UTF-8"?> <inputEventAdaptor name="jmsProxy" statistics="disable" trace="enable" type="jms" xmlns="http://wso2.org/carbon/eventadaptormanager"> <property name="java.naming.provider.url">tcp://localhost:61616</property> <property name="transport.jms.SubscriptionDurable">false</property> <property name=

Creating instances of Esper's epl

大兔子大兔子 提交于 2019-12-24 07:27:31
问题 I am playing around with Esper, learning how to use more advanced concepts. I have a program that fires mock stock events of 3 different stocks. I currently have a module with match_recognize pattern EPL that looks like this: module queries; import events.*; import configDemo.*; import annotations.*; create schema MyTickEvent as TickEvent; @Name('compareStocks') @Description('Compare the difference amount between two stocks') @Subscriber(className='configDemo.MySubscriber') select * from