azure-eventhub

IoT Hub message routing with DeviceLifecycleEvents source does not work

好久不见. 提交于 2021-02-16 05:32:30
问题 I am already using an IoT Hub and have configured message routing to route TwinChangeEvents to an Event Hub. An Azure function with an EventHubTrigger processes the messages. This works perfectly fine. Now, I wanted to configure message routing to route DeviceLifecycleEvents to another Event Hub. Again, another Azure function with an EventHubTrigger should process the messages. However, the function is not triggered when I connect or disconnect devices. When I change the data source of the

What is the best parameters to integrate Azure Functions with Event hubs chain

女生的网名这么多〃 提交于 2021-02-11 14:21:42
问题 we need to setup 4 EventHub and 3 Azure Functions. So what is the best way to have high throughput and Scalable parameters that we can set to have a system that can handle 75k message/sec? Local.settings.json hosts.json prefetch Count max batch side 回答1: This article is definitely worth a read and is something I based some of my work on, I needed to achieve 50k p/sec. https://azure.microsoft.com/en-gb/blog/processing-100-000-events-per-second-on-azure-functions/ An important consideration is

Stream Analytics Job deployed as Azure Resource Manager (ARM) template

瘦欲@ 提交于 2021-02-10 18:49:29
问题 I am trying to setup an output EventHub for a Stream Analytics Job defined as a JSON template. Without the output bit the template is successfully deployed, however when adding the output definition it fails with: Deployment failed. Correlation ID: <SOME_UUID>. { "code": "BadRequest", "message": "The JSON provided in the request body is invalid. Property 'eventHubName' value 'parameters('eh_name')' is not acceptable.", "details": { "code": "400", "message": "The JSON provided in the request

Reading schema of streaming Dataframe in Spark Structured Streaming [duplicate]

南笙酒味 提交于 2021-02-04 21:05:17
问题 This question already has an answer here : Why does using cache on streaming Datasets fail with “AnalysisException: Queries with streaming sources must be executed with writeStream.start()”? (1 answer) Closed 13 days ago . I'm new with Apache Spark Structured Streaming. I'm trying to read some events from Event Hub (in XML format) and trying to create new Spark DF from the nested XML. Im using the code example described in https://github.com/databricks/spark-xml and in batch mode is running

Reading schema of streaming Dataframe in Spark Structured Streaming [duplicate]

折月煮酒 提交于 2021-02-04 21:01:35
问题 This question already has an answer here : Why does using cache on streaming Datasets fail with “AnalysisException: Queries with streaming sources must be executed with writeStream.start()”? (1 answer) Closed 13 days ago . I'm new with Apache Spark Structured Streaming. I'm trying to read some events from Event Hub (in XML format) and trying to create new Spark DF from the nested XML. Im using the code example described in https://github.com/databricks/spark-xml and in batch mode is running

How to debug EventHubTrigger locally?

喜夏-厌秋 提交于 2021-01-28 04:06:06
问题 I'm trying to debug an Azure Function locally. It's an EventHubTrigger. The problem is that I need to debug the code locally because I still don't have the "real" settings. My code currently looks like this: public static class Notificator { [FunctionName("Notificator")] public static async Task Run([EventHubTrigger("achievements")]UserAchivementNofication notification, ILogger log) { } } But when I try to debug it, I'm getting this error: Microsoft.Azure.WebJobs.Host: Error indexing method

Azure Event Hubs - REST API Authentication

荒凉一梦 提交于 2021-01-27 20:06:19
问题 I have a Python program that generates streaming data. Each piece of data is stored as a variable called message . Since Event Hubs does not offer a Python SDK, I've opted to use their REST API. I've attempted to push the data to my Event Hub instance as a part of my Python program that generates data as follows: r = requests.post("https://{Instance name}.servicebus.windows.net/{Path}/messages", message) print(r.status_code, r.reason) Upon executing this code, I receive a 401 Unauthorized

Azure eventhub library for python

白昼怎懂夜的黑 提交于 2021-01-01 09:38:07
问题 I am using eventhub for ingesting a lot of events. I have multiple consumers which are running behing a scaling group reading these events from the eventhub which has multiple partitions. I was going through the Azure SDK in python and was confused as to what to use. There is eventhubconsumerclient, eventprocessorHost .... I would like to use a library where my multiple consumer can connect using the consumer group, the partitions are assigned dynamically to each consumer and checkpointing is

Azure Event Hub - Receiving events Sequentially

余生长醉 提交于 2020-12-15 06:48:05
问题 I am using below code receive the events from Azure Event-Hub https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-dotnet-framework-getstarted-send#receive-events I want to Handle the requests coming to event-hub Sequentially . For example someone sent 5 events in very quick time, I want to complete request 1 processing then i want take the second request to process. How can i handle the events coming to event-hub Sequentially? 回答1: From the docs, In order to make it sequential you