aws-step-functions

AWS GetActivityTask beloging from the same state machine execution

ぃ、小莉子 提交于 2019-12-11 15:15:02
问题 Thanks in advance for your time and response. I have an AWS state machine with the following activities. Pull first available data file from an external FTP server Process the data (processing time can vary) Upload the processed data to another FTP server I have a java application running in an EC2 instance which has 3 threads and polls the activities using code as shown below. The java application invokes appropriate workers to do the actual work for steps #1,2 and 3. The important point

Anyone has experience with triggering step function with S3 event?

给你一囗甜甜゛ 提交于 2019-12-11 15:08:44
问题 I'm learning about step function and specifically, I'm trying to figure out how to trigger state machine execution with S3 event. I was reading this post: https://docs.aws.amazon.com/step-functions/latest/dg/tutorial-cloudwatch-events-s3.html. This documentation does give a rough guide of how to configure stuff but I'm still unclear about the following questions: What does the state machine input mean? So this documentation doesn't explain too much about what does each field in the input mean

Step Functions with Lambdas using Lambda Proxy Integration

ⅰ亾dé卋堺 提交于 2019-12-11 06:37:25
问题 I have written a bunch of Lambda functions that are exposed as Rest endpoints through API Gateway. I have chosen the "Lambda Proxy Integration" since it seemed like a straightforward way to get started. Now I want to chain together 2 of these functions via AWS Step Functions. The general integration and configuration works fine except how to create the proper inputs for each task. Using the console I can start an Execution and give the following JSON: { "headers": { "Authorization": "Bearer

How do you run functions in parallel?

感情迁移 提交于 2019-12-11 05:29:21
问题 My desire is to retrieve x number of records from a database based on some custom select statement, the output will be an array of json data. I then want to pass each element in the array into another lambda function in parallel. So if 1000 records are returned, 1000 lambda functions need to be executed in parallel (I increase my account limit to what I need). If 30 out of 1000 fail, the main task that was retrieving the records needs to know about it. I'm struggling to put together this

Strange results from using AWS S3 SELECT to get CSV data into SQL table

邮差的信 提交于 2019-12-11 01:32:27
问题 I have written an AWS State Machine in C# to load data from a CSV file from an S3 Bucket, into a SQL Server database table but I'm getting really odd data into the table. The two main functions are as follows, the first gets the response payload, the second breaks this up into lines that can then be inserted. private static async Task<ISelectObjectContentEventStream> GetSelectObjectContentEventStream(S3Object s3Object, AmazonS3Client s3Client, ObjectDefinition definition) { var response =

Can a lambda in an AWS Step Function know the “execution name” of the step function that launched it?

纵饮孤独 提交于 2019-12-10 15:17:54
问题 I have this step function that can sometimes fail and I'd like to record this in a (dynamo) DB. What would be handy is if I could just create a new error handling step and that guy would just pick up the "execution name" from somewhere (didn't find it in the context) and record this as a failure. Is that possible? 回答1: No. Unless you pass that information in the event, Lambda doesn't know whether or not it's part of a step function. Step functions orchestrate lambdas and maintain state

Managing error flow in AWS step-functions

六月ゝ 毕业季﹏ 提交于 2019-12-10 10:35:46
问题 I have an AWS step-function/state-machine of Lambda functions written primarily in Javascript (although one is in Java) and I'd like to manage the error processing better. I have no problem with having an error condition being caught and then forwarded to another state in the flow. So for instance, the following state definition in my state machine passes execution to the NotifyOfError state where I am able to email and sms appropriately about the error state. Closure: Type: Task Resource: >-

AWS Lambda chaining best practice

不羁的心 提交于 2019-12-10 10:04:35
问题 I'm looking a solution for my problem and maybe someone could give me some ideas. I have a API Gateway plugged to a aws lambda A. I have to handle cases like this: Lambda A should call lambda B and if there are any results, return to the API Gateway. Lambda A should call lambda B and if no results, it will call lambda C, and then return whatever the results are to the APi Gateway . So, my problem is how to chain these lambdas, because I don't want to have a huge lambda. At first, I thought

Passthrough input to output in AWS Step Functions

怎甘沉沦 提交于 2019-12-10 04:22:59
问题 How can I passthrough the input to a Task state in an AWS Step Functions to the output? After reading the Input and Output Processing page in the AWS docs, I have played with various combinations of InputPath , ResultPath and OutputPath . State definition: "First State": { "Type": "Task", "Resource": "[My Lambda ARN]", "Next": "Second State", "InputPath": "$.someKey", "OutputPath": "$" } Input: { "someKey": "someValue" } Expected Result I would like the output of the First State (and thus the

AWS API Gateway with Step Function

我怕爱的太早我们不能终老 提交于 2019-12-09 01:52:32
问题 I want a sample to integrate AWS API Gateway with Step Function. I have read this tutorial Creating a Step Functions API Using API Gateway but that tutorial needs me to send request in format of { "input": "{}", "name": "PostmanExecution", "stateMachineArn": "arn:aws:states:us-east-1:123456789012:stateMachine:Custom" } I want to send normal request and configure this stateMachineArn in API Gateway only, so that clients dont need to send this. 回答1: Create your API Gateway resource and method.