Implementing idempotency for AWS Spot Instance Requests

问题

I'm using the Java AWS SDK to make EC2 spot instance requests. As opposed to on demand instances, the API for spot requests does not have anything similar to ClientToken and thus does not support idempotency out of the box.

The most straightforward way I could think of to do this was to set the LaunchGroup property to a unique UUID; when I check for that I call DescribeSpotInstanceRequests and see if I already have a request with the same launch group.

To my surprise, it seems that there's a delay before the describe call returns the spot requests sent before. I wrote a JUnit test for this and it seems that in order for it to be consistent I would have to set a timeout of at least 60s between the two calls (request spot instance and describe spot instance requests). I need to have a granularity of 10s, because my requests can get repeated by the application at this interval, in case of any failure - i.e. something breaks after I sent the request but before I could read the result I got back from Amazon. In that case I don't want to have the request repeated, I just want to see that it got registered and move on.

@Test
public void testRunSpotInstances() throws Exception {

    activity.execute(execution);

    timeout(TIMEOUT);

    // shouldn't do anything
    activity.execute(execution);

    timeout(TIMEOUT);

    DescribeSpotInstanceRequestsResult result = client.describeSpotInstanceRequests(
            new DescribeSpotInstanceRequestsRequest().withFilters(new Filter()
                .withName("launch-group").withValues(BUSINESS_KEY)));

    assertThat(result.getSpotInstanceRequests()).hasSize(1);

    timeout(TIMEOUT);
}

The test works every time if TIMEOUT is set to 60s; for 40-50s it works intermittently. Anything below this fails every time.

Has anyone managed to work around this delay? Is implementing idempotency for spot requests possible using just the AWS API and not having state saved in the client application?

回答1:

In that case I don't want to have the request repeated, I just want to see that it got registered and move on.

If you got a 200 back, then it's registered. It may not show up right away, but it's registered and you can move on in your flow.

Is implementing idempotency for spot requests possible using just the AWS API and not having state saved in the client application?

I don't believe so. I have the same sort of issue with Amazon's EMR. The way that I work around it is to have a component who's job it is to observe clusters. When I make a request for an EMR cluster, I get back a cluster id, which I then pass off to some observer. The observer will then call my other components when that cluster changes state. Not being acknowledged by EMR right away is a valid case and is not treated like an exception.

I have no idea if that's appropriate for you. Perhaps you could try maintaining the SpotInstanceRequestId. In my case, I only keep them in memory, but you could keep them somewhere persistent if need be.

来源：https://stackoverflow.com/questions/14734882/implementing-idempotency-for-aws-spot-instance-requests