Why are AWS Batch Jobs stuck in RUNNABLE?

假装没事ソ 提交于 2019-12-03 01:26:14

There are other reasons why a Job can get stuck in RUNNABLE:

  • Insufficient permissions for the role associated to the Computed Environment
  • No internet access from the Compute Environment instance. You will need to associate a NAT or Internet Gateway to the Compute Environment subnet.
    • Make sure to check the "Enable auto-assign public IPv4 address" setting on your Compute Environment's subnet. (Pointed out by @thisisbrians in the comments)
  • Problems with your image. You need to use an ECS optimized AMI or make sure you have the ECS container agent working. More info at aws docs
  • You're trying to launch instances for which you account is limited to 0 instances (EC2 console > limits, in the left menu). (Read more on gergely-danyi comment)
  • And as mentioned insufficient resources

Also, make sure to read the AWS Batch troubleshooting

The roles should be defined using, at least, the next policies and trusted relationships. If not, they will get stuck in RUNNABLE as they don't have the enough privileges to start:

 AWSBatchServiceRole

  • Attached policies: AWSBatchServiceRole
  • Trusted relationship: batch.amazonaws.com

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
             "Service": "batch.amazonaws.com"
           },
          "Action": "sts:AssumeRole"
        }
      ]
    }
    

ecsInstanceRole

  • Attached policies: AmazonEC2ContainerServiceforEC2Role
  • Trusted relationship: ec2.amazonaws.com

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
             "Service": "ec2.amazonaws.com"
           },
          "Action": "sts:AssumeRole"
        }
      ]
    }
    

I just fought with this for a while, and found the answer.

One possible reason jobs can get stuck in Runnable is because there are no instances to run the job on. If this is the case, looking at the auto scaling group as mentioned in the above answer can show you the actual error that's preventing instances from being started, guiding you to the exact problem rather than leaving you to try any number solutions to problems you don't have. Error messages are our friends.

Your compute environment might be invalid. Check AWS Batch -> Compute Environments -> Status column. Mine said invalid, and this symbol was next to the compute environment name:

Clicking on the compute environment gave me more information - my AMI ID was wrong.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!