amazon-ecs | 易学教程

How to scale in/out EC2 instances based on ECS cluster resources availability?

阅读更多关于 How to scale in/out EC2 instances based on ECS cluster resources availability?

问题 I have multiple services running in my ECS cluster. Each service contains one or more tasks based on CPU utilization or a number of users. I have deployed these containers with EC2 launch type. Now, I want to increase/decrease the number of EC2 instances based on available resources in the cluster. Let's say there are four ECS tasks running in two m5.large instances. Now, if an ECS service increases the number of tasks and there aren't enough resources available in the cluster, how can I spin

How to speed up deployments on AWS Fargate?

阅读更多关于 How to speed up deployments on AWS Fargate?

问题 After migrating from EC2 cluster instances to AWS Fargate, I realized that deployments take a lot longer. Before they would take 1-2 minutes, now some deplyoments take up to 5 minutes. This post claims that their deployments on Fargate even take up to 10 minutes. Does anybody know of a way to speed them up? I can't find any documentation on this topic. 回答1: Through further googling I found this Reddit thread. An AWS employee wrote: With regard to time to provision and start a container it is

STOPPED (CannotPullContainerError: API error (500)?

阅读更多关于 STOPPED (CannotPullContainerError: API error (500)?

问题 I'm getting this error when running a task on my Amazon Fargate cluster. Has anyone seen run into this before? 回答1: Public IP is mandatory for Farget. For details https://github.com/aws/amazon-ecs-agent/issues/1128 回答2: Go to the docs for an answer to this one. https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_cannot_pull_image.html Since you are encountering a 500 error, I would heed the advice of the first error's description, "Connection timed out": When a Fargate task is

ECS Airflow 1.10.2 performance issues. Operators and tasks take 10x longer

阅读更多关于 ECS Airflow 1.10.2 performance issues. Operators and tasks take 10x longer

问题 We moved to puckel/Airflow-1.10.2 to try and resolve a poor performance we've had in multiple environments. We are running on ECS Airflow 1.10.2 on AWS ECS. Interestingly, the CPU/mem never jump above 80%. The Airflow metadb stays very underutilized as well. Below I've listed the configuration we're using, the DagBag parsing time plus the detailed execution times from the cProfile output of just running DagBag() in pure Python. A few of our DAGs import a function from create_subdag_functions

Required Cloudformation Script for Blue/Green deployment on ECS

阅读更多关于 Required Cloudformation Script for Blue/Green deployment on ECS

I am trying to write a cloud-formation template for AWS ECS with blue green deployment support . This blue-green feature was added recently by AWS in ECS and couldn't find any reference for updating it in cloud-formation template. They have given documentation on, how to do it through UI but not through cloud-formation. I guess, AWS might not updated their cloud-formation documentation as it is a new feature. Any help to find the documentation would be appreciated. Thanking you in advance. Currently cloudformation does not support the DeploymentController parameter in which you can specify

Private docker registry authentication in aws ecs optimized AMI is not successful

阅读更多关于 Private docker registry authentication in aws ecs optimized AMI is not successful

问题 I am writing a terraform script for creating a ECS auto scaling cluster. I have created a cluster and added ec2 container instances in to it.My task definition file contains a image that is from a Private docker repository.I go through the aws official documentation and find a page for Private Registry Authentication and tried both of the ways as described there. using dockercfg the docker way I put my ecs.config file in the S3 bucket and during the instance boot time I passed the user data

How to debug failed fargate task initialization

阅读更多关于 How to debug failed fargate task initialization

问题 I have a fargate task which I have scheduled to run with CloudWatch Event rules, and output a timestamp to a database on a successful run. It also outputs a logfile to CloudWatch for every time it runs. However, there was 1 time where the log file was not created, and the database not updated. I suspect the task was never even started, or had failed to start. In CloudWatch, the event rule shows trigger and invocation at the time I expected the task to run, so I assume the task at least

Missing log lines when writing to cloudwatch from ECS Docker containers

阅读更多关于 Missing log lines when writing to cloudwatch from ECS Docker containers

问题 (Docker container on AWS-ECS exits before all the logs are printed to CloudWatch Logs) Why are some streams of a CloudWatch Logs Group incomplete (i.e., the Fargate Docker Container exits successfully but the logs stop being updated abruptly)? Seeing this intermittently, in almost all log groups, however, not on every log stream/task run. I'm running on version 1.3.0 Description: A Dockerfile runs node.js or Python scripts using the CMD command. These are not servers/long-running processes,

How to scale in/out EC2 instances based on ECS cluster resources availability?

阅读更多关于 How to scale in/out EC2 instances based on ECS cluster resources availability?

I have multiple services running in my ECS cluster. Each service contains one or more tasks based on CPU utilization or a number of users. I have deployed these containers with EC2 launch type. Now, I want to increase/decrease the number of EC2 instances based on available resources in the cluster. Let's say there are four ECS tasks running in two m5.large instances. Now, if an ECS service increases the number of tasks and there aren't enough resources available in the cluster, how can I spin up an instance and add to the cluster? And same goes for vice versa. If there is instance running with

AWS ECS Fargate pull image from a cross account ECR repo

阅读更多关于 AWS ECS Fargate pull image from a cross account ECR repo

I have 2 AWS accounts: - account A that has an ECR repo. - account b that has an ECS cluster running Fargate. I have created a "cross-account" role in account A with trust relations to account B, also I have attached the "AmazonEC2ContainerRegistryPowerUser" policy to this role. I gave access to the ECR repository in account A by adding account B's id and the "cross-account" role to the repository policy. I attached a policy to the fargate "TaskExecutionRole" allowing fargate to assume the "cross-account" role. When trying to deploy a Fargate task in account B with a reference to an image in