AWS Elastic Beanstalk: Add custom logs to CloudWatch?

Question

How to add custom logs to CloudWatch? Defaults logs are sent but how to add a custom one?

I already added a file like this: (in .ebextensions)

files         


        

Answer 1

Looking at the AWS docs it's not immediately apparent, but there are a few things you need to do.

(Our environment is an Amazon Linux AMI - Rails App on the Ruby 2.6 Puma Platform).

First, create a Policy in IAM to give your EB generated EC2 instances access to work with CloudWatch log groups and stream to them - we named ours "EB-Cloudwatch-LogStream-Access".

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:DescribeLogStreams",
                "logs:CreateLogGroup",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:log-group:/aws/elasticbeanstalk/*:log-stream:*"
        }
    ]
}

Once you have created this, make sure the policy is attached (in IAM > Roles) to your IAM Instance Profile and Service Role that are associated with your EB environment (check the environment's configuration page: Configuration > Security > IAM instance profile | Service Role).

Then, provide a .config file in your .ebextensions directory such as setup_stream_to_cloudwatch.config or 0x_setup_stream_to_cloudwatch.config. In our project we have made it the last extension .config file to run during our deploys by setting a high number for 0x (eg. 09_setup_stream_to_cloudwatch.config).

Then, provide the following, replacing your_log_file with the appropriate filename, keeping in mind that some log files live in /var/log on an Amazon Linux AMI and some (such as those generated by your application) may live in a path such as /var/app/current/log:

files:
  '/etc/awslogs/config/logs.conf':
    mode: '000600'
    owner: root
    group: root
    content: |
      [/var/app/current/log/your_log_file.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/app/current/log/your_log_file.log"]]}`
      log_stream_name = {instance_id}
      file = /var/app/current/log/your_log_file.log*
commands:
  "01":
    command: chkconfig awslogs on
  "02":
    command: service awslogs restart # note that this works for Amazon Linux AMI only - other Linux instances likely use `systemd`

Deploy your application, and you should be set!