How to auto deploying git repositories with submodules on AWS?

Question

I have a submodule in my git repository and my directory structure is like,

app
  -- folder1
  -- folder2
  -- submodule @5855

I have deployed

Answer 1

I faced the same issue on AWS CodeBuild. I tick Use Git submodules like below image to update my submodule.

When I run the build I got following error,

CLIENT_ERROR: Submodule error error creating SSH agent: "SSH agent requested but SSH_AUTH_SOCK not-specified" for primary source and source version refs/heads/dev

So I googled the above error and got this DOWNLOAD_SOURCE Fails with Git submodules thread from AWS Forum. They've mentioned,

The submodules must be configured as https and not ssh.

I think this is useless, what will happen someone setup submodule as ssh. I also did the same, here is my .gitmodules file.

[submodule "common"]
    path = common
    url = git@bitbucket.org:organization_id/common.git

Really I don't want to change it to https. Then I found this Working with Git Submodules in CodePipeline article from medium. I would like to visualize what I did to solve this issue and there was an error that didn't mention in that article. Let's do this in more secure way.

---

First go to the AWS Key Management Service (KMS) and go to the Customer managed keys section and click the Create key to create the key.

1. Click the Symmetric and click Next.

2. Give any name (ex:- bitbucket-credentials) to create Alias and click Next.

3. Most probably you already have an AWS Role to configure any of Developer Tools on AWS, so in my case I created a AWS Role call ecsCodeBuildRole for AWS CodeBuild and give the Define key administrative permissions for it and click Next.

4. Next give the Define key usage permissions for your AWS Role and click Next.

5. Finally review what you did so far and and click Finish to create the CMK.

6. You can review it like below.

---

So AWS Key Management Service (KMS) part is done, now go the AWS Systems Manager and find the Parameter Store section. Click Create parameter.

1. Just name it as id_rsa and put the same things like below.

2. For the value section, just run cat ~/.ssh/id_rsa command in your terminal and you'll get the output like below. Add it to the value section.

-----BEGIN RSA PRIVATE KEY-----
qdjbXp+42VTnccC7pxOZcofomfwGXPWuqcv99sQEPtToODvGIxWoooJUpb6qMIWY
1zccEuwAhmqcPvpsJyWhcctZB/wWglNvViZcOYjrQ8HBUBKJT8pF
-----END RSA PRIVATE KEY-----

3. Create another parameter and name it as id_rsa.pub. Follow the same steps like above.

4. For the value section, just run cat ~/.ssh/id_rsa.pub command in your terminal and you'll get the output like below. Add it to the value section.

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGtf8jjkogWxRGNGjJlOZ1G+pWExgDOdA5wVML6TMTT2YtvhPJD60nPx5TfA8zRzGWubwrPp40SPAhSs5wiAAg38HlS4pz9X wasdkiller@wasdkiller

---

As my research(maybe I'm wrong, please correct me) I don't have any other way to pass ssh credentials to the AWS CodeBuild without this much of effort. So I changed my buildspec.yml file manually like this.

version: 0.2

env:
  parameter-store:
    ssh_key: id_rsa
    ssh_pub: id_rsa.pub

phases:
  install:
    commands:
      - mkdir -p ~/.ssh
      - echo "$ssh_key" > ~/.ssh/id_rsa   
      - echo "$ssh_pub" > ~/.ssh/id_rsa.pub
      - chmod 600 ~/.ssh/id_rsa
      - eval "$(ssh-agent -s)"
      - git submodule update --init --recursive

When you continue you'll get below error surely,

Decrypted Variables Error: AccessDeniedException: User: arn:aws:sts::organization_id:assumed-role/ecsCodeBuildRole/AWSCodeBuild-12896abb-bdcf-4cfc-a12b-bcf30d6e96ab is not authorized to perform: ssm:GetParameters on resource: arn:aws:ssm:ap-southeast-2:organization_id:parameter/wasd status code: 400, request id: 23b94bc2-961e-4d86-9b73-d16e3bda357c

It'll ask you for ssm:GetParameters permission, Just attach AmazonSSMReadOnlyAccess policy or create policy manually with the ssm:GetParameters permission and attach it to your AWS Role, it'll solve this issue.