We have a docker container running on an ec2 host. Within that docker container we run some aws cli commands. We haven't defined any AWS credentials within the container. This implies that the container inherits Instance Profile of the host ec2.
Is my assumption true? If so, how exactly does the container inherit the instance profile credentials? Secondly (possibly related) what exactly does the aws cli do to obtain the instance profile credentials? Does it make a call to the metadata endpoint (169.254.169.254)? For example if the credentials are picked up from the environment variables, the credentials are hard coded and can be seen but where do the credentials for an instance profile actually reside?
That's correct, the credentials are of the host machine. It gets them from the metadata endpoint, as you suspected.
One solution/workaround to give narrower access is ec2metadataproxy. I haven't used it yet.
The security group access is based on the host container too, unfortunately.