问题
AWS lambda function does not work when attempting to work with credentials provided using the DefaultCredentialProvider.
I need to pass the credentials to S3 for it to run.
Code
def initializeAwsCredentials():AWSCredentials = {
var credentials: AWSCredentials = null
try {
credentials = new ProfileCredentialsProvider().getCredentials
} catch {
case e: Exception => {
throw new AmazonClientException(
"Cannot load the credentials from the credential profiles file. " +
"Please make sure that your credentials file is at the correct " +
"location (~/.aws/credentials), and is in valid format.",
e);
}
}
return credentials
}
def buildS3API(credentials: AWSCredentials): AmazonS3 = {
new AmazonS3Client(credentials)
}
// inside handle request
val credentials = initializeAwsCredentials()
println("Credetials have been retrieved successfully")
println("Build S3 API using the constructor provided")
val s3 = buildS3API(credentials)
s3.setRegion(region)
println("S3 API is now available")
Error
{
"errorMessage": "Cannot load the credentials from the credential profiles file. Please make sure that your credentials file is at the correct location (~/.aws/credentials), and is in valid format.",
"errorType": "com.amazonaws.AmazonClientException",
"stackTrace": [
"example.Main$.initializeAwsCredentials(Hello.scala:52)",
"example.Main$.handleRequest(Hello.scala:125)",
"example.Main.handleRequest(Hello.scala)",
"sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
"sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
"sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
"java.lang.reflect.Method.invoke(Method.java:498)"
],
"cause": {
"errorMessage": "java.lang.NullPointerException",
"errorType": "java.lang.NullPointerException",
"stackTrace": [
"com.amazonaws.auth.profile.ProfilesConfigFile.<init>(ProfilesConfigFile.java:143)",
"com.amazonaws.auth.profile.ProfilesConfigFile.<init>(ProfilesConfigFile.java:132)",
"com.amazonaws.auth.profile.ProfilesConfigFile.<init>(ProfilesConfigFile.java:99)",
"com.amazonaws.auth.profile.ProfileCredentialsProvider.getCredentials(ProfileCredentialsProvider.java:135)",
"example.Main$.initializeAwsCredentials(Hello.scala:45)",
"example.Main$.handleRequest(Hello.scala:125)",
"example.Main.handleRequest(Hello.scala)",
"sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
"sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
"sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
"java.lang.reflect.Method.invoke(Method.java:498)"
]
}
}
Update
Using the InstanceProfileCredentialsProvider instead throws the error:
val provider: InstanceProfileCredentialsProvider = new InstanceProfileCredentialsProvider()
credentials = provider.getCredentials()
gives me the error:
"cause": {
"errorMessage": "Unable to load credentials from Amazon EC2 metadata service",
"errorType": "com.amazonaws.AmazonClientException",
"stackTrace": [
"com.amazonaws.auth.InstanceProfileCredentialsProvider.handleError(InstanceProfileCredentialsProvider.java:244)",
"com.amazonaws.auth.InstanceProfileCredentialsProvider.loadCredentials(InstanceProfileCredentialsProvider.java:225)",
"com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:124)",
"example.Main$.initializeAwsCredentials(Hello.scala:46)",
"example.Main$.handleRequest(Hello.scala:126)",
"example.Main.handleRequest(Hello.scala)",
"sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
"sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
"sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
"java.lang.reflect.Method.invoke(Method.java:498)"
],
"cause": {
"errorMessage": "Connection refused (Connection refused)",
"errorType": "java.net.ConnectException",
"stackTrace": [
"java.net.PlainSocketImpl.socketConnect(Native Method)",
"java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)",
"java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)",
"java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)",
"java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)",
"java.net.Socket.connect(Socket.java:589)",
"sun.net.NetworkClient.doConnect(NetworkClient.java:175)",
"sun.net.www.http.HttpClient.openServer(HttpClient.java:463)",
"sun.net.www.http.HttpClient.openServer(HttpClient.java:558)",
"sun.net.www.http.HttpClient.<init>(HttpClient.java:242)",
"sun.net.www.http.HttpClient.New(HttpClient.java:339)",
"sun.net.www.http.HttpClient.New(HttpClient.java:357)",
"sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1202)",
"sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138)",
"sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032)",
"sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:966)",
"com.amazonaws.internal.EC2MetadataClient.readResource(EC2MetadataClient.java:90)",
"com.amazonaws.internal.EC2MetadataClient.getDefaultCredentials(EC2MetadataClient.java:55)",
"com.amazonaws.auth.InstanceProfileCredentialsProvider.loadCredentials(InstanceProfileCredentialsProvider.java:186)",
"com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:124)",
"example.Main$.initializeAwsCredentials(Hello.scala:46)",
"example.Main$.handleRequest(Hello.scala:126)",
"example.Main.handleRequest(Hello.scala)",
"sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
"sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
"sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
"java.lang.reflect.Method.invoke(Method.java:498)"
]
}
}
}
Configuring the following as environment variables when using lambda also fails:
Lambda was unable to configure your environment variables because the
environment variables you have provided contains reserved keys that are
currently not supported for modification. Reserved keys used in this
request: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
回答1:
I am not sure you need an explicit credentials provider. Inside the AWS Lambda the credentials are automatically provided via the role the lambda can assume. I know I never did it explicitly.
http://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/loading-node-credentials-lambda.html
回答2:
For Lambda functions you will want to use an IAM role for credentials. Then you would either use the DefaultAWSCredentialsProviderChain or InstanceProfileCredentialsProvider to retrieve the credentials from the IAM role.
Class InstanceProfileCredentialsProvider
Here is an example using InstanceProfileCredentialsProvider:
AWSCredentialsProvider credentialsProvider = null;
try {
credentialsProvider = new InstanceProfileCredentialsProvider();
// Verify we can fetch credentials
credentialsProvider.getCredentials();
System.out.println("Obtained credentials.");
} catch (AmazonClientException e) {
System.out.println("Unable to obtain credentials", e);
return -1;
}
System.out.println("Using credentials with access key id: " + credentialsProvider.getCredentials().getAWSAccessKeyId());
回答3:
In the most recent versions of the AWS SDK's for Java (1.11.4xx), almost all services have "Client Builders" that can be used to quickly create your clients.
val snsClient = AmazonSNSClientBuilder.defaultClient()
val s3Client = AmazonS3ClientBuilder.defaultClient()
val dynamoDbClient = AmazonDynamoDBAsyncClientBuilder.defaultClient()
val sesClient = AmazonSimpleEmailServiceAsyncClientBuilder.defaultClient()
// ...
In Lambda the defaultClient() works really well because it will create a client that will use the appropriate provider. This provider uses the credentials with the permissions defined in the lambda execution role.
In a local environment, the defaultClient also works well, as it picks up the host credentials. This works because the defaultClient uses on the default credentials provider chain which looks for
- Env Vars
- Java system properties
- Credentials profile file
- ECS container credentials
- Instance profile credentials
This method is also nice and concise, but you can also use the client builders to create clients using specific credentials "setup/configuration".
AWS SDK for Java v2
If you want to use the new version of the Java SDK's (>=2.1) there are create methods that can be used to get the clients (although I've only used it for experimenting with the new SDK)
val s3Client = S3AsyncClient.create()
val dynamoDbClient = DynamoDbAsyncClient.create()
// ...
回答4:
I had similar situation in java. Hopefully same methodology applies here. I switched my local to EnvironmentVariableCredentialsProvider, supplied credentials via env tab in "run config" of my local java program. Then I uploaded the jar to Lambda function, there is Env configuration too, just supply credentials there.
来源：https://stackoverflow.com/questions/47763573/aws-credentials-for-lambda-when-working-with-scala-not-working