Spring Cloud AWS with Transfermanager : Unable to complete transfer: Connection pool shut down

问题

I am using Spring Boot 1.5.1.RELEASE with Spring Cloud AWS 1.1.3.RELEASE to upload files to the AWS S3 bucket.

I wanted to use the TransferManager to upload the files to the S3. But unfortunately, I am getting the following error message during the upload and the files are not uploaded to the S3:

2017-02-26 12:36:27.004 ERROR 32696 --- [ask-scheduler-8] o.s.integration.handler.LoggingHandler   : com.amazonaws.AmazonClientException: Unable to complete transfer: Connection pool shut down
    at com.amazonaws.services.s3.transfer.internal.AbstractTransfer.unwrapExecutionException(AbstractTransfer.java:277)
    at com.amazonaws.services.s3.transfer.internal.AbstractTransfer.rethrowExecutionException(AbstractTransfer.java:261)
    at com.amazonaws.services.s3.transfer.internal.UploadImpl.waitForUploadResult(UploadImpl.java:66)
    at com.my.package.aws.S3Configuration.withTransferManager(S3Configuration.java:50)
    at sun.reflect.GeneratedMethodAccessor119.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
    at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

These are my Java config files :

@EnableContextResourceLoader
@EnableContextCredentials(accessKey="XXXX", secretKey="YYYY")
@EnableAsync
@ComponentScan("com.my.package")
@EnableJpaRepositories(basePackages = "com.my.package.repository")
@EnableScheduling
@SpringBootApplication
public class S3UploadApplication {

    public static void main(final String[] args) {
        SpringApplication.run(S3UploadApplication.class, args);
    }
}

And the configuration bean for S3 with :

@Configuration
public class S3Configuration {
@Autowired
private AmazonS3Client amazonS3client;

public void withTransferManager() {
TransferManager transferManager = new TransferManager(this.amazonS3client);
ObjectMetadata objectMetadata = new ObjectMetadata();
      objectMetadata.setSSEAlgorithm(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);
Upload uploaded=transferManager.upload("myBucket","test.txt",new FileInputStream(new File("TestFile.txt")),objectMetadata);
uploaded.waitForCompletion();
transferManager.shutdownNow();
 }
}

I also tried this solution , that is creating the two Beans BasicAWSCredentials and the AmazonS3Client explicitly and configuring it acorrdingly, but still the same error is shown.

回答1:

I think you are shutdown the S3 Client in some place. Or TransferManager related code is called more that once. After seeing this issue in the aws sdk https://github.com/aws/aws-sdk-java/issues/1282

So solution is create a fresh instance of S3 Client and TransferManager each time that you need to use it, or just do no shutdown them.

From the issue, the best practice seems to be do not shutdown S3 client/Transfer Manager because it provides better performance.

回答2:

From above code, a single AmazonS3 client is reused where in for every s3 operation transfermanager object is created. Shutdownnow() method closes the transfermanager as well as AmazonS3 client.

Thus AmazonS3 client can't be reused again.

Document of shutdown method from aws:

https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/TransferManager.html

Instead of shutdownnow (), try using shutdownnow(false) which will only close the transfermanager and thus AmazonS3 client still can be used.

来源：https://stackoverflow.com/questions/42469437/spring-cloud-aws-with-transfermanager-unable-to-complete-transfer-connection

标签

amazon-s3

spring-boot