NSURLSession and amazon S3 uploads

Question

I have an app which is currently uploading images to amazon S3. I have been trying to switch it from using NSURLConnection to NSURLSession so that the uploads can continue w

Answer 1

I don't know NSURLSessionUploadTask very well yet but I can tell you how I would debug this.

I would use a tool like Charles to be able to see HTTP(S) requests that my application makes. The problem is likely that the NSURLSessionUploadTask ignores a header that you set or it uses a different HTTP method than Amazon's S3 expects for the file upload. This can be easily verified with an intercepting proxy.

Also, when Amazon S3 returns an error like 403, it actually sends back an XML document that has some more information about the error. Maybe there is a delegate method for NSURLSession that can retrieve the response body? If not then Charles will certainly give you more insight.

Answer 2

For background uploading/downloading you need to use NSURLSession with background configuration. Since AWS SDK 2.0.7 you can use pre signed requests:

PreSigned URL Builder** - The SDK now includes support for pre-signed Amazon Simple Storage Service (S3) URLs. You can use these URLS to perform background transfers using the NSURLSession class.

Init background NSURLSession and AWS Services

- (void)initBackgroundURLSessionAndAWS
{
    NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:AWSS3BackgroundSessionUploadIdentifier];
    self.urlSession = [NSURLSession sessionWithConfiguration:configuration delegate:self delegateQueue:nil];
    AWSServiceConfiguration *configuration = [AWSServiceConfiguration configurationWithRegion:DefaultServiceRegionType credentialsProvider:credentialsProvider];
    [AWSServiceManager defaultServiceManager].defaultServiceConfiguration = configuration;
    self.awss3 = [[AWSS3 alloc] initWithConfiguration:configuration];
}

Implement upload file function

- (void)uploadFile
{
    AWSS3GetPreSignedURLRequest *getPreSignedURLRequest = [AWSS3GetPreSignedURLRequest new];
    getPreSignedURLRequest.bucket = @"your_bucket";
    getPreSignedURLRequest.key = @"your_key";
    getPreSignedURLRequest.HTTPMethod = AWSHTTPMethodPUT;
    getPreSignedURLRequest.expires = [NSDate dateWithTimeIntervalSinceNow:3600];
    //Important: must set contentType for PUT request
    getPreSignedURLRequest.contentType = @"your_contentType";

    [[[AWSS3PreSignedURLBuilder defaultS3PreSignedURLBuilder] getPreSignedURL:getPreSignedURLRequest] continueWithBlock:^id(BFTask *task) {
        if (task.error)
        {
            NSLog(@"Error BFTask: %@", task.error);
        }
        else
        {
            NSURL *presignedURL = task.result;
            NSLog(@"upload presignedURL is: \n%@", presignedURL);

            NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:presignedURL];
            request.cachePolicy = NSURLRequestReloadIgnoringLocalCacheData;
            [request setHTTPMethod:@"PUT"];
            [request setValue:contentType forHTTPHeaderField:@"Content-Type"];

//          Background NSURLSessions do not support the block interfaces, delegate only.
            NSURLSessionUploadTask *uploadTask = [self.session uploadTaskWithRequest:request fromFile:@"file_path"];

            [uploadTask resume];
        }
        return nil;
    }];
}

NSURLSession Delegate Function:

- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didCompleteWithError:(NSError *)error
{
    if (error)
    {
        NSLog(@"S3 UploadTask: %@ completed with error: %@", task, [error localizedDescription]);
    }
    else
    {
//      AWSS3GetPreSignedURLRequest does not contain ACL property, so it has to be set after file was uploaded
        AWSS3PutObjectAclRequest *aclRequest = [AWSS3PutObjectAclRequest new];
        aclRequest.bucket = @"your_bucket";
        aclRequest.key = @"yout_key";
        aclRequest.ACL = AWSS3ObjectCannedACLPublicRead;

        [[self.awss3 putObjectAcl:aclRequest] continueWithBlock:^id(BFTask *bftask) {
            dispatch_async(dispatch_get_main_queue(), ^{
                if (bftask.error)
                {
                    NSLog(@"Error putObjectAcl: %@", [bftask.error localizedDescription]);
                }
                else
                {
                    NSLog(@"ACL for an uploaded file was changed successfully!");
                }
            });
            return nil;
        }];
    }
}

Answer 3

Here is my code to run the task:

AmazonS3Client *s3Client = [[AmazonS3Client alloc] initWithAccessKey:accessKey withSecretKey:secretKey];
S3PutObjectRequest *s3PutObjectRequest = [[S3PutObjectRequest alloc] initWithKey:[url lastPathComponent] inBucket:bucket];
s3PutObjectRequest.cannedACL = [S3CannedACL publicRead];
s3PutObjectRequest.endpoint = s3Client.endpoint;
s3PutObjectRequest.contentType = fileMIMEType([url absoluteString]);
[s3PutObjectRequest configureURLRequest];

NSMutableURLRequest *request = [s3Client signS3Request:s3PutObjectRequest];
NSMutableURLRequest *request2 = [[NSMutableURLRequest alloc]initWithURL:request.URL];
[request2 setHTTPMethod:request.HTTPMethod];
[request2 setAllHTTPHeaderFields:[request allHTTPHeaderFields]];

NSURLSessionUploadTask *task = [[self backgroundURLSession] uploadTaskWithRequest:request2 fromFile:url];
[task resume];

I open sourced my S3 background uploaded https://github.com/genadyo/S3Uploader/

Answer 4

Recently Amazon has updated there AWS api to 2.2.4. speciality of this update is that, it supports background uploading, you don't have to use NSURLSession to upload videos its pretty simple, you can use following source block to test it, I have tested against with my older version, it is 30 - 40 % faster than the previous version

in AppDelegate.m didFinishLaunchingWithOptions method // ~GM~ setup cognito for AWS V2 configurations

AWSStaticCredentialsProvider *staticProvider = [[AWSStaticCredentialsProvider alloc] initWithAccessKey:@"xxxx secretKey:@"xxxx"];  

AWSServiceConfiguration *configuration = [[AWSServiceConfiguration alloc] initWithRegion:AWSRegionUSWest2                                                                 credentialsProvider:staticProvider];

AWSServiceManager.defaultServiceManager.defaultServiceConfiguration = configuration;

in handleEventsForBackgroundURLSession method

[AWSS3TransferUtility interceptApplication:application
       handleEventsForBackgroundURLSession:identifier
                         completionHandler:completionHandler];

in upload class

NSURL *fileURL = // The file to upload.

AWSS3TransferUtilityUploadExpression *expression = [AWSS3TransferUtilityUploadExpression new];
expression.uploadProgress = ^(AWSS3TransferUtilityTask *task, int64_t bytesSent, int64_t totalBytesSent, int64_t totalBytesExpectedToSend) {
    dispatch_async(dispatch_get_main_queue(), ^{
        // Do something e.g. Update a progress bar.
    });
};

AWSS3TransferUtilityUploadCompletionHandlerBlock completionHandler = ^(AWSS3TransferUtilityUploadTask *task, NSError *error) {
    dispatch_async(dispatch_get_main_queue(), ^{
        // Do something e.g. Alert a user for transfer completion.
        // On failed uploads, `error` contains the error object.
    });
};

AWSS3TransferUtility *transferUtility = [AWSS3TransferUtility defaultS3TransferUtility];
[[transferUtility uploadFile:fileURL
                      bucket:@"YourBucketName"
                         key:@"YourObjectKeyName"
                 contentType:@"text/plain"
                  expression:expression
            completionHander:completionHandler] continueWithBlock:^id(AWSTask *task) {
    if (task.error) {
        NSLog(@"Error: %@", task.error);
    }
    if (task.exception) {
        NSLog(@"Exception: %@", task.exception);
    }
    if (task.result) {
        AWSS3TransferUtilityUploadTask *uploadTask = task.result;
        // Do something with uploadTask.
    }

    return nil;
}];

More references: https://aws.amazon.com/blogs/mobile/amazon-s3-transfer-utility-for-ios/

Answer 5

I made it work based on Zeev Vax answer. I want to provide some insight on problems I ran into and offer minor improvements.

Build a normal PutRequest, for instance

S3PutObjectRequest* putRequest = [[S3PutObjectRequest alloc] initWithKey:keyName inBucket:bucketName];

putRequest.credentials = credentials;
putRequest.filename = theFilePath;

Now we need to do some work the S3Client usually does for us

// set the endpoint, so it is not null
putRequest.endpoint = s3Client.endpoint;

// if you are using session based authentication, otherwise leave it out
putRequest.securityToken = messageTokenDTO.securityToken;

// sign the request (also computes md5 checksums etc.)
NSMutableURLRequest *request = [s3Client signS3Request:putRequest];

Now copy all of that to a new request. Amazon use their own NSUrlRequest class which would cause an exception

NSMutableURLRequest* request2 = [[NSMutableURLRequest alloc]initWithURL:request.URL];
[request2 setHTTPMethod:request.HTTPMethod];
[request2 setAllHTTPHeaderFields:[request allHTTPHeaderFields]];

Now we can start the actual transfer

NSURLSession* backgroundSession = [self backgroundSession];
_uploadTask = [backgroundSession uploadTaskWithRequest:request2 fromFile:[NSURL fileURLWithPath:theFilePath]];
[_uploadTask resume];

This is the code that creates the background session:

- (NSURLSession *)backgroundSession {
    static NSURLSession *session = nil;
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{

        NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration backgroundSessionConfiguration:@"com.example.my.unique.id"];
        session = [NSURLSession sessionWithConfiguration:configuration delegate:self delegateQueue:nil];
    });

    return session;
}

It took me a while to figure out that the session / task delegate needs to handle an auth challenge (we are in fact authentication to s3). So just implement

- (void)URLSession:(NSURLSession *)session didReceiveChallenge:(NSURLAuthenticationChallenge *)challenge completionHandler:(void (^)(NSURLSessionAuthChallengeDisposition, NSURLCredential *))completionHandler {
    NSLog(@"session did receive challenge");
    completionHandler(NSURLSessionAuthChallengePerformDefaultHandling, nil);
}

Answer 6

I just spent sometime on that, and finally succeeded. The best way is to use AWS library to create the request with the signed headers and than copy the request. It is critical to copy the request since NSURLSessionTask would fail other wise. In the code example below I used AFNetworking and sub-classed AFHTTPSessionManager, but this code also works with NSURLSession.

    @implementation MyAFHTTPSessionManager
    {

    }

    static MyAFHTTPSessionManager *sessionManager = nil;
    + (instancetype)manager {
        if (!sessionManager)
            sessionManager = [[MyAFHTTPSessionManager alloc] init];
        return sessionManager;
    }

    - (id)init {
        NSURLSessionConfiguration *sessionConfiguration = [NSURLSessionConfiguration          backgroundSessionConfiguration:toutBackgroundSessionNameAF];
        sessionConfiguration.timeoutIntervalForRequest = 30;
        sessionConfiguration.timeoutIntervalForResource = 300;
        self = [super initWithSessionConfiguration:sessionConfiguration];
        if (self)
        {
        }
        return self;
    }

    - (NSURLSessionDataTask *)POSTDataToS3:(NSURL *)fromFile
                               Key:(NSString *)key
                         completionHandler:(void (^)(NSURLResponse *response, id responseObject, NSError *error))completionHandler
    {
        S3PutObjectRequest *s3Request = [[S3PutObjectRequest alloc] initWithKey:key inBucket:_s3Bucket];
        s3Request.cannedACL = [S3CannedACL publicReadWrite];
        s3Request.securityToken = [CTUserDefaults awsS3SessionToken];
        [s3Request configureURLRequest];
        NSMutableURLRequest *request = [_s3Client signS3Request:s3Request];
        // For some reason, the signed S3 request comes back with '(null)' as a host.
        NSString *urlString = [NSString stringWithFormat:@"%@/%@/%@", _s3Client.endpoint, _s3Bucket, [key stringWithURLEncoding]] ;
        request.URL = [NSURL URLWithString:urlString];
        // Have to create a new request and copy all the headers otherwise the NSURLSessionDataTask will fail (since request get a pointer back to AmazonURLRequest which is a subclass of NSMutableURLRequest)
        NSMutableURLRequest *request2 = [[NSMutableURLRequest alloc] initWithURL:[NSURL URLWithString:urlString]];
        [request2 setHTTPMethod:@"PUT"];
        [request2 setAllHTTPHeaderFields:[request allHTTPHeaderFields]];
        NSURLSessionDataTask *task = [self uploadTaskWithRequest:request2
                                                fromFile:fromFile
                                                progress:nil 
                                       completionHandler:completionHandler];
        return task;
    }

    @end    

Another good resource is the apple sample code hereand look for "Simple Background Transfer"