How do I set the orientation for a frame-by-frame-generated video using AVFoundation?

Question

I am writing an iPhone app which takes video from the camera, runs it through some OpenGL shader code and then writes the output to a video file using AVFoundation. The app runs in lanscape orientation (either) and therefore all video recorded should be landscape.

The current code I use before starting recording to get the video the right way round is:

[[self videoWriterInput] setTransform:CGAffineTransformScale(CGAffineTransformMakeRotation(M_PI), -1.0, 1.0)];

where videoWriterInput is an instance of AVAssetWriterInput and the aim is to compensate for the landscape mode and the reveresed orientation of OpenGL.

This produces video which when downloaded and played on Quicktime player plays correctly. However, if I add the recorded video to the iPhone photo library, the thumbnail displays correctly but the video plays rotated 90 degrees if the phone is held in landscape. If the phone is held in portrait the video plays correctly but is cropped horizontally to fit the portrait dimensions.

According to this Apple tech note the capture output for AVCaptureVideoDataOutput, which I use for processing the video frames, does not support setting the video orientation.

Has anyone successfully recorded landscape generated video which can be added to the iPhone library and plays correctly in landscape and if so how?

Answer 1

Your post opened my eyes about how Apples Videos app plays back video. I recorded several items with my app with the device in the four orientations. They all played back properly oriented. I just noticed that the Videos app doesn't support rotation like the player in the Photos Album app. The Videos app expects you to hold the device (at least my iPod touch) in landscape. I did some portrait recordings, added them to iTunes, and all, including the one created with Apple's camera app, did not rotate when rotating the device to portrait orientation.

Anyway...

My app is a time lapse app that does not do any extra processing on the frames like you are doing, so YMMV on the following. I have my app set up so that it does not rotate the window as the device is rotated. This way I'm always dealing with one orientation of the device. I use AVFoundation to grab every Nth frame from the video stream and write that out.

As I set up for recording, I do the following.

inputWriterBuffer = [AVAssetWriterInput assetWriterInputWithMediaType: AVMediaTypeVideo outputSettings: outputSettings];
    // I call this explicitly before recording starts. Video plays back the right way up.
[self detectOrientation];
inputWriterBuffer.transform = playbackTransform;

That detectOrientation calls the following method. I've reduced the actual code for clarity here. In my app I also rotate some buttons, so notice they do not get the same transformation. The thing to pay attention to is how I'm setting up the playbackTransform ivar.

-(void) detectOrientation {
CGAffineTransform buttonTransform;

switch ([[UIDevice currentDevice] orientation]) {
    case UIDeviceOrientationUnknown:
        NULL;
    case UIDeviceOrientationFaceUp:
        NULL;
    case UIDeviceOrientationFaceDown:
        NULL;
        break;
    case UIDeviceOrientationPortrait:
        [UIButton beginAnimations: @"myButtonTwist" context: nil];
        [UIButton setAnimationDuration: 0.25];
        buttonTransform = CGAffineTransformMakeRotation( ( 0 * M_PI ) / 180 );
        recordingStarStop.transform = buttonTransform;
        [UIButton commitAnimations];            

        playbackTransform = CGAffineTransformMakeRotation( ( 90 * M_PI ) / 180 );
        break;
    case UIDeviceOrientationLandscapeLeft:
        [UIButton beginAnimations: @"myButtonTwist" context: nil];
        [UIButton setAnimationDuration: 0.25];
        buttonTransform = CGAffineTransformMakeRotation( ( 90 * M_PI ) / 180 );
        recordingStarStop.transform = buttonTransform;
        [UIButton commitAnimations];            

        // Transform depends on which camera is supplying video
        if (theProject.backCamera == YES) playbackTransform = CGAffineTransformMakeRotation( 0 / 180 );
        else playbackTransform = CGAffineTransformMakeRotation( ( -180 * M_PI ) / 180 );

        break;
    case UIDeviceOrientationLandscapeRight:
        [UIButton beginAnimations: @"myButtonTwist" context: nil];
        [UIButton setAnimationDuration: 0.25];
        buttonTransform = CGAffineTransformMakeRotation( ( -90 * M_PI ) / 180 );
        recordingStarStop.transform = buttonTransform;
        [UIButton commitAnimations];

        // Transform depends on which camera is supplying video
        if (theProject.backCamera == YES) playbackTransform = CGAffineTransformMakeRotation( ( -180 * M_PI ) / 180 );
        else playbackTransform = CGAffineTransformMakeRotation( 0 / 180 );

        break;
    case UIDeviceOrientationPortraitUpsideDown:
        [UIButton beginAnimations: @"myButtonTwist" context: nil];
        [UIButton setAnimationDuration: 0.25];
        buttonTransform = CGAffineTransformMakeRotation( ( 180 * M_PI ) / 180 );
        recordingStarStop.transform = buttonTransform;
        [UIButton commitAnimations];

        playbackTransform = CGAffineTransformMakeRotation( ( -90 * M_PI ) / 180 );
        break;
    default:
        playbackTransform = CGAffineTransformMakeRotation( 0 / 180 ); // Use the default, although there are likely other issues if we get here.
        break;
}
}

As a side note, since I want the method called when ever the device is rotated, and I've turned off automatic rotation, I have the following in my viewDidLoad method.

[[UIDevice currentDevice] beginGeneratingDeviceOrientationNotifications];
[[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(detectOrientation) name:@"UIDeviceOrientationDidChangeNotification" object:nil];

That's a tip I found in this SOF Q&A.

Answer 2

A lot simpler...

In your vertex shader:

uniform float preferredRotation; . . . mat4 rotationMatrix = mat4( cos(preferredRotation), -sin(preferredRotation), 0.0, 0.0, sin(preferredRotation), cos(preferredRotation), 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0);

/* 90-degrees
mat4 rotationMatrix = mat4( cos(preferredRotation), -sin(preferredRotation), (1.0 - cos(preferredRotation)) - sin(preferredRotation), 0.0,
                           -sin(preferredRotation),  cos(preferredRotation), 0.0, 0.0,
                           0.0,                     0.0, 1.0, 0.0,
                           0.0,                     0.0, 0.0, 1.0);

In the view or view controller that calls the shader:

if ([videoTrack statusOfValueForKey:@"preferredTransform" error:nil] == AVKeyValueStatusLoaded) { CGAffineTransform preferredTransform = [videoTrack preferredTransform]; self.playerView.preferredRotation = -1 * atan2(preferredTransform.b, preferredTransform.a);

...and so on...