Rotate camera preview to Portrait Android OpenCV Camera

Question

I am trying to use OpenCV 2.4.3.2 to create a camera app and do some opencv processing. I would like it to be able to have multiple UI orientations, not just Landscape.

Answer 1

You have to consider a few things:

• onPreviewFrame() always delivers the raw camera data in its assambled rotation
• getSupportedPreviewSizes() gives corresponding aspect ratios
• Algorithm needs to analyze the frame in portrait to detect objects correct.
• the Bitmap created (Java-side) to store the resulting frame also needs the correct aspect ratio

So, for a fast and high-resolution solution i changed JavaCameraView.java and my JNI-part. in JavaCameraView.java:

...

      if (sizes != null) {
         /* Select the size that fits surface considering maximum size allowed */
         Size frameSize;
         if(width > height)
         {
            frameSize = calculateCameraFrameSize(sizes, new JavaCameraSizeAccessor(), width, height);
         }else{
            frameSize = calculateCameraFrameSize(sizes, new JavaCameraSizeAccessor(), height, width);
         }
...

         mCamera.setParameters(params);
         params = mCamera.getParameters();

         int bufFrameWidth, bufFrameHeight;
         bufFrameWidth = params.getPreviewSize().width;
         bufFrameHeight = params.getPreviewSize().height;

         if(width > height) {
             mFrameWidth = params.getPreviewSize().width;
             mFrameHeight = params.getPreviewSize().height;
         }else{
             mFrameWidth = params.getPreviewSize().height;
             mFrameHeight = params.getPreviewSize().width;
         }
...

         mFrameChain = new Mat[2];
         mFrameChain[0] = new Mat(bufFrameHeight + (bufFrameHeight/2), bufFrameWidth, CvType.CV_8UC1);
         mFrameChain[1] = new Mat(bufFrameHeight + (bufFrameHeight/2), bufFrameWidth, CvType.CV_8UC1);

         AllocateCache();

         mCameraFrame = new JavaCameraFrame[2];
         mCameraFrame[0] = new JavaCameraFrame(mFrameChain[0], bufFrameWidth, bufFrameHeight);
         mCameraFrame[1] = new JavaCameraFrame(mFrameChain[1], bufFrameWidth, bufFrameHeight);

With these changes, we made sure we are using the highest resultion available for portrait (switches height/width in calculateCameraFrameSize). We are still handling landscape as input from onPreviewFrame() but created a Bitmap to draw in portrait (AllocateCache).

Last, we need to give the algorithm the portrait-frame in order to let him detect "standing" objects and return it for saving and rendering the bitmap. So following modifications to your Activity:

public Mat rot90(Mat matImage, int rotflag){
    //1=CW, 2=CCW, 3=180
    Mat rotated = new Mat();
    if (rotflag == 1){
        rotated = matImage.t();
        flip(rotated, rotated, 1); //transpose+flip(1)=CW
    } else if (rotflag == 2) {
        rotated = matImage.t();
        flip(rotated, rotated,0); //transpose+flip(0)=CCW
    } else if (rotflag ==3){
        flip(matImage, rotated,-1);    //flip(-1)=180
    } else if (rotflag != 0){ //if not 0,1,2,3:
       Log.e(TAG, "Unknown rotation flag("+rotflag+")");
    }
    return rotated;
}

public Mat onCameraFrame(CvCameraViewFrame inputFrame) {

    mRgba = rot90(inputFrame.rgba(), 1);
    mGray = rot90(inputFrame.gray(), 1);
...