OpenCV for Android: Convert Camera preview from YUV to RGB with Imgproc.cvtColor

Question

I get a runtime error if I try to convert camera preview YUV byte array to a RGB(A) byte array with Imgproc.cvtColor( mYUV_Mat, mRgba_Mat, Imgproc.COLOR_YUV420sp2RGBA, 4 ) in onPreviewFrame(byte[] data, Camera camera):

Preview.java:

     mCamera.setPreviewCallback(new PreviewCallback() {
          public void onPreviewFrame(byte[] data, Camera camera)
          {
            // Pass YUV data to draw-on-top companion
            System.arraycopy(data, 0, mDrawOnTop.mYUVData, 0, data.length);
            mDrawOnTop.invalidate();
          }
       });

DrawOnTop.java:

public class DrawOnTop extends View {
Bitmap mBitmap;
Mat mYUV_Mat;
protected void onDraw(Canvas canvas) {
    if (mBitmap != null)
    {

        canvasWidth = canvas.getWidth();
        canvasHeight = canvas.getHeight();
        int newImageWidth = 640;
        int newImageHeight = 480;
        marginWidth = (canvasWidth - newImageWidth)/2;

        if( mYUV_Mat != null ) mYUV_Mat.release();

        //mYUV_Mat = new Mat( newImageWidth, newImageHeight, CvType.CV_8UC1 );
        mYUV_Mat = new Mat( newImageWidth, newImageHeight, CvType.CV_8UC4 );
        mYUV_Mat.put( 0, 0, mYUVData );

        //Mat mRgba_Mat = new Mat();
        Mat mRgba_Mat = new Mat(newImageWidth,newImageHeight,CvType.CV_8UC4);

        //Mat mRgba_Mat = mYUV_Mat;

        //Imgproc.cvtColor( mYUV_Mat, mRgba_Mat, Imgproc.COLOR_YUV2RGBA_NV21, 4 );
        //Imgproc.cvtColor( mYUV_Mat, mRgba_Mat, Imgproc.COLOR_YUV420sp2RGB, 4 );
        Imgproc.cvtColor( mYUV_Mat, mRgba_Mat, Imgproc.COLOR_YUV420sp2RGBA, 4 );


        // Draw Bitmap New:
        Bitmap mBitmap = Bitmap.createBitmap( newImageWidth, newImageHeight, Bitmap.Config.ARGB_8888 );
        Utils.matToBitmap( mRgba_Mat, mBitmap );
        mRgba_Mat.release();
}
}

The conversion mYUV_Mat.put( 0, 0, mYUVData ) runs correctly. But the attempts to convert mYUV_Mat to mRgb_Mat using Imgproc.cvtColor lead all to runtime errors ("Source not found." with Eclipse).

What is the correct Imgproc.cvtColor command for my goal?

(I don't want to use a Java YUV2RGB(A) decode method because it's to slow for image processing. I want to use the OpenCV Imgproc.cvtColor method because I can do native calls)

Answer 1

Maybe the Imgproc library isn't properly included in your project, but other OpenCV libraries are? The line that crashes is the first line where you use a method from Imgproc, which would explain why earlier parts of the code run correctly.

Your code looks fine, except you can use the no-argument constructor for mRgba_Mat (since most OpenCV4Android functions, including cvtColor, can infer the required size of the destination matrix), and you're allocating a lot of wasted space for mYUV_Mat. You don't need a full 4 channels if you just allocate YUV matrices 50% more space than their RGB counterparts:

mYUV_Mat = new Mat( newImageHeight + newImageHeight / 2, newImageWidth, CvType.CV_8UC1 );