camera2 api convert yuv420 to rgb green out

Question

i trying convert image from YUV_420_888 to rgb and i have some trouble with output image. In ImageReader i get image in format YUV_420_888 (using camera 2 api for get this image preview).

imageReader = ImageReader.newInstance(1920,1080,ImageFormat.YUV_420_888,10);

In android sdk for YuvImage class writing, that YuvImage using only NV21, YUY2.

as we can see difference between N21 and yuv420 not big and i try convert data to N21

YUV420:

and N21:

in onImageAvailable i get separately each Planes and put them in correct place (as on image)

ByteArrayOutputStream outputStream = new ByteArrayOutputStream();

ByteBuffer bufferY = image.getPlanes()[0].getBuffer();
byte[] data0 = new byte[bufferY.remaining()];
bufferY.get(data0);

ByteBuffer bufferU = image.getPlanes()[1].getBuffer();
byte[] data1 = new byte[bufferU.remaining()];
bufferU.get(data1);

ByteBuffer bufferV = image.getPlanes()[2].getBuffer();
byte[] data2 = new byte[bufferV.remaining()];
bufferV.get(data2);
...
outputStream.write(data0);
for (int i=0;i<bufferV.remaining();i++) {
    outputStream.write(data1[i]);
    outputStream.write(data2[i]);
}

after create YuvImage, convert to Bitmap, view and save

final YuvImage yuvImage = new YuvImage(outputStream.toByteArray(), ImageFormat.NV21, 1920,1080, null);
ByteArrayOutputStream outBitmap = new ByteArrayOutputStream();

yuvImage.compressToJpeg(new Rect(0, 0,1920, 1080), 95, outBitmap);

byte[] imageBytes = outBitmap.toByteArray();

final Bitmap imageBitmap = BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.length);
mImageView.setImageBitmap(imageBitmap);
...
imageBitmap.compress(Bitmap.CompressFormat.JPEG, 95, out);

but my saved image is green and pink:

what did i miss??

Answer 1

There are two main problems on your conversion attempt:

1. We can not assume that the U and V planes are isolated, they might contain interleaved data (e.g. U-PLANE = {U1, V1, U2, V2, ...} ). In fact, it might even be a NV21 style interleaving already. The key here is looking at the plane's row stride and pixel stride and also check what we can assume about the YUV_420_888 format.
2. The fact that you've commented that most of the U an V planes data are zeros indicates that you are experiencing an Android bug on the generation of images in YUV_420_888. This means that even if you get the conversion right, the image would still look green if you are affected by the bug, which was only fixed at the Android 5.1.1 and up, so it is worth to check which version you are using besides fixing the code.

Answer 2

I have implemented the YUV_420 logic (exactly as shown in the above diagram) in RenderScript, see full code here:

Conversion YUV_420 _888 to Bitmap, complete code

It produces perfect bimaps for API 22, but for API 21 it shows the "green idyll". From this I can confirm, the results you found. As already mentioned by Silvaren above, the reason seems to be an Android bug in API 21. Looking at my rs code it is clear, that if U and V information is missing (i.e. zero) the G(reen) ARGB component becomes huge during the conversion.

I see similar green pictures on my Galaxy S5 (still API 21) - here even upside down ;-). I suspect that most devices at API 21 currently do not yet use Camera2 for their device-camera apps. There is a free app called "Manual Camera Compatibility" which allows to test this. From this I see that indeed the S5/API21 still not uses Camera2...fortunately not...

Answer 3

bufferV.get(data2) increases the the position of the ByteBuffer. That's why the loop for (int i=0;i<bufferV.remaining();i++) produces 0 iterations. You can easily rewrite it as

for (int i=0; i<data1.length; i++) {
    outputStream.write(data1[i]);
    outputStream.write(data2[i]);
}

Answer 4

I got an image of ImageFormat.YUV_420_888 and was successful to save to jpeg file, and could view it correctly on windows.
I am sharing here :

private final Image mImage;
private final File mFile;
private final int mImageFormat;


ByteArrayOutputStream outputbytes = new ByteArrayOutputStream();

ByteBuffer bufferY = mImage.getPlanes()[0].getBuffer();
byte[] data0 = new byte[bufferY.remaining()];
bufferY.get(data0);

ByteBuffer bufferU = mImage.getPlanes()[1].getBuffer();
byte[] data1 = new byte[bufferU.remaining()];
bufferU.get(data1);

ByteBuffer bufferV = mImage.getPlanes()[2].getBuffer();
byte[] data2 = new byte[bufferV.remaining()];
bufferV.get(data2);

try
{
    outputbytes.write(data0);
    outputbytes.write(data2);
    outputbytes.write(data1);


    final YuvImage yuvImage = new YuvImage(outputbytes.toByteArray(), ImageFormat.NV21, mImage.getWidth(),mImage.getHeight(), null);
    ByteArrayOutputStream outBitmap = new ByteArrayOutputStream();

    yuvImage.compressToJpeg(new Rect(0, 0,mImage.getWidth(), mImage.getHeight()), 95, outBitmap);


    FileOutputStream outputfile = null;
    outputfile = new FileOutputStream(mFile);
    outputfile.write(outBitmap.toByteArray());
}
catch (IOException e)
{
    e.printStackTrace();
}
finally
{
    mImage.close();
}