Converting YUV_420_888 to JPEG and saving file results distorted image

问题:

I've used the ImageUtil class provided in https://stackoverflow.com/a/40152147/2949966 within my git repo: https://github.com/ahasbini/cameraview/tree/camera_preview_imp (note the implementation is in camera_preview_imp branch) to implement a frame preview callback. An ImageReader is set to preview frames in the ImageFormat.YUV_420_888 format which will be converted into ImageFormat.JPEG using the ImageUtil class and send it to the frame callback. The demo app saves a frame from the callback to a file every 50 frames. All of the saved frame images are coming out distorted similar to below:

If I've changed the ImageReader to use ImageFormat.JPEG instead by doing the following changes in Camera2:

mPreviewImageReader = ImageReader.newInstance(previewSize.getWidth(),     previewSize.getHeight(), ImageFormat.JPEG, /* maxImages */ 2); mCamera.createCaptureSession(Arrays.asList(surface, mPreviewImageReader.getSurface()),     mSessionCallback, null); 

the image is coming properly without any distortions however the frame rate drops significantly and the view starts to lag. Hence I believe the ImageUtil class is not converting properly.

回答1:

Solution provided by @volodymyr-kulyk does not take into consideration the row stride of the planes within the image. Below code does the trick (image is of android.media.Image type):

data = NV21toJPEG(YUV420toNV21(image), image.getWidth(), image.getHeight(), 100); 

And the implementations:

private static byte[] NV21toJPEG(byte[] nv21, int width, int height, int quality) {     ByteArrayOutputStream out = new ByteArrayOutputStream();     YuvImage yuv = new YuvImage(nv21, ImageFormat.NV21, width, height, null);     yuv.compressToJpeg(new Rect(0, 0, width, height), quality, out);     return out.toByteArray(); }  private static byte[] YUV420toNV21(Image image) {     Rect crop = image.getCropRect();     int format = image.getFormat();     int width = crop.width();     int height = crop.height();     Image.Plane[] planes = image.getPlanes();     byte[] data = new byte[width * height * ImageFormat.getBitsPerPixel(format) / 8];     byte[] rowData = new byte[planes[0].getRowStride()];      int channelOffset = 0;     int outputStride = 1;     for (int i = 0; i < planes.length; i++) {         switch (i) {             case 0:                 channelOffset = 0;                 outputStride = 1;                 break;             case 1:                 channelOffset = width * height + 1;                 outputStride = 2;                 break;             case 2:                 channelOffset = width * height;                 outputStride = 2;                 break;         }          ByteBuffer buffer = planes[i].getBuffer();         int rowStride = planes[i].getRowStride();         int pixelStride = planes[i].getPixelStride();          int shift = (i == 0) ? 0 : 1;         int w = width >> shift;         int h = height >> shift;         buffer.position(rowStride * (crop.top >> shift) + pixelStride * (crop.left >> shift));         for (int row = 0; row < h; row++) {             int length;             if (pixelStride == 1 && outputStride == 1) {                 length = w;                 buffer.get(data, channelOffset, length);                 channelOffset += length;             } else {                 length = (w - 1) * pixelStride + 1;                 buffer.get(rowData, 0, length);                 for (int col = 0; col < w; col++) {                     data[channelOffset] = rowData[col * pixelStride];                     channelOffset += outputStride;                 }             }             if (row < h - 1) {                 buffer.position(buffer.position() + rowStride - length);             }         }     }     return data; } 

Method was gotten from the following link.

回答2:

Updated ImageUtil:

public final class ImageUtil {      public static byte[] NV21toJPEG(byte[] nv21, int width, int height, int quality) {         ByteArrayOutputStream out = new ByteArrayOutputStream();         YuvImage yuv = new YuvImage(nv21, ImageFormat.NV21, width, height, null);         yuv.compressToJpeg(new Rect(0, 0, width, height), quality, out);         return out.toByteArray();     }      // nv12: true = NV12, false = NV21     public static byte[] YUV_420_888toNV(ByteBuffer yBuffer, ByteBuffer uBuffer, ByteBuffer vBuffer, boolean nv12) {         byte[] nv;          int ySize = yBuffer.remaining();         int uSize = uBuffer.remaining();         int vSize = vBuffer.remaining();          nv = new byte[ySize + uSize + vSize];          yBuffer.get(nv, 0, ySize);         if (nv12) {//U and V are swapped             vBuffer.get(nv, ySize, vSize);             uBuffer.get(nv, ySize + vSize, uSize);         } else {             uBuffer.get(nv, ySize , uSize);             vBuffer.get(nv, ySize + uSize, vSize);         }         return nv;     }      public static byte[] YUV_420_888toI420SemiPlanar(ByteBuffer yBuffer, ByteBuffer uBuffer, ByteBuffer vBuffer,                                                      int width, int height, boolean deInterleaveUV) {         byte[] data = YUV_420_888toNV(yBuffer, uBuffer, vBuffer, deInterleaveUV);         int size = width * height;         if (deInterleaveUV) {             byte[] buffer = new byte[3 * width * height / 2];              // De-interleave U and V             for (int i = 0; i < size / 4; i += 1) {                 buffer[i] = data[size + 2 * i + 1];                 buffer[size / 4 + i] = data[size + 2 * i];             }             System.arraycopy(buffer, 0, data, size, size / 2);         } else {             for (int i = size; i < data.length; i += 2) {                 byte b1 = data[i];                 data[i] = data[i + 1];                 data[i + 1] = b1;             }         }         return data;     } } 

Operations to write in file byte[] data as JPEG:

//image.getPlanes()[0].getBuffer(), image.getPlanes()[1].getBuffer() //image.getPlanes()[2].getBuffer(), image.getWidth(), image.getHeight() byte[] nv21 = ImageUtil.YUV_420_888toI420SemiPlanar(yBuffer, uBuffer, vBuffer, width, height, false); byte[] data = ImageUtil.NV21toJPEG(nv21, width, height, 100); //now write `data` to file 

!!! do not forget to close image after processing !!!

image.close(); 

回答3:

Camera2 YUV_420_888 to Jpeg in Java(Android):

@Override public void onImageAvailable(ImageReader reader){     Image image = null;      try {         image = reader.acquireLatestImage();         if (image != null) {              byte[] nv21;             ByteBuffer yBuffer = mImage.getPlanes()[0].getBuffer();             ByteBuffer uBuffer = mImage.getPlanes()[1].getBuffer();             ByteBuffer vBuffer = mImage.getPlanes()[2].getBuffer();              int ySize = yBuffer.remaining();             int uSize = uBuffer.remaining();             int vSize = vBuffer.remaining();              nv21 = new byte[ySize + uSize + vSize];              //U and V are swapped             yBuffer.get(nv21, 0, ySize);             vBuffer.get(nv21, ySize, vSize);             uBuffer.get(nv21, ySize + vSize, uSize);              String savingFilepath = getYUV2jpg(nv21);            }     } catch (Exception e) {         Log.w(TAG, e.getMessage());     }finally{         image.close();// don't forget to close     } }    public String getYUV2jpg(byte[] data) {     File imageFile = new File("your parent directory", "picture.jpeg");//no i18n     BufferedOutputStream bos = null;     try {         bos = new BufferedOutputStream(new FileOutputStream(imageFile));         bos.write(data);         bos.flush();         bos.close();     } catch (IOException e) {          return e.getMessage();     } finally {         try {             if (bos != null) {                 bos.close();             }         } catch (Exception e) {             e.printStackTrace();         }      }     return imageFile.getAbsolutePath(); } 

Note: Handle the image rotation issue.

文章来源: Converting YUV_420_888 to JPEG and saving file results distorted image