How to use image format YUV_420_888 for MLKIT of Google

问题

ImageReader get each frame from camera preview is image with format YUV_420_888, i want to use it as input for MLKIT.

In document of google, i can run detector with input is:

• Bitmap
• media.Image
• ByteBuffer
• ByteArray
• A File

I tried convert YUV_420_888 to above objects but I failed

回答1:

I used this function to convert the image frame to NV21 format and set the image type in metadata in FirebaseVisionImage

fun YUV_420_888toNV21(image: ImageProxy): ByteArray {

            val width = image.width
            val height = image.height
            val ySize = width * height
            val uvSize = width * height / 4

            val nv21 = ByteArray(ySize + uvSize * 2)
            val yBuffer = image.planes[0].buffer // Y
            val uBuffer = image.planes[1].buffer // U
            val vBuffer = image.planes[2].buffer // V

            var rowStride = image.planes[0].rowStride
            assert(image.planes[0].pixelStride == 1)

            var pos = 0

            //may need to flip the buffers if you get underflow exception

            if (rowStride == width) { // likely
                yBuffer.get(nv21, 0, ySize)
                pos += ySize

            } else {
                var yBufferPos = (width - rowStride).toLong() // not an actual position
                while (pos < ySize) {
                    yBufferPos = yBufferPos + (rowStride - width).toLong()
                    yBuffer.position(yBufferPos.toInt())
                    yBuffer.get(nv21, pos, width)
                    pos += width
                }
            }

            rowStride = image.planes[2].rowStride
            val pixelStride = image.planes[2].pixelStride

            assert(rowStride == image.planes[1].rowStride)
            assert(pixelStride == image.planes[1].pixelStride)

            if (pixelStride == 2 && rowStride == width && uBuffer.get(0) == vBuffer.get(1)) {
                // maybe V an U planes overlap as per NV21, which means vBuffer[1] is alias of uBuffer[0]
                val savePixel = vBuffer.get(1)
                vBuffer.put(1, 0.toByte())
                if (uBuffer.get(0).toInt() == 0) {
                    vBuffer.put(1, 255.toByte())
                    if (uBuffer.get(0).toInt() == 255) {
                        vBuffer.put(1, savePixel)
                        vBuffer.get(nv21, ySize, uvSize)
                        //Log.d("NV211",DataConverter.jsonify(nv21))
                        return nv21 // shortcut
                    }
                }

                // unfortunately, the check failed. We must save U and V pixel by pixel
                vBuffer.put(1, savePixel)
            }

            // other optimizations could check if (pixelStride == 1) or (pixelStride == 2),
            // but performance gain would be less significant

            for (row in 0 until height / 2) {
                for (col in 0 until width / 2) {
                    val vuPos = col * pixelStride + row * rowStride
                    nv21[pos++] = vBuffer.get(vuPos)
                    nv21[pos++] = uBuffer.get(vuPos)
                }
            }
            //Log.d("NV212",DataConverter.jsonify(nv21))
            return nv21
        }

来源：https://stackoverflow.com/questions/51144854/how-to-use-image-format-yuv-420-888-for-mlkit-of-google