Resize (downsize) YUV420sp image

Question

I am trying to resize (scale down) an image which comes in YUV420sp format. Is it possible to do such image resizing without converting it into RGB, so directly manipulating

Answer 1

Here is a Java function I use to scale down a YUV 420 (or NV21) by a factor of two.

The function takes the image in a byte array along with the width and height of the original image as an input and returns an image in a byte array which has width and heigh both equal to the half of the original width and height.

As a basis for my code I used this: Rotate an YUV byte array on Android

public static byte[] halveYUV420(byte[] data, int imageWidth, int imageHeight) {
    byte[] yuv = new byte[imageWidth/2 * imageHeight/2 * 3 / 2];
    // halve yuma
    int i = 0;
    for (int y = 0; y < imageHeight; y+=2) {
        for (int x = 0; x < imageWidth; x+=2) {
            yuv[i] = data[y * imageWidth + x];
            i++;
        }
    }
    // halve U and V color components
    for (int y = 0; y < imageHeight / 2; y+=2) {
        for (int x = 0; x < imageWidth; x += 4) {
            yuv[i] = data[(imageWidth * imageHeight) + (y * imageWidth) + x];
            i++;
            yuv[i] = data[(imageWidth * imageHeight) + (y * imageWidth) + (x + 1)];
            i++;
        }
    }
    return yuv;
}

Answer 2

YUV 4:2:0 planar looks like this:

----------------------
|     Y      | Cb|Cr |
----------------------

where:

Y = width x height pixels
Cb = Y / 4 pixels
Cr = Y / 4 pixels

Total num pixels (bytes) = width * height * 3 / 2

And the subsamling used like this:

Which means that each chroma-pixel-value is shared between 4 luma-pixels.

One approach is just to remove pixels, making sure that corresponding Y-Cb-Cr relationship are kept/recalculated.

Something close to the Nearest-neighbor interpolation but reversed.

Another approach is to first convert the 4:2:0 subsampling to 4:4:4

Here you have a 1 to 1 mapping between luma and chroma data.

This is the correct way to interpolate chroma between 4:2:0 and 4:2:2 (luma is already at correct resolution) Code in python, follow html-link for c-dito. Code is not very pythonic, just a direct translation of the c-version.

def __conv420to422(self, src, dst):
    """
    420 to 422 - vertical 1:2 interpolation filter

    Bit-exact with
    http://www.mpeg.org/MPEG/video/mssg-free-mpeg-software.html
    """
    w = self.width >> 1
    h = self.height >> 1

    for i in xrange(w):
        for j in xrange(h):
            j2 = j << 1
            jm3 = 0 if (j<3) else j-3
            jm2 = 0 if (j<2) else j-2
            jm1 = 0 if (j<1) else j-1
            jp1 = j+1 if (j<h-1) else h-1
            jp2 = j+2 if (j<h-2) else h-1
            jp3 = j+3 if (j<h-3) else h-1

            pel = (3*src[i+w*jm3]
                 -16*src[i+w*jm2]
                 +67*src[i+w*jm1]
                +227*src[i+w*j]
                 -32*src[i+w*jp1]
                  +7*src[i+w*jp2]+128)>>8

            dst[i+w*j2] = pel if pel > 0 else 0
            dst[i+w*j2] = pel if pel < 255 else 255

            pel = (3*src[i+w*jp3]
                 -16*src[i+w*jp2]
                 +67*src[i+w*jp1]
                +227*src[i+w*j]
                 -32*src[i+w*jm1]
                 +7*src[i+w*jm2]+128)>>8

            dst[i+w*(j2+1)] = pel if pel > 0 else 0
            dst[i+w*(j2+1)] = pel if pel < 255 else 255
    return dst

Run this twice to get 4:4:4. Then it's just a matter of removing rows and columns.

Or you can just quadruple the chroma-pixels to go from 4:2:0 to 4:4:4, remove rows and columns and then average 4 Cb/Cr values into 1 to get back to 4:2:0 again, it all depends on how strict you need to be :-)

Answer 3

YUV420sp has the Y in one plane and the U&V in another. If you split the U& V into separate planes, you can then perform the same scaling operation on each of the 3 planes in turn, without first having to go from 4:2:0 -> 4:4:4.

Have a look at the source code for libyuv; it just scales the planes: https://code.google.com/p/libyuv/source/browse/trunk/source/scale.cc