Processing Android camera frames in real time

Question

I\'m trying to create an Android application that will process camera frames in real time. To start off with, I just want to display a grayscale version of what the camera s

Answer 1

This very old post has caught my attention now.

The API available in '11 was much more limited. Today one can use SurfaceTexture (see example) to preview camera stream after (some) manipulations.

Answer 2

This is not an easy task to achieve, with the current Android tools/API available. In general, realtime image-processing is better done at the NDK level. To just show the black and white, you can still do it in java. The byte array containing the frame data is in YUV format, where the Y-Plane comes first. So, if you get the just the Y-plane alone (first width x height bytes), it already gives you the black and white.

I did achieve this through extensive work and trials. You can view the app at google: https://play.google.com/store/apps/details?id=com.nm.camerafx

Answer 3

You can get extensive guidance from the OpenCV4Android SDK. Look into their available examples, specifically Tutorial 1 Basic. 0 Android Camera

But, as it was in my case, for intensive image processing, this will get slower than acceptable for a real-time image processing application. A good replacement for their onPreviewFrame 's byte array conversion to YUVImage:

YuvImage yuvImage = new YuvImage(frame, ImageFormat.NV21, width, height, null);

Create a rectangle the same size as the image.

Create a ByteArrayOutputStream and pass this, the rectangle and the compression value to compressToJpeg():

ByteArrayOutputStream baos = new ByteArrayOutputStream(); yuvimage.compressToJpeg(imageSizeRectangle, 100, baos);

byte [] imageData = baos.toByteArray();

Bitmap previewBitmap = BitmapFactory.decodeByteArray(imageData , 0, imageData .length);

Rendering these previewFrames on a surface and the best practices involved is a new dimension. =)