PreviewCallback onPreviewFrame does not change data

Question

I want to do some image processing with images from camera and display it on a SurfaceView but I don\'t know how to modify the camera frame. I tried to use setPreviewCallbac

Answer 1

You cannot modify the preview data sent to a SurfaceView, if you're using the setPreviewDisplay() call. The preview video stream is managed entirely outside of your application and isn't accessible to it.

There are a few options you can take:

1. You can place a second view on top of the SurfaceView, such as an ImageView or another SurfaceView, and draw the data received by the onPreviewFrame callback into this view. You'll have to do some color/pixel format conversion from the preview callback format (usually NV21) for display, and obviously you have to run your image processing on that data first as well. This isn't very efficient, unless you're willing to write some JNI code.

2. On Android 3.0 or newer, you can use the Camera.setPreviewTexture() method, and pipe the camera preview stream into an OpenGL texture by using a SurfaceTexture object, which you can then manipulate in OpenGL before displaying. Then you don't need the preview callbacks at all. This is more efficient if GPU processing is sufficient. You can also use the OpenGL readPixels call to get the processed preview data back to your application, if you want to display it/process it some other way.

Answer 2

Maybe it will be helpfull to someone. I have solved this problem by using the OpenCV library for retrieving Frames from a Camera. In the OpenCv 3 there is a method onCameraFrame(CvCameraViewFrame inputFrame):

    public Mat onCameraFrame(CvCameraViewFrame inputFrame) {

    // here you can do something with inputFrame before it appears on the preview

    return inputFrame.rgba();
}

You can just try the Camera Preview project from the samples folder.

Or you can do this in the ndk https://vec.io/posts/how-to-render-image-buffer-in-android-ndk-native-code But I haven't tryed this jet.

Or here you can find decoding from YUV to RGB in C/C++ with NDK https://github.com/youten/YUV420SP