Pass frame(mat data in android ) from android to native c++ and detect face

问题

I give the camera frame and init OpenCV in an Android project but I don't know how to pass the frame to a C++ method and execute face detection from there ...

How can I?

I use CameraBridgeViewBase.CvCameraViewListener2 and then I get frame like this:

public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
    mRgba = inputFrame.rgba();
    Core.transpose(mRgba, mRgbaT);
    Imgproc.resize(mRgbaT, mRgbaF, mRgbaF.size(), 0,0, 0);
    Core.flip(mRgbaF, mRgba, 1 );
    return mRgba; 
}

I just don't know how to send a camera frame from android to c++ and then send back that camera frame from c++ to android.

回答1:

You have to setup OpenCV on Android with Native Development Kit support (NDK). This Android NDK enables you to implement your OpenCV image processing pipeline in C++ and call that C++ code from Android Kotlin/Java code through JNI (Java Native Interface).

You will first have to define a java native function in MainActivity.java :

public native void processFrame(long matAddrGray);

Your will then define a C++ equivalent in a native-lib.cpp file :

void JNICALL Java_com_example_nativeopencvandroidtemplate_MainActivity_processFrame(JNIEnv *env, jobject instance, jlong matAddrGray)

Calling the processFrame native function from java will actually invoke the C++ equivalent :

 public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
        Mat matGray = inputFrame.gray();
        processFrame(matGray.getNativeObjAddr());
        return matGray;
    }

You can then retrieve your Mat object in C++ and process it :

void JNICALL Java_com_example_nativeopencvandroidtemplate_MainActivity_processFrame(JNIEnv *env, jobject instance, jlong matAddrGray) {
     Mat &mGr = *(Mat *) matAddrGray;

     // process Mat
     ...
}

Here is a tutorial on Github with a sample OpenCV native application and instructions on how to setup OpenCV 4.1.1 for Android in Android Studio with NDK support.

Disclaimer: I wrote that tutorial

来源：https://stackoverflow.com/questions/57975365/pass-framemat-data-in-android-from-android-to-native-c-and-detect-face