Get Mat object from Camera frame Android for OpenCV

Question

I am developing an android application based on Tutorial 2 given by OpenCV4Android. What I want to achieve is to get the Mat object of the camera frame when I touch the phone's screen (onTouch()). I want to modify the code such that I store the image as Mat instead of a jpeg file on the phone's memory. The Mat will then go under further processing.

Do I need to involve the onCameraFrame() function? Any guidance will be much appreciated. I am very new to Android developing and OpenCV as well.

Note: I am using Android version 4.2.2 and OpenCV2.4.8.

Edit:

After editing onTouch() and onCameraFrame() functions these are the code snippets:

public Mat onCameraFrame(CvCameraViewFrame inputFrame) {

        Mat img = inputFrame.rgba();

        if(touched) {
            int imgsize = (int) (img.total()*img.channels());
            byte[] data = new byte[imgsize];
            img.get(0,0,data);
            int col = img.cols();
            int row = img.rows();
            Toast.makeText(this, "size:"+imgsize+" row:"+row+" col:"+col, Toast.LENGTH_SHORT).show();

            StartSocket(row, col, imgsize, data);

            touched = false;
        }

        return img;
    }

    @Override
    public boolean onTouch(View v, MotionEvent event) {
        Log.i(TAG,"onTouch event"); 
        touched = true;
        return true;
    }

What I did is basically convert the Mat object into a ByteArray called data[] and send it to a server through a socket along with other information. The socket works fine as a separate project when i try to send text files, so I believe there is nothing wrong with it.

And I don't have the LogCat since I am running the app directly on my device.

Answer 1

if you want to process images from the camera, yes, you'll need the onCameraFrame() method, as it provides your image

public class MyActivity extends Activity implements CvCameraViewListener2,OnTouchListener {
    boolean touched=false;
    public boolean onTouch(View v, MotionEvent event) {
         touched = true;
         return true;
    }
    public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
        Mat rgba = inputFrame.rgba();
        if ( touched ) {
            // do some processing on rgba Mat
            touched = false;
        }
        return rgba;
    }
}

Answer 2

I figured out the solution to the problem. Ofcourse I wouldn't have done it without @berak help. From LogCat I found out that the failure was because of an issue with the threads. So I moved the following lines from onTouch() to StartSocket() function:

int imgsize = (int) (img.total()*img.channels());
            byte[] data = new byte[imgsize];
            img.get(0,0,data);
            int col = img.cols();
            int row = img.rows();
            Toast.makeText(this, "size:"+imgsize+" row:"+row+" col:"+col, Toast.LENGTH_SHORT).show();

I will put the relevant code snippet here so that others get use of it.

public class Tutorial3Activity extends Activity implements CvCameraViewListener2, OnTouchListener {

    static TextView msgReceived;
    String serverMsg = "";

    boolean touched = false;

    public Mat onCameraFrame(CvCameraViewFrame inputFrame) {

        Mat image = inputFrame.rgba();

        if(touched) {
            StartSocket(image); 
            touched = false;
        }

        return image;
    }

    @Override
    public boolean onTouch(View v, MotionEvent event) {
        Log.i(TAG,"onTouch event"); 
        touched = true;

        return true;
    }

    private void StartSocket(final Mat img) {
        Thread t = new Thread(new Runnable(){

            @Override
            public void run() {
                try {
                    final int imgsize = (int) (img.total()*img.channels());
                    byte[] data = new byte[imgsize];
                    img.get(0,0,data);
                    final int col = img.cols();
                    final int row = img.rows();

                    mHandler.post(new Runnable() {
                        @Override
                        public void run() {
                            Toast.makeText(getBaseContext(), "size:"+imgsize+" row:"+row+" col:"+col, Toast.LENGTH_SHORT).show();
                        }
                    });

                    //--- SEND IMAGE TO SERVER ---//
                    Socket s = new Socket ("192.168.1.25", 6000);

                    DataOutputStream dout = new DataOutputStream(s.getOutputStream());
                    dout.writeInt(col);
                    dout.writeInt(row);
                    dout.writeInt(imgsize);
                    dout.write(data);

                    mHandler.post(new Runnable() {
                        @Override
                        public void run() {
                            Toast.makeText(getBaseContext(), "Dout written", Toast.LENGTH_SHORT).show();
                        }
                    });

                    dout.flush();
                                        dout.close();
                    s.close(); //close socket

                    } catch (UnknownHostException e) {
                        e.printStackTrace();
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
            });

            t.start();
    }