ANDROID - color detection using openCV - how to?

Question

my goal is to display a threshed image using the HSV color space in a way that only yellow objects will be shown. i use this code (based on a code given by the openCV 2.3.1 android samples):

protected Bitmap processFrame(VideoCapture capture) {
    //capture.retrieve(mGray, Highgui.CV_CAP_ANDROID_GREY_FRAME);
    //Imgproc.cvtColor(mGray, mRgba, Imgproc.COLOR_GRAY2RGBA, 4);

    capture.retrieve(mHSV, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA);
    Imgproc.cvtColor(mHSV, mRgba, Imgproc.COLOR_RGB2HSV, 4);
    //Core.inRange(mRgba, new Scalar(20, 100, 100), new Scalar(30, 255, 255), mRgba);

    Bitmap bmp = Bitmap.createBitmap(mRgba.cols(), mRgba.rows(), Bitmap.Config.ARGB_8888);

    if (Utils.matToBitmap(mRgba, bmp))
        return bmp;

    bmp.recycle();
    return null;
}

the base (Abstract)class contains the "run" method:

protected abstract Bitmap processFrame(VideoCapture capture);

public void run() {
...
bmp = processFrame(mCamera);
...
canvas.drawBitmap(bmp, (canvas.getWidth() - bmp.getWidth()) / 2, (canvas.getHeight() - bmp.getHeight()) / 2, null);
...
}

i get this distorted preview which i think i can understand (HSV format) but why is it repeating itself (i`v draw a green line to emphasize it) 4 time? and what is the black horizontal line?

what am i doing wrong?

one last thing, what is the logic behind:

Imgproc.cvtColor(mHSV, mRgba, Imgproc.COLOR_RGB2HSV, 4);

why is it COLOR_RGB2HSV? shouldnt it be COLOR_HSV2RGB?

Let's say i'v passed this problem, how can i make a gray level image with the yellow objects in their native color? i thought using the Core.inRange() method but when i do this i get black screen.

yes, i guess i look like a total jerk but i need to start from somewhere, don't i?

10x!

Update 1: i tried to do RGB->HSV->RGB this way:

 @Override
protected Bitmap processFrame(VideoCapture capture) {
    //capture.retrieve(mGray, Highgui.CV_CAP_ANDROID_GREY_FRAME);
    //Imgproc.cvtColor(mGray, mRgba, Imgproc.COLOR_GRAY2RGBA, 4);

    capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGB);
    Imgproc.cvtColor(mRgba, mHSV, Imgproc.COLOR_RGB2HSV,0);
    //Imgproc.cvtColor(mRgba, mRgba, Imgproc.COLOR_BGR2RGB, 4);
    //Core.inRange(mRgba, new Scalar(20, 100, 100), new Scalar(30, 255, 255), mRgba);
    Imgproc.cvtColor(mHSV,mRgba , Imgproc.COLOR_HSV2RGB,0);

    Bitmap bmp = Bitmap.createBitmap(mRgba.cols(), mRgba.rows(), Bitmap.Config.ARGB_8888);

    if (Utils.matToBitmap(mRgba, bmp))
        return bmp;

    bmp.recycle();
    return null;
}

and i got:

?

Update 2:

i finally understand that before setting a frame, it must be converted into RGBA space. so i now tried the threshold with the code as follow:

capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA);
    Imgproc.cvtColor(mRgba, mHSV, Imgproc.COLOR_RGB2HSV,0);
    Core.inRange(mHSV, new Scalar(20, 100, 100), new Scalar(30, 255, 255), mHSVThreshed);
    Imgproc.cvtColor(mHSVThreshed, mRgba, Imgproc.COLOR_HSV2RGB, 0);
    Imgproc.cvtColor(mRgba, mRgba, Imgproc.COLOR_RGB2RGBA, 0);
    Bitmap bmp = Bitmap.createBitmap(mRgba.cols(), mRgba.rows(), Bitmap.Config.ARGB_8888);

but now it gives me force shutdown... any ideas?

Answer 1

I think mHSVThreshed is a binary mat

so maybe this line :

Imgproc.cvtColor(mHSVThreshed, mRgba, Imgproc.COLOR_HSV2RGB, 0);

should change to :

Imgproc.cvtColor(mHSVThreshed, mRgba, Imgproc.COLOR_GRAY2RGB, 0);

I spent a lot of time dealing with the "showing" problem too...

hope this help...

Answer 2

friends. i give you the result of 1 month of hard work and help from friends across the ocean:

Ethan was right. but the code needed some fixing.

the code:

    capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_COLOR_FRAME_BGRA);
    Imgproc.cvtColor(mRgba, mHSV, Imgproc.COLOR_BGR2HSV,3);
    Core.inRange(mHSV, new Scalar(0, 100, 30), new Scalar(5, 255, 255), mHSVThreshed);
    Imgproc.cvtColor(mHSVThreshed, mRgba, Imgproc.COLOR_GRAY2BGR, 0);
    Imgproc.cvtColor(mRgba, mRgba2, Imgproc.COLOR_BGR2RGBA, 0);
    Bitmap bmp = Bitmap.createBitmap(mRgba2.cols(), mRgba2.rows(), Bitmap.Config.ARGB_8888);


    if (Utils.matToBitmap(mRgba2, bmp))...

first, the mat is binary 0 or 255 so the transform to gray level is more "natural". second, the conversion from HSVto RGBis in fact HSV-BGR!!. and last thing is that the preview is expecting RGBA Bitmap.

thats it. hope other can benefit from this post. SHALOM!

Answer 3

Well as far as I see it you fetch the image frame in RGBA and save it under the name "mHSV"

capture.retrieve(mHSV, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA);

so you should there store it as mRgba

in the cvtColor you need to transform it to HSV via COLOR_RGBA2HSV. Assuming you have changed the names this would be:

Imgproc.cvtColor(mRgba, mRSV, Imgproc.COLOR_RGB2HSV, 0);

And I assume this repetition of the images comes from the "4" in you cvtColor function since your HSV picture will only have 3 channels. Put in a 0 there and it should be detected automatically...

I hope it helps...

Answer 4

why is it COLOR_RGB2HSV? shouldnt it be COLOR_HSV2RGB?

I would say that it should :). Looks like a problem on how opencv grabs the pixels. Check that both your input and output images have the same size and numberof channels.

This

capture.retrieve(mHSV, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA);

looks also weird to me, Are you storing the RGBA image that you get into an image at HSV format? That would explain the problem. Try to do something like that : RGB(capture retrieve) => HSV(cvt color) => color detection => RGB (cvt color again)