Android: how to display camera preview with callback?

Question

What I need to do is quite simple, I want to manually display preview from camera using camera callback and I want to get at least 15fps on a real device. I don\'t even need

Answer 1

Is there any special reason that you are forced to use GLES 1.0 ?

Because if not, see the accepted answer here: Android SDK: Get raw preview camera image without displaying it

Generally it mentions using Camera.setPreviewTexture() in combination with GLES 2.0. In GLES 2.0 you can render a full-screen-quad all over the screen, and create whatever effect you want.

It's most likely the fastest way possible.

Answer 2

I think Michael's on the right track. First you can try this method to convert from RGB to Grayscale. Clearly it's doing almost the same thing as his,but a little more succinctly for what you want.

//YUV Space to Greyscale
static public void YUVtoGrayScale(int[] rgb, byte[] yuv420sp, int width, int height){
    final int frameSize = width * height;
    for (int pix = 0; pix < frameSize; pix++){
        int pixVal = (0xff & ((int) yuv420sp[pix])) - 16;
        if (pixVal < 0) pixVal = 0;
        if (pixVal > 255) pixVal = 255;
        rgb[pix] = 0xff000000 | (pixVal << 16) | (pixVal << 8) | pixVal;
    }
}

}

Second, don't create a ton of work for the garbage collector. Your bitmaps and arrays are going to be a fixed size. Create them once, not in onFramePreview.

Doing that you'll end up with something that looks like this:

    public PreviewCallback callback = new PreviewCallback() {
    @Override
    public void onPreviewFrame(byte[] data, Camera camera) {
        if ( (mSelectView == null) || !inPreview )
            return;
        if (mSelectView.mBitmap == null)
        {
            //initialize SelectView bitmaps, arrays, etc
            //mSelectView.mBitmap = Bitmap.createBitmap(mSelectView.mImageWidth, mSelectView.mImageHeight, Bitmap.Config.RGB_565);
           //etc

        }
        //Pass Image Data to SelectView
        System.arraycopy(data, 0, mSelectView.mYUVData, 0, data.length);
        mSelectView.invalidate();
    }
};

And then the canvas where you want to put it looks like this:

class SelectView extends View {
Bitmap mBitmap;
Bitmap croppedView;
byte[] mYUVData;
int[] mRGBData;
int mImageHeight;
int mImageWidth;

public SelectView(Context context){
    super(context);
    mBitmap = null;
    croppedView = null;
}

@Override
protected void onDraw(Canvas canvas){
    if (mBitmap != null)
    {
        int canvasWidth = canvas.getWidth();
        int canvasHeight = canvas.getHeight();
        // Convert from YUV to Greyscale
        YUVtoGrayScale(mRGBData, mYUVData, mImageWidth, mImageHeight);
            mBitmap.setPixels(mRGBData, 0, mImageWidth, 0, 0, mImageWidth, mImageHeight);
            Rect crop = new Rect(180, 220, 290, 400);
        Rect dst = new Rect(0, 0, canvasWidth, (int)(canvasHeight/2));
        canvas.drawBitmap(mBitmap, crop, dst, null);
    }
    super.onDraw(canvas);
}

This example shows a cropped and distorted selection of the camera preview in real time, but you get the idea. It runs at high FPS on a Nexus S in greyscale and should work for your needs as well.

Answer 3

I'm working on exactly the same issue, but haven't got quite as far as you have.

Have you considered drawing the pixels directly to the canvas without encoding them to JPEG first? Inside the OpenCV kit http://sourceforge.net/projects/opencvlibrary/files/opencv-android/2.3.1/OpenCV-2.3.1-android-bin.tar.bz2/download (which doesn't actually use opencv; don't worry), there's a project called tutorial-0-androidcamera that demonstrates converting the YUV pixels to RGB and then writing them directly to a bitmap.

The relevant code is essentially:

public void onPreviewFrame(byte[] data, Camera camera, int width, int height) {
    int frameSize = width*height;
    int[] rgba = new int[frameSize+1];

    // Convert YUV to RGB
    for (int i = 0; i < height; i++)
        for (int j = 0; j < width; j++) {
            int y = (0xff & ((int) data[i * width + j]));
            int u = (0xff & ((int) data[frameSize + (i >> 1) * width + (j & ~1) + 0]));
            int v = (0xff & ((int) data[frameSize + (i >> 1) * width + (j & ~1) + 1]));
            y = y < 16 ? 16 : y;

            int r = Math.round(1.164f * (y - 16) + 1.596f * (v - 128));
            int g = Math.round(1.164f * (y - 16) - 0.813f * (v - 128) - 0.391f * (u - 128));
            int b = Math.round(1.164f * (y - 16) + 2.018f * (u - 128));

            r = r < 0 ? 0 : (r > 255 ? 255 : r);
            g = g < 0 ? 0 : (g > 255 ? 255 : g);
            b = b < 0 ? 0 : (b > 255 ? 255 : b);

            rgba[i * width + j] = 0xff000000 + (b << 16) + (g << 8) + r;
        }

    Bitmap bmp = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
    bmp.setPixels(rgba, 0/* offset */, width /* stride */, 0, 0, width, height);
    Canvas canvas = mHolder.lockCanvas();
    if (canvas != null) {
        canvas.drawBitmap(bmp, (canvas.getWidth() - width) / 2, (canvas.getHeight() - height) / 2, null);
        mHolder.unlockCanvasAndPost(canvas);
    } else {
        Log.w(TAG, "Canvas is null!");
    }
    bmp.recycle();
}

Of course you'd have to adapt it to meet your needs (ex. not allocating rgba each frame), but it might be a start. I'd love to see if it works for you or not -- i'm still fighting problems orthogonal to yours at the moment.

Answer 4

Is this not what you want? Just use a SurfaceView in your layout, then somewhere in your init like onResume():

SurfaceView surfaceView = ...
SurfaceHolder holder = surfaceView.getHolder();
...
Camera camera = ...;
camera.setPreviewDisplay(holder);

It just sends the frames straight to the view as fast as they arrive.

If you want grayscale, modify the camera parameters with setColorEffect("mono").

Answer 5

I believe I read in a blog that the grayscale data is in the first x*y bytes. Yuv should represent luminance, so the data is there, although it isn't a perfect grayscale. Its great for relative brightness, but not grayscale, as each color isn't as bright as each other in rgb. Green is usually given a stronger weight in luminosity conversions. Hope this helps!

Answer 6

For very basic and simple effects, there is

Camera.Parameters parameters = mCamera.getParameters();
parameters.setColorEffect(Parameters.EFFECT_AQUA);

I figured out that this effects do DIFFERENTLY depending on the device. For instance, on my phone (galaxy s II) it looks kinda like a comic effect as in contrast to the galaxy s 1 it is 'just' a blue shade.

It's pro: It's working as live-preview.

I looked around some other camera apps and they obviously also faced this problem. So what did they do? They are capturing the default camera image, applying a filter to the bitmap data, and show this image in a simple ImageView. It's for sure not that cool as in live preview, but you won't ever face performance problems.