Extract Video frames in Android

Question

I wanted to know if it is possible to extract frames from a running Video in Android? I need to extract frames at regular intervals and send them for further processing.

Would someone be able to find an answer for me?

Thanks,

Abhi

Answer 1

You can use MediaMetadataRetriever:

I'm currently using it, by calling:

mediaMetadataRetriever.getFrameAtTime(timeUs,MediaMetadataRetriever.OPTION_CLOSEST);

I get the frame's bitmap.

Note that it's only supported since: API Level 10.

Answer 2

You can use the code below:

String SrcPath="/sdcard/Pictures/Video Task/";

MediaMetadataRetriever mediaMetadataRetriever = new MediaMetadataRetriever();

mediaMetadataRetriever.setDataSource(SrcPath);
String METADATA_KEY_DURATION = mediaMetadataRetriever
        .extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION);

Bitmap bmpOriginal = mediaMetadataRetriever.getFrameAtTime(0);
int bmpVideoHeight = bmpOriginal.getHeight();
int bmpVideoWidth = bmpOriginal.getWidth();

Log.d("LOGTAG", "bmpVideoWidth:'" + bmpVideoWidth + "'  bmpVideoHeight:'" + bmpVideoHeight + "'");

byte [] lastSavedByteArray = new byte[0];

float factor = 0.20f;
int scaleWidth = (int) ( (float) bmpVideoWidth * factor );
int scaleHeight = (int) ( (float) bmpVideoHeight * factor );
int max = (int) Long.parseLong(METADATA_KEY_DURATION);
for ( int index = 0 ; index < max ; index++ )
{
    bmpOriginal = mediaMetadataRetriever.getFrameAtTime(index * 1000, MediaMetadataRetriever.OPTION_CLOSEST);
    bmpVideoHeight = bmpOriginal == null ? -1 : bmpOriginal.getHeight();
    bmpVideoWidth = bmpOriginal == null ? -1 : bmpOriginal.getWidth();
    int byteCount = bmpOriginal.getWidth() * bmpOriginal.getHeight() * 4;
    ByteBuffer tmpByteBuffer = ByteBuffer.allocate(byteCount);
    bmpOriginal.copyPixelsToBuffer(tmpByteBuffer);
    byte [] tmpByteArray = tmpByteBuffer.array();

    if ( !Arrays.equals(tmpByteArray, lastSavedByteArray))
    {
        int quality = 100;
        String mediaStorageDir="/sdcard/Pictures/Video Task/";
        File outputFile = new File(mediaStorageDir , "IMG_" + ( index + 1 )
                + "_" + max + "_quality_" + quality + "_w" + scaleWidth + "_h" + scaleHeight + ".png");
        Log.e("Output Files::>>",""+outputFile);
        OutputStream outputStream = null;
        try {
            outputStream = new FileOutputStream(outputFile);
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        }

        Bitmap bmpScaledSize = Bitmap.createScaledBitmap(bmpOriginal, scaleWidth, scaleHeight, false);

        bmpScaledSize.compress(Bitmap.CompressFormat.PNG, quality, outputStream);

        try {
            outputStream.close();
        } catch (IOException e) {
            e.printStackTrace();
        }

        lastSavedByteArray = tmpByteArray;
    }
}
capturedImageView.setImageBitmap(bmpOriginal);
mediaMetadataRetriever.release();

Answer 3

Assuming you are using the MediaPlayer, I believe it is not possible to extract video (or audio) frames. The mediaplayer runs it's own process [mediaplayerservice]. And, only this process has access to these frames. It is considered a security violation if the raw frames are accessible by other processes. Say for example you are playing some content that is copyright protected, then providing access to these frames should not be allowed.