Why does camera frame rate fluctuate when fusing another sensor?

Question

I am a new baby in Android programming so that I am facing a problem related to videos when doing my research project. In the project, we would like to build an application which uses both videos recorded by a built-in front camera and acceleration sensed by a built-in accelerometer.

My problem is that the frame rate of these videos usually fluctuates from 15 fps to 30 fps even though I have set the camera fixed at 30 fps in the source code. I tried to close the data stream of the accelerometer to test the consistency of the frame rate. However it has been still fluctuated. I tested this problem in both Note 3, Galaxy S4, and S5.

The following code is the function I used to set up the front camera in my devices

private boolean prepareVideoRecorder(String filename, boolean isFrontCam){
    mCamera = Camera.open;
    mMediaRecorder = new MediaRecorder();

    if(mCamera == null) return false;

    Camera.Size size = getBestPreviewSize(1280, 720, mCamera.getParameters());

    if(IS_TABLET){
        mCamera.setDisplayOrientation(0);
    }else{
        mCamera.setDisplayOrientation(90);
    }

    // Step 1: Unlock and set camera to MediaRecorder
    mCamera.unlock();
    mMediaRecorder.setCamera(mCamera);

    // Step 2: Set sources
    mMediaRecorder.setVideoSource(MediaRecorder.VideoSource.CAMERA);

    // Step 3: Set a CamcorderProfile (requires API Level 8 or higher)
    CamcorderProfile profile = CamcorderProfile.get(CamcorderProfile.QUALITY_HIGH);

    mMediaRecorder.setOutputFormat(profile.fileFormat);
    mMediaRecorder.setVideoEncoder(profile.videoCodec);
    mMediaRecorder.setVideoEncodingBitRate(profile.videoBitRate);
    mMediaRecorder.setVideoFrameRate(profile.videoFrameRate);
    mMediaRecorder.setVideoSize(profile.videoFrameWidth, profile.videoFrameHeight);

    // Step 4: Set output file
    mMediaRecorder.setOutputFile(filename);

    // Step 5: Set the preview output
    mMediaRecorder.setPreviewDisplay(mPreview.getHolder().getSurface());

    if(isFrontCam){

        if(IS_TABLET){
            mMediaRecorder.setOrientationHint(180);
        }else{
            mMediaRecorder.setOrientationHint(270);
        }
    }
    else{
        if(IS_TABLET){
            mMediaRecorder.setOrientationHint(0);
        }else{
            mMediaRecorder.setOrientationHint(90);
        }
    }

    mMediaRecorder.setVideoSize(size.width, size.height);
    mMediaRecorder.setVideoFrameRate(30);

    // Step 6: Prepare configured MediaRecorder
    try {
        mMediaRecorder.prepare();
    } catch (IllegalStateException e) {
        Log.d(TAG, "IllegalStateException preparing MediaRecorder: " + e.getMessage());
        Toast.makeText(getBaseContext(), "IllegalStateException preparing MediaRecorder", Toast.LENGTH_SHORT).show();
        releaseMediaRecorder();
        return false;
    } catch (IOException e) {
        Log.d(TAG, "IOException preparing MediaRecorder: " + e.getMessage());
        Toast.makeText(getBaseContext(), "IOException preparing MediaRecorder", Toast.LENGTH_SHORT).show();
        releaseMediaRecorder();
        return false;
    }
    return true;
}

Could anyone explain to me why the frame rate of the video fluctuates in such a way when programming an Android application related to the camera? Is there any solution for this problem that you can suggest me? I have read this topic in other websites as well but it is not clear and hard for me to understand.

Thank you in advance.

Answer 1

I have just fixed this problem by using setAutoExposureLock(true) and setAutoWhiteBalanceLock(true) for the Camera.Parameters. After some searching efforts, I found that the reason why the frame rate fluctuates was related to the light condition ... so as the exposure control of the camera.

I attached the code that I use below.

Camera.Parameters params = mCamera.getParameters();
params.setAutoExposureLock(true);
params.setAutoWhiteBalanceLock(true);
mCamera.setParameters(params);