Noise occurs while encoding audio from Microphone and latency in video streaming using IMFSourceReader

问题

I'm developing an application to capture.MP4 file (video(USB Camera)and audio(Any Microphone) together )during video streaming using SourceReader.

My code is returning success for all API's and able to save the video. I'm facing below 3 issues.

1. After 10secs of recording , fps goes into 0 and resume to 1fps(Note: Saving video for 1080p 60fps). FPS is not resuming to 60fps.
2. While playing the recorded file, noise audio only playing for the whole file. Recorded audio file is not playing.
3. To stop the record, I'm using sinkwriter finalize method. My application is not responding to call this method and this method taking the time to release it(Time is taken to release ~3-5mins).

Please find my code in the following dropbox link: https://www.dropbox.com/s/eddwdc1z9wsoh04/VideoRecord.txt?dl=0

Without any problem, I'm able to capture audio and video using capture engine technique but this technique supports from Windows 8. My application has to support from Windows 7.

Please let me know whether am missing anything to configure audio and video? Help me to solve this issue.

Thanks in advance

回答1:

I see few mistakes. First, you call ReadSample with MF_SOURCE_READER_FIRST_VIDEO_STREAM. Which will read only the video samples. You should call ReadSample (first time and inside OnReadSample) with MF_SOURCE_READER_ANY_STREAM instead, in order to make it read both video and audio samples.

Second, the following two lines are wrong:

hr = m_pWriter->WriteSample(0, pSample);
hr = m_pWriter->WriteSample(1, pSample);

You should call WriteSample with corresponding stream index according to the dwStreamIndex value in OnReadSample. For example in your case:

if(dwStreamIndex == 0)
    hr = m_pWriter->WriteSample(m_dwVideoStreamIndex, pSample);
else if(dwStreamIndex == 1)
    hr = m_pWriter->WriteSample(m_dwAudioStreamIndex, pSample);

来源：https://stackoverflow.com/questions/43238552/noise-occurs-while-encoding-audio-from-microphone-and-latency-in-video-streaming