FFmpeg can't decode H264 stream/frame data

Question

Recently I had chance to work with two devices that are streaming the H264 through RTSP. And I've ran into some problem trying to decompress this stream using FFmpeg library.

Every time the "avcodec_decode_video2" is called - FFmpeg just says something like:

[h264 @ 00339220] no frame!

My raw H264 stream I frame data starts like this: "65 88 84 21 3F F8 F8 0D..." (as far as I understand this 0x65 indicates that it's a IDR frame?)

Other frames for one device starts like: "41 9A 22 07 F3 4E 48 CC...."

and for other device - like this: "61 9A 25 C1 1C 45 62 39...."

• Am I missing some frame data here?
• Does FFmpeg needs to have some extra parameters set up?

I was expecting at least "00 00 00 01" bytes at the start for the frame data... but this is what I've got..

Answer 1

Ok, managed to make things working.

• I needed to include the sequence (SPS) and picture parameter sets (PPS) for my frame data before sending frame to the FFmpeg.
• I needed to add 4 extra bytes "00 00 00 01" after SPS and PPS data.

Here is a little picture showing what I mean:

Bytes "65 88..." is where my original frame data begins.

This SPS and PPS information was not included in RTP packet. I'm using Live555 library for RTSP streaming, so I've used subsessions "fmtp_spropparametersets" function to get what I need. This information was Base64 encoded. (Sample: Something like this "Z0KAKNoC0EkQ,aM48gA==") Note that there are two "parameters" SPS and PPS seperated by "," and those parameters doesn't have a "00 00 00 01" included, so you need to add them.

Some code sample (I'm using Qt library here):

QByteArray        ba          = pSubSession->fmtp_spropparametersets();
QList<QByteArray> recordsList = ba.split(',');

for (int i = 0; i < recordsList.size(); i++)
{
   mExtraData.append(char(0x00));
   mExtraData.append(char(0x00));
   mExtraData.append(char(0x00));
   mExtraData.append(char(0x01));

   mExtraData += QByteArray::fromBase64(recordsList.at(i));
}

Now for every frame I do something like this:

QByteArray ba = QByteArray(4, 0); // Prepare the "00 00 00 01"
           ba[3] = 0x01;

mpTrackVideo->buffer.insert(0, mExtraData);
mpTrackVideo->buffer.insert(mExtraData.size(), ba);

Year ago I thought I had H264 stream support integrated in my project till I've had chance to test it with some other devices... So you need to keep in mind that some devices might send SPS and PPS data for every I frame... and some might not!