Segmentation fault while avcodec_encode_video2

Question

I have some problems while trying to encode a AVFrame to a packet.

Before reading the whole code, the input stuff is working, I tested it. The output stuff is from a

Answer 1

Finally i solved my problem.

The problem is (apart from the documentation of libav) avpacket is not a (real) copy of the picture in the packet. it just points to the data of the packet. You have to make a copy, or better you have to let it libav do.

So first i created a new avframe for the output and a buffer on which the output avframe is pointing to.

AVFrame *outpic = avcodec_alloc_frame();
nbytes = avpicture_get_size(codecCtxOut->pix_fmt, codecCtxOut->width, codecCtxOut->height);
uint8_t* outbuffer = (uint8_t*)av_malloc(nbytes);

This buffer is used for the conversion from input to output. Then in the loop i have to fillup the outpic (avframe) with the buffer. I have found in the code that this function is filling up the plane pointers with the buffer. see here

avpicture_fill((AVPicture*)outpic, outbuffer, AV_PIX_FMT_YUV420P, codecCtxOut->width, codecCtxOut->height);

Then i converted the inpic to outpic using sws_scale. But first you have to setup the swscontext.

SwsContext* swsCtx_ = sws_getContext(codecCtxIn->width, codecCtxIn->height,
                                     codecCtxIn->pix_fmt,
                                     codecCtxOut->width, codecCtxOut->height,
                                     codecCtxOut->pix_fmt,
                                     SWS_BICUBIC, NULL, NULL, NULL);

sws_scale(swsCtx_, inpic->data, inpic->linesize, 0, codecCtxIn->height, outpic->data, outpic->linesize);

Then you can encode the outpic into pktout (avpacket for output). But first do free the output packet, otherwise you will get an error and a leak... see here

av_free_packet(pktOut);

if(avcodec_encode_video2(streamOut->codec, pktOut, outpic, &fff) < 0) {
  std::cout << "shit frame" << std::endl;
  continue;
}
// and write it to the file
formatOut->write_packet(formatCtxOut, pktOut);

So now it works (nearly fine) for me. Still a small memory leak, but this i can spot later.