Gstreamer: Pausing/resuming video in RTP streams

Question

I'm constructing a gstreamer pipeline that receives two RTP streams from an networked source:

1. ILBC Audio stream + corresponding RTCP stream
2. H263 Video stream + corresponding RTCP stream

Everything is put into one gstreamer pipeline so it will use the RTCP from both streams to synchronize audio/video. So far I've come up with this (using gst-launch for prototyping):

gst-launch -vvv  gstrtpbin name=rtpbin
  udpsrc caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H263-2000" port=40000 ! rtpbin.recv_rtp_sink_0
  rtpbin. ! rtph263pdepay ! ffdec_h263 ! xvimagesink
  udpsrc port=40001 ! rtpbin.recv_rtcp_sink_0
  rtpbin.send_rtcp_src_0 ! udpsink port=40002 sync=false async=false

  udpsrc caps="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)PCMU,encoding-params=(string)1,octet-align=(string)1" port=60000 rtpbin.recv_rtp_sink_1
  rtpbin. ! rtppcmudepay ! autoaudiosink
  udpsrc port=60001 ! rtpbin.recv_rtcp_sink_1 
  rtpbin.send_rtcp_src_1 ! udpsink port=60002 sync=false async=false

This pipeline works well if the networked source starts out with sending both video and audio. If the videostream is paused later on, gstreamer will still playback audio and even will start playing back the video when the networked source resumes the video stream.

My problem is however that if the networked source starts out with only an audio stream (video might be added later on), the pipeline seems to pause/freeze until the video stream starts as well.

Since video is optional (and can be added/removed at will by the user) in my application, is there any way I can hook up for instance an 'videotestsrc' that will provide some kind of fallback video data to keep the pipeline running when there is no networked video data?

I've tried experimenting with 'videotestsrc' and a thing called 'videomixer' but I think that mixer still requires both streams to be alive. Any feedback is greatly appreciated!

Answer 1

I present a simple function for pause resume by changing bins. In the following example I provide the logic to change destination bin on the fly dynamically. This shall not completely stop the pipeline which is what you seek I believe. A similar logic could be used for src bins. Here you may remove your network source bin and related decoder/demux bins and add videotestsrc bins.

private static void dynamic_bin_replacement(Pipeline pipe, Element src_bin, Element dst_bin_new, Element dst_bin_old) {
pipe.pause();
src_bin.unlink(dst_bin_old);                     
pipe.remove(dst_bin_old);
pipe.add(dst_bin_new);
dst_bin_new.syncStateWithParent();
src_bin.link(dst_bin_new);
pipe.ready();                    
pipe.play();
}

The other logic you may want to try is "PADLOCKING". Please take a look at the following posts

http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-block.txt

and

http://web.archiveorange.com/archive/v/8yxpz7FmOlGqxVYtkPb4

and

Adding and removing audio sources to/from GStreamer pipeline on-the-go

UPDATE

1. Try output-selector and input-selector bins as they seem to be better alternative. I found them most reliable and have had immense luck with them. I use fakesink or fakesrc respectively as the other end of the selector.

2. valve bin is another alternative that I found doesn't even need fakesink or fakesrc bins. It is also extremely reliable.

Also the correct state transition order for media file source

NULL -> READY -> PAUSED -> PLAYING (Upwards)

PLAYING -> PAUSED -> READY -> NULL (Downwards)

My order in the above example should be corrected where ready() should come before pause(). Also I would tend to think un-linking should be performed after null() state and not after pause(). I haven't tried these changes but theoretically they should work.

See the following link for detailed info

http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-states.txt?h=BRANCH-RELEASE-0_10_19