Tuesday, November 11, 2014

Live RTSP streaming using the Video Processing Project

In this post, we take a look at how we can stream live video and audio from a web cam using the video processing project.

We will use the following VPP filters:

  • CSIR VPP Scale Filter
  • CSIR VPP H264 Encoder
  • CSIR VPP RTSP Sink Filter
  • CSIR VPP RTSP Source Filter
Additionally you will need either an AMR or AAC encoders/decoders. Igor Janos from Monogram made some available on his blog a couple of years ago.

You will also need GraphEdit, GraphStudio or GraphStudioNext (recommended) to construct the graphs.

The VPP RTSP filters use the Live555 Streaming Media libraries which implement the RTSP/RTP/RTCP stacks. 

To stream audio and video from a webcam or live capture source setup a graph as shown in Figure 1.
Figure 1: RTSP Sink Graph
It is important to limit the source resolution (either via IAMStreamConfig or via the VPP Scale Filter ) and framerate (either via IAMStreamConfig or via the VPP Frame Skipping Filter) as this affects both the bitrate and the time required to encode the image. The current VPP Encoder can comfortably encode CIF video at 15 frames a second when in Mode 1. Mode 1 means that the encoder limits the bits per frame to the specified Frame Bit Limit parameter. In this example we have selected 5000 bits per frame. At e.g. 10 frames a second, this equals 50kb/s. Note that the filters must be configured before the graph is played.

Once the graph is played, start another instance of GraphStudioNext.
Figure 2: RTSP Source Graph
Insert the CSIR VPP RTSP Source Filter into the graph which will bring up a dialog (see Figure 2) allowing you to input the RTSP URI which will be of the format rtsp://<<ip>[:<<rtsp_port>>]/live. The VPP RTSP Sink Filter creates a session named "live". Finally render the output pins of the RTSP Source Filter which results in a graph similar to the one in Figure 3. The decoders will vary depending on what is installed on the machine.

Figure 3: Rendered RTSP Source Graph 
 Or alternatively use VLC. "Ctrl + N" brings up a dialog allowing you to enter the RTSP URI as shown in Figure 4.
Figure 4: RTSP with VLC
The Caching parameter should be set sufficiently big (at least 500ms, or even better 1000ms).
When this parameter is too small, VLC struggles to synchronise the audio and video resulting in many frames being discarded. This can be seen on the logging window if the verbosity is high enough.


  1. Hi
    Nice Article.
    I have one confusion What does this line mean "The VPP RTSP filters use the Live555 Streaming Media libraries which implement the RTSP/RTP/RTCP stacks. "

    After completing filter I run the Graph filter. Its running properly. but from VLC I am unable to get RTSP stream.


    1. Basically, the filter wraps the live555 library which implements RTSP,RTP and RTCP. Have you tried turning on verbose logging in VLC to see where it fails (e.g. on connect, or RTSP interchange, etc)?