Skip to main content

Case Study: MediaMTX WebRTC Stream Stalls & Keyframe Tuning

📌 What is this for? (Quick Summary for New Interns)​

In simple words: Have you ever opened the live camera feed and only seen a black screen with a spinning circle? This case study explains why that happened (web browsers need a full picture snapshot, called an IDR keyframe, every second) and how setting FFmpeg flags -g 15 and -bf 0 fixed the problem forever.

1. Problem Statement​

  • Symptoms:
    • The forwarder script on the Raspberry Pi reported successful RTSP transmission to rtsp://cam.tutr.tech:8554/cam1.
    • MediaMTX logged active RTSP publishers and acknowledged WebRTC WHEP connection requests.
    • However, in Chrome and Edge desktop browsers, the HTML5 <video> element remained completely black with a permanent spinning loading indicator.
    • No network errors were recorded in the browser DevTools; the WebRTC peer connection reached connected state, but frames were never rendered.

2. Root Cause Analysis​

  1. RTSP Keyframe Starvation:

    • Standard IP surveillance cameras often use a Group of Pictures (GOP) size of 150 to 300 frames. At 15 FPS, that means an Intra-coded (IDR) keyframe is only transmitted every 10 to 20 seconds.
    • Traditional RTSP players like VLC can buffer seconds of predictive (P-frames) before catching up to the next keyframe.
    • WebRTC, by contrast, operates under ultra-low-latency real-time rules. A browser WebRTC video decoder refuses to initialize playback until it receives an IDR keyframe containing SPS (Sequence Parameter Set) and PPS (Picture Parameter Set) NAL units.
    • If an operator connected to the stream 1 second after an IDR frame was emitted, the browser waited up to 19 seconds before decoding its first frame, often causing browser timeout logic to stall completely.

  2. Bidirectional Predictive Frames (B-Frames):

    • The initial FFmpeg encoder configuration used standard predictive coding with default B-frames.
    • B-frames require the decoder to inspect both preceding and future frames before displaying the current frame. This caused out-of-order frame arrival in the browser's WebRTC jitter buffer, leading to dropped RTP packets and decoder freezes.

3. The Definitive Solution​

In stream_service/app.py, the FFmpeg transcoding pipeline was re-architected with three mandatory flags:

cmd = [
    'ffmpeg', '-re',
    # Input source (physical RTSP or synthetic fallback)
    ...,
    '-c:v', 'libx264',
    '-preset', 'ultrafast',
    '-tune', 'zerolatency',
    '-profile:v', 'baseline',      # [FIX 1] Universal decoding baseline
    '-pix_fmt', 'yuv420p',
    '-g', '15',                   # [FIX 2] Mandatory 1-second keyframe interval
    '-bf', '0',                   # [FIX 3] Zero B-frames eliminates jitter buffer stalls
    '-b:v', '1200k',
    '-maxrate', '1500k',
    '-bufsize', '2400k',
    '-f', 'rtsp',
    '-rtsp_transport', 'tcp',
    target_rtsp
]

Results:​

  • Time-to-First-Frame (TTFF): Dropped from 10–20 seconds to under 800ms.
  • Jitter Buffer Latency: Sustained at < 250ms glass-to-glass.
  • Playback Reliability: 100% immediate playback across desktop Chrome, Edge, Safari, and mobile WebRTC players.