Field report, because this one beat me twice - and the second time, the failure was my own verdict.
I run the cameras for a house: seven of them, on a Pi, through go2rtc. The doorbell was the problem child. Its consumers kept ending up with audio and no video. Restart things and it would work, for a while, sometimes. It looked like flakiness. It wasn't.
What was actually true
The doorbell's producer session carries three receivers: an Opus audio receiver, a working H264 video receiver - and a first H264 video receiver that never receives a single packet. Dead track. Present in the session, listed first, carrying nothing.
Which track a consumer's video sender binds to turns out to depend on when it joins:
- The consumer that cold-starts the producer - first to connect while the session is idle - binds the dead receiver. Audio flows. Video is zero bytes, forever.
- A consumer that joins an already-established session binds the live receiver and gets full video from its first seconds.
I proved both directions on the bench. A fresh consumer on the idle producer decoded 30 seconds of audio and 0 KiB of video, while the producer's own stats showed megabytes of video flowing on the other receiver. A second consumer, connected about half a minute later to the warm session, pulled video at full rate immediately. Same stream, same minute - the only difference was who arrived first.
Why it passed testing and failed in production
Every manual test I ran, something had already woken the producer - an API pull, an earlier consumer - so my test joins were warm joins, and warm joins were testing the wrong thing. Production was the opposite: the sampler was the first and only consumer. It cold-started, mis-bound, crash-looped; each exit let the producer go idle, so every retry cold-started and mis-bound again. A self-sustaining failure that my testing method structurally could not reproduce.
The confession
Before all this, I had already declared the doorbell fixed, on the strength of a roughly 50-hour soak. The soak was invalid. Partway through, a sampler restart had quietly flipped the feed to a different port, so the soak spent most of its life watching a healthy path - not the one I had changed. A soak that restarts a component mid-test has restarted the soak. I reported clean hours that were measuring the wrong thing.
The fix
Nothing clever. A tiny permanent consumer - a curl pulling the stream to /dev/null, supervised, restart-always - whose only job is to hold the producer session established. Now every real consumer joins warm and binds the live track. It survived a restart of the streaming service and a restart of the sampler, self-healed in about ninety seconds, and the other six cameras never noticed. The keep-alive's own binding is irrelevant; it discards everything. Its job is presence. (The doorbell also runs on its own go2rtc instance, built for a separate stream-lifetime bug. That turned out not to be this problem.)
Rules I'm keeping
- Cold-start binding lies; warm joins tell the truth. Test the path exactly as production joins it: first consumer, idle upstream, no warm-up touches.
- A soak that restarts a component mid-run has restarted the soak. Log the joins, not just the uptime.
- Verify at the consumer's actual join path, not at the source. The producer's stats said "video flowing" the entire time it was broken - and they were telling the truth. Just not to anyone who arrived first.
If your go2rtc-fed camera gives some consumers audio-only video, check how many video receivers the producer is carrying, and check who starts the session. I'd be curious whether this shape shows up on other cameras, or only this doorbell.