r/computervision • • 20h ago

Discussion 24 AI Camera Streams on 1* RTX 5090: Zero Dropped Frames, but Still Missing Deadlines

4 Upvotes

Imagine you're building an AI monitoring system for a warehouse with dozens of cameras. You need to detect people, track their movements, and process incoming video without falling behind.

How many cameras can you assign to a single GPU before you need another?

If your object detector processes 300 frames per second and each camera needs 10 FPS, supporting 30 cameras sounds reasonable. But that calculation doesn't account for video decoding, object tracking, or delays between receiving a frame and finishing its processing.

I tested this on an RTX 5090 using YOLO11n and ByteTrack. At 24 simulated cameras, the pipeline sustained 240 FPS without skipping any scheduled frames during repeated 60-second tests. However, some batches still finished after their deadlines.

Offline demonstration of 24 simulated streams using time-offset segments from one source video, with YOLO detections and ByteTrack IDs.

Building the experiment

Rather than loading a separate detection model for each camera, I grouped frames into batches and processed them through a shared YOLO11n model. Each stream maintained its own ByteTrack instance.

ComponentConfiguration
GPURTX 5090, 32 GB
CPU / RAM allocation25 core equivalents / 90 GiB
DetectionYOLO11n, FP16
TrackingIndependent ByteTrack instances
Video processingOpenCV, CPU decoding
Target10 FPS per camera

I created the simulated streams using multiple readers of the same local 768 × 576 video. There were no physical cameras or RTSP connections involved.

The benchmark included decoding, detection, and tracking, but excluded network transport, rendering, and output video encoding.

From model inference to video processing

I started by benchmarking YOLO11n with preloaded frames. At a batch size of eight, it reached approximately 404 FPS.

Based on that number, processing 32 streams at 10 FPS each seemed feasible. Once I added video readers and ByteTrack, however, the available processing margin became much smaller.

The initial 24-camera tracking test skipped 11.5% of scheduled frames, even though processing times at lower camera counts suggested the pipeline should have been able to keep up.

I suspected that initialization overhead was affecting the results. After moving detector and tracker warm-up outside the timed window, the 24-camera test processed every scheduled frame.

I kept this warm-up procedure for the remaining steady-state benchmarks.

Finding the camera limit

I increased the camera count and extended the tests to 60 seconds per configuration.

CamerasAchieved FPSSkipped framesP95 latency
242400%92.98 ms
262600%108.74 ms
28274.871.83%191.09 ms
Camera capacity and latency graph

At 24 cameras, the pipeline processed all 14,400 scheduled frames with a P95 completion latency of 92.98 ms.

Adding just two more cameras pushed P95 latency above 100 ms, even though no frames were skipped. At 28 cameras, scheduler skips appeared and P95 latency nearly doubled relative to the frame interval.

Since each stream was targeting 10 FPS, a new synchronized batch was scheduled every 100 ms. A batch could finish late without necessarily causing the scheduler to skip subsequent frames.

I added a deadline-miss counter and repeated the 24-camera test three times. All 43,200 scheduled frames were processed, but 77 of the 1,800 batches finished after their 100 ms deadline (4.28%).

An individual 60-second RTX 5090 benchmark showing 14,400 processed frames, zero skipped frames, and a 2.33% late-batch rate. Across three separate trials, the late-batch rate was 4.28%.

An individual 60-second benchmark with 14,400 processed frames, zero skipped frames, and a 2.33% late-batch rate. The separate three-trial deadline test recorded a combined rate of 4.28%.

An application doing periodic video analytics might tolerate these occasional delays. For a monitoring system that needs consistently timely detections, I'd want to account for them when deciding how many streams to assign to each GPU.

What happened when I parallelized decoding

Video decoding was taking up a noticeable part of the processing interval, so I tried distributing frame reading across multiple CPU threads.

In an initial 28-camera test, four decoder workers reduced mean decoding time from 24.40 ms to 14.84 ms.

To check whether this improved the entire pipeline, I ran nine 60-second trials at 24 cameras, with three runs each for sequential decoding, two workers, and four workers.

MetricSequential2 workers4 workers
Aggregate FPS240210.53240
Skipped frames0%12.28%0%
Late batches4.00%95.28%18.17%
Mean P95 latency99.02 ms213.22 ms111.89 ms
Decoder comparison chart

With four workers, mean decoding time decreased from 22.77 ms to 13.05 ms. However, detection time increased from 49.91 ms to 57.88 ms, and tracking time increased from 16.39 ms to 20.34 ms.

The time saved during decoding was lost in the subsequent stages. The four-worker configuration maintained 240 FPS but completed considerably more batches late.

The two-worker configuration performed poorly across all three trials, skipping an average of 12.28% of scheduled frames.

I haven't isolated the cause of these regressions. Thread contention, CPU scheduling, and interactions between OpenCV and the inference pipeline are possible contributors. I'd need more detailed profiling to determine what's happening.

For this implementation, I'd stick with sequential decoding.

Would I deploy 24 cameras on one GPU?

Under the conditions I tested, 24 simulated streams at 10 FPS were manageable, although the deadline misses suggest limited processing headroom.

For a real monitoring system, I wouldn't assign 24 live cameras to a GPU based on these results alone. Independent RTSP feeds, different codecs, varying scene complexity, and output encoding could all change the workload. I'd also need longer-duration tests and measurements of detection and tracking quality.

The experiment doesn't establish whether an RTX 5090 is necessary, either. I only tested one GPU. Running the same pipeline on an RTX 4080 or a less expensive GPU would help determine whether the additional hardware investment is justified.

I'd also like to try asynchronous frame ingestion and per-camera buffering. The current implementation processes synchronized batches sequentially, which may be contributing to the latency behavior.

For someone planning a multi-camera system, I'd benchmark the complete application before deciding how much GPU capacity to rent or purchase. The 240 FPS figure alone would have missed a problem that became visible as soon as I started counting late batches.

For those running multi-camera CV systems, how do you determine the number of streams to assign to each GPU? Do you work with a latency target, a deadline-miss threshold, or leave a fixed amount of processing headroom?


r/computervision • • 12h ago

Help: Project Replacing expensive polarized dome lighting with real-time Fresnel specular separation in PyTorch — is metallic glare still a major bottleneck on your lines?

0 Upvotes

I'm a computational optics researcher building lightweight wave-optics/Fresnel inverse solvers (ShivInterferentia) to strip specular glare from shiny metal parts and PCBs using a single standard camera at 35+ FPS.

Before I optimize this into a deployable C++/TensorRT SDK, I want to sanity-check with folks running real factory or robotics vision pipelines:

  1. How much time/budget do you currently lose fighting specular glare or bad lighting on shiny surfaces?
  2. What edge hardware (Jetson, x86 CPU, industrial smart camera) would this need to run on to actually be useful in production?

If anyone has a tricky, glare-ruined test dataset they want me to run through our solver for free as a benchmark, drop a comment or DM—happy to share the restored output and benchmark metrics.


r/computervision • • 20h ago

Showcase tamserve update: track anything you name on live cameras, at 95% of SAM 3's accuracy and 2.5x less GPU time (open source)

3 Upvotes

Last week I shared tamserve, an open-source serving engine for SAM 2-class video trackers. One copy of the model tracks every object on every camera, so a GPU handles far more cameras and memory stays flat. Since then more than 3,000 people have visited the repo and over 260 have starred it. Thank you.

The project has grown with more support this past week:

  • Track by name. Give each camera phrases such as "dog" or "the red mug". SAM 3 looks every few frames to find matches, and a small tracker follows them in between. On Meta's own SAM 3 benchmark it scores 56.3 HOTA against 59.4 for SAM 3's own tracker, in 41 ms of GPU time a frame against 101 on an RTX 4090.
  • People. A tracker tuned for people scores 89.6 HOTA against 87.8 on 116 home videos, in 8.9 ms a frame against 126.
  • Several GPUs and machines. A router spreads cameras over GPUs and keeps going when a camera or a machine drops. Two RTX 4090s tracked 476 frames a second from 32 cameras.
  • A look rate per camera. Suggested in the comments last time: a quiet camera can be looked at less often, which frees budget for a busy one.
  • Apple silicon. It runs on M-series Macs, with part of the model on the Neural Engine while the GPU does the rest. That is about twice as fast as the models' own code on the same Mac, and it keeps up with a live 30 fps camera.

What it is not good at:

  • In a crowd it only matches SAM 3's own tracker if it looks at every frame, which costs more.
  • Tracking by name tops out at about 30 frames a second in total on one RTX 4090, however many cameras share it.
  • A Mac is about a fifth of an RTX 4090, and I have only measured an M4 Max.
  • SAM 3's weights are gated by Meta, so you accept their licence to track by name.

Code, and every measurement with the command that reproduces it: https://github.com/aryamandarda/tamserve

If you run the Mac benchmark on a different chip, I would like to see the numbers.


r/computervision • • 23h ago

Commercial I indexed 90M movie & TV frames into a 768-d vector database so you can identify any scene via a single REST call

126 Upvotes

Hey everyone,

Over the past months, I’ve been building a frame-level cinema & TV scene recognition engine (**Visual DNA API**). The goal was to make an API that works like Shazam, but for single movie/TV screenshots—including cropped, mirrored, or subtitled frames.

**What it does:**

- `POST /v1/identify`: Send any frame (PNG/JPG/WEBP) → get back the movie/series title, release year, episode code, `tmdb_id`, and confidence score in JSON.

- `POST /v1/search`: Describe a scene in plain English (e.g., *"rainy neon cyberpunk street at night"*) → retrieve matching titles.

**Privacy & stack highlights:**

- 90M+ 768-d CLIP vectors indexed in Qdrant on EU servers.

- Volatile RAM processing: uploaded frames are vectorized in memory and immediately discarded (zero disk storage, zero biometric recognition).

**Try it without signing up:**

I’ll drop the live scanner (5 free daily scans + raw JSON output) and the open-source quickstart repo (Python, Node.js, Telegram & Discord bots) in the comments below.

If you test it with a tricky screenshot and it misses, drop the movie name here—edge cases help me tune the indexing pipeline!


r/computervision • • 17h ago

Showcase Full Visual SLAM & 3D Hand Tracking on a DIY AR mask: MediaPipe plus triangulation in C++, driving a full desktop UI

28 Upvotes

This is my solo-built system, VIOX. VIOX is the computer you wear. It's an AR personal computer that renders its own entire computer setup in an AR mask. VIOX doesn't use an external phone or laptop connection, because it isn't made to extend another computer's display, it IS the computer, with a hip-mounted LattePanda Sigma computing module.


r/computervision • • 35m ago

Help: Project How can I improve vehicle detection FPS on an old Intel CPU without sacrificing detection accuracy?

2 Upvotes

How can I improve vehicle detection FPS on an old Intel CPU without sacrificing detection accuracy?

Hi everyone! I’m working on an undergraduate thesis involving vehicle detection using two RTSP cameras connected to a single laptop.

Hardware and setup:

  • Intel i5-7200U (2 cores, 4 threads), 8 GB RAM, Windows 10
  • 2× Tapo C200C RTSP cameras
  • OpenVINO CPU inference, ByteTrack, Flask

We need to detect eight vehicle classes: car, motorcycle, truck, bus, bicycle, EVs like golfcar which is commonly called ebike in philippines, philippine jeepney, and philippine tricycle.

Our current approach
We’re using two object detection models:

  • YOLO26n (COCO pretrained): Detects standard vehicle classes.
  • Custom YOLOv8n: Detects Philippine-specific vehicles (e-bikes, jeepneys, and tricycles).

We use two models because COCO-pretrained YOLO doesn’t include dedicated classes for these Philippine vehicles.

Both models are quantized to INT8. However, since both cameras share one CPU and run both detectors, we’re only achieving around 1.5–2.4 detection FPS per camera during live RTSP testing.

The main problem
We also trained a single unified YOLO26n model covering standard and Philippine vehicles.
It ran significantly faster, but missed more vehicles during our tests, making it less reliable for our use case.
So we’re stuck with a trade-off:

  • One model: Faster inference, but more missed detections.
  • Two models: Covers the required vehicle classes, but inference is too slow on our CPU.

Hardware upgrades aren’t an option because of our limited thesis budget.

My questions:

What would be the best way to reliably detect all eight vehicle classes using only one lightweight model on this hardware?

Should we improve our unified YOLO training
approach, try a different lightweight architecture, or optimize OpenVINO inference differently?

We’re aiming for around 10 detection FPS per camera, although we understand that might be unrealistic for an i5-7200U.

I’d really appreciate recommendations from anyone who has worked with CPU-only vehicle detection or similar low-resource setups.

Help is so much appreciated Thank you!


r/computervision • • 7h ago

Help: Project I made a tutorial covering some really cool Computer Vision architectures

Thumbnail
youtu.be
4 Upvotes

Video covers:

  1. Photo library search, text removal, label generation
  2. Document layout detection models and RT-DETR (D-Fine) architectures
  3. Tennis ball, player, courtmap tracking system with CenterMap-style heatmap prediction
  4. And my favourite: World Models/Neural Computers with VQ-VAE + Spatio-temporal Transformers

Project repo: https://github.com/avbiswas/computer_vision_recipes

Video: https://youtu.be/mPmdI-0ISx4