r/computervision • • 23h ago

Commercial I indexed 90M movie & TV frames into a 768-d vector database so you can identify any scene via a single REST call

126 Upvotes

Hey everyone,

Over the past months, I’ve been building a frame-level cinema & TV scene recognition engine (**Visual DNA API**). The goal was to make an API that works like Shazam, but for single movie/TV screenshots—including cropped, mirrored, or subtitled frames.

**What it does:**

- `POST /v1/identify`: Send any frame (PNG/JPG/WEBP) → get back the movie/series title, release year, episode code, `tmdb_id`, and confidence score in JSON.

- `POST /v1/search`: Describe a scene in plain English (e.g., *"rainy neon cyberpunk street at night"*) → retrieve matching titles.

**Privacy & stack highlights:**

- 90M+ 768-d CLIP vectors indexed in Qdrant on EU servers.

- Volatile RAM processing: uploaded frames are vectorized in memory and immediately discarded (zero disk storage, zero biometric recognition).

**Try it without signing up:**

I’ll drop the live scanner (5 free daily scans + raw JSON output) and the open-source quickstart repo (Python, Node.js, Telegram & Discord bots) in the comments below.

If you test it with a tricky screenshot and it misses, drop the movie name here—edge cases help me tune the indexing pipeline!


r/computervision • • 17h ago

Showcase Full Visual SLAM & 3D Hand Tracking on a DIY AR mask: MediaPipe plus triangulation in C++, driving a full desktop UI

28 Upvotes

This is my solo-built system, VIOX. VIOX is the computer you wear. It's an AR personal computer that renders its own entire computer setup in an AR mask. VIOX doesn't use an external phone or laptop connection, because it isn't made to extend another computer's display, it IS the computer, with a hip-mounted LattePanda Sigma computing module.


r/computervision • • 7h ago

Help: Project I made a tutorial covering some really cool Computer Vision architectures

Thumbnail
youtu.be
4 Upvotes

Video covers:

  1. Photo library search, text removal, label generation
  2. Document layout detection models and RT-DETR (D-Fine) architectures
  3. Tennis ball, player, courtmap tracking system with CenterMap-style heatmap prediction
  4. And my favourite: World Models/Neural Computers with VQ-VAE + Spatio-temporal Transformers

Project repo: https://github.com/avbiswas/computer_vision_recipes

Video: https://youtu.be/mPmdI-0ISx4


r/computervision • • 20h ago

Discussion 24 AI Camera Streams on 1* RTX 5090: Zero Dropped Frames, but Still Missing Deadlines

4 Upvotes

Imagine you're building an AI monitoring system for a warehouse with dozens of cameras. You need to detect people, track their movements, and process incoming video without falling behind.

How many cameras can you assign to a single GPU before you need another?

If your object detector processes 300 frames per second and each camera needs 10 FPS, supporting 30 cameras sounds reasonable. But that calculation doesn't account for video decoding, object tracking, or delays between receiving a frame and finishing its processing.

I tested this on an RTX 5090 using YOLO11n and ByteTrack. At 24 simulated cameras, the pipeline sustained 240 FPS without skipping any scheduled frames during repeated 60-second tests. However, some batches still finished after their deadlines.

Offline demonstration of 24 simulated streams using time-offset segments from one source video, with YOLO detections and ByteTrack IDs.

Building the experiment

Rather than loading a separate detection model for each camera, I grouped frames into batches and processed them through a shared YOLO11n model. Each stream maintained its own ByteTrack instance.

ComponentConfiguration
GPURTX 5090, 32 GB
CPU / RAM allocation25 core equivalents / 90 GiB
DetectionYOLO11n, FP16
TrackingIndependent ByteTrack instances
Video processingOpenCV, CPU decoding
Target10 FPS per camera

I created the simulated streams using multiple readers of the same local 768 × 576 video. There were no physical cameras or RTSP connections involved.

The benchmark included decoding, detection, and tracking, but excluded network transport, rendering, and output video encoding.

From model inference to video processing

I started by benchmarking YOLO11n with preloaded frames. At a batch size of eight, it reached approximately 404 FPS.

Based on that number, processing 32 streams at 10 FPS each seemed feasible. Once I added video readers and ByteTrack, however, the available processing margin became much smaller.

The initial 24-camera tracking test skipped 11.5% of scheduled frames, even though processing times at lower camera counts suggested the pipeline should have been able to keep up.

I suspected that initialization overhead was affecting the results. After moving detector and tracker warm-up outside the timed window, the 24-camera test processed every scheduled frame.

I kept this warm-up procedure for the remaining steady-state benchmarks.

Finding the camera limit

I increased the camera count and extended the tests to 60 seconds per configuration.

CamerasAchieved FPSSkipped framesP95 latency
242400%92.98 ms
262600%108.74 ms
28274.871.83%191.09 ms
Camera capacity and latency graph

At 24 cameras, the pipeline processed all 14,400 scheduled frames with a P95 completion latency of 92.98 ms.

Adding just two more cameras pushed P95 latency above 100 ms, even though no frames were skipped. At 28 cameras, scheduler skips appeared and P95 latency nearly doubled relative to the frame interval.

Since each stream was targeting 10 FPS, a new synchronized batch was scheduled every 100 ms. A batch could finish late without necessarily causing the scheduler to skip subsequent frames.

I added a deadline-miss counter and repeated the 24-camera test three times. All 43,200 scheduled frames were processed, but 77 of the 1,800 batches finished after their 100 ms deadline (4.28%).

An individual 60-second RTX 5090 benchmark showing 14,400 processed frames, zero skipped frames, and a 2.33% late-batch rate. Across three separate trials, the late-batch rate was 4.28%.

An individual 60-second benchmark with 14,400 processed frames, zero skipped frames, and a 2.33% late-batch rate. The separate three-trial deadline test recorded a combined rate of 4.28%.

An application doing periodic video analytics might tolerate these occasional delays. For a monitoring system that needs consistently timely detections, I'd want to account for them when deciding how many streams to assign to each GPU.

What happened when I parallelized decoding

Video decoding was taking up a noticeable part of the processing interval, so I tried distributing frame reading across multiple CPU threads.

In an initial 28-camera test, four decoder workers reduced mean decoding time from 24.40 ms to 14.84 ms.

To check whether this improved the entire pipeline, I ran nine 60-second trials at 24 cameras, with three runs each for sequential decoding, two workers, and four workers.

MetricSequential2 workers4 workers
Aggregate FPS240210.53240
Skipped frames0%12.28%0%
Late batches4.00%95.28%18.17%
Mean P95 latency99.02 ms213.22 ms111.89 ms
Decoder comparison chart

With four workers, mean decoding time decreased from 22.77 ms to 13.05 ms. However, detection time increased from 49.91 ms to 57.88 ms, and tracking time increased from 16.39 ms to 20.34 ms.

The time saved during decoding was lost in the subsequent stages. The four-worker configuration maintained 240 FPS but completed considerably more batches late.

The two-worker configuration performed poorly across all three trials, skipping an average of 12.28% of scheduled frames.

I haven't isolated the cause of these regressions. Thread contention, CPU scheduling, and interactions between OpenCV and the inference pipeline are possible contributors. I'd need more detailed profiling to determine what's happening.

For this implementation, I'd stick with sequential decoding.

Would I deploy 24 cameras on one GPU?

Under the conditions I tested, 24 simulated streams at 10 FPS were manageable, although the deadline misses suggest limited processing headroom.

For a real monitoring system, I wouldn't assign 24 live cameras to a GPU based on these results alone. Independent RTSP feeds, different codecs, varying scene complexity, and output encoding could all change the workload. I'd also need longer-duration tests and measurements of detection and tracking quality.

The experiment doesn't establish whether an RTX 5090 is necessary, either. I only tested one GPU. Running the same pipeline on an RTX 4080 or a less expensive GPU would help determine whether the additional hardware investment is justified.

I'd also like to try asynchronous frame ingestion and per-camera buffering. The current implementation processes synchronized batches sequentially, which may be contributing to the latency behavior.

For someone planning a multi-camera system, I'd benchmark the complete application before deciding how much GPU capacity to rent or purchase. The 240 FPS figure alone would have missed a problem that became visible as soon as I started counting late batches.

For those running multi-camera CV systems, how do you determine the number of streams to assign to each GPU? Do you work with a latency target, a deadline-miss threshold, or leave a fixed amount of processing headroom?


r/computervision • • 20h ago

Showcase tamserve update: track anything you name on live cameras, at 95% of SAM 3's accuracy and 2.5x less GPU time (open source)

3 Upvotes

Last week I shared tamserve, an open-source serving engine for SAM 2-class video trackers. One copy of the model tracks every object on every camera, so a GPU handles far more cameras and memory stays flat. Since then more than 3,000 people have visited the repo and over 260 have starred it. Thank you.

The project has grown with more support this past week:

  • Track by name. Give each camera phrases such as "dog" or "the red mug". SAM 3 looks every few frames to find matches, and a small tracker follows them in between. On Meta's own SAM 3 benchmark it scores 56.3 HOTA against 59.4 for SAM 3's own tracker, in 41 ms of GPU time a frame against 101 on an RTX 4090.
  • People. A tracker tuned for people scores 89.6 HOTA against 87.8 on 116 home videos, in 8.9 ms a frame against 126.
  • Several GPUs and machines. A router spreads cameras over GPUs and keeps going when a camera or a machine drops. Two RTX 4090s tracked 476 frames a second from 32 cameras.
  • A look rate per camera. Suggested in the comments last time: a quiet camera can be looked at less often, which frees budget for a busy one.
  • Apple silicon. It runs on M-series Macs, with part of the model on the Neural Engine while the GPU does the rest. That is about twice as fast as the models' own code on the same Mac, and it keeps up with a live 30 fps camera.

What it is not good at:

  • In a crowd it only matches SAM 3's own tracker if it looks at every frame, which costs more.
  • Tracking by name tops out at about 30 frames a second in total on one RTX 4090, however many cameras share it.
  • A Mac is about a fifth of an RTX 4090, and I have only measured an M4 Max.
  • SAM 3's weights are gated by Meta, so you accept their licence to track by name.

Code, and every measurement with the command that reproduces it: https://github.com/aryamandarda/tamserve

If you run the Mac benchmark on a different chip, I would like to see the numbers.


r/computervision • • 18h ago

Help: Project How do you actually investigate the synthetic-to-real gap in CV, beyond tuning by intuition? Looking for practitioners' experience

2 Upvotes

I'd like to hear from people who have worked with synthetic or cross-domain training data in practice: How do you investigate what's actually wrong with your data, instead of tweaking it on intuition until the numbers improve?

Most advice I find stops at "close the domain gap" or "use domain randomization." It rarely explains how people found out which factors mattered for their problem. I'm after the real workflow: what you checked first, which analyses told you something useful, which were a waste of time, and what surprised you.

My case, as a concrete example

I'm working on UAV detection (fixed-wing vs. quadcopter) in MWIR thermal imagery. The general question isn't specific to this, but it helps to have something concrete.

  • Model: YOLO26m
  • Target data: small annotated MWIR set, split into K folds by video (no frames from one video in both train and test)
  • Public-data baseline: trained on MMFW-UAV + Anti-UAV-300 (LWIR, 5k images per class), evaluated on the MWIR held-out folds
  • In-domain baseline: trained on the MWIR training folds, evaluated on the same held-out folds

The public-data model is much worse: [X] vs. [Y] mAP50-95 (mean ± std over folds). A large public LWIR dataset loses to a small amount of in-domain data.

My target domain doesn't exist publicly, so the plan is to generate synthetic MWIR data, either physics/3D-rendered or learning-based (image translation, diffusion). I can list the obvious suspects: waveband, sensor noise, UAV models, backgrounds. What I don't have is a systematic way to tell which of them, or which unknown factors, actually drive the gap, or to check whether a change to the generator fixed the right thing.

What I'd like to hear about, from your experience on any CV task

  1. Locating the gap. How do you work out which factors cause the drop between training data (synthetic or another domain) and real target data? Did any metrics, controlled ablations or experiment designs actually give you an answer?
  2. Seeing what the model learned. What do you use in practice to understand what a model relies on and where it behaves differently across domains? For example, feature-space comparisons, attribution maps, error breakdowns by size/contrast/background, or probing individual factors. Which of these led you to a real fix?
  3. Turning findings into better data. How do you feed those findings back into the data or generator so each iteration is driven by evidence? What did that loop look like on your project?

r/computervision • • 6h ago

Discussion i built a free 36M-param tool that lets LLMs “watch” videos locally (MCP server, runs on laptop cpu, apache-2.0)

Thumbnail reddit.com
1 Upvotes