r/computervision • • 24m ago

Showcase Building and Refining a Lumigraph-based 4D Mesh System: 10x–50x Smaller than 4DGS

5 Upvotes

I'm trying to make it work with just 2 cameras and its trainable on most gpu over 20 gbvram. demo will be soon, but currently, im out of agent quota for this week to port it properly. i hope yall can give me some pointers.


r/computervision • • 2h ago

Help: Project How can I improve vehicle detection FPS on an old Intel CPU without sacrificing detection accuracy?

3 Upvotes

How can I improve vehicle detection FPS on an old Intel CPU without sacrificing detection accuracy?

Hi everyone! I’m working on an undergraduate thesis involving vehicle detection using two RTSP cameras connected to a single laptop.

Hardware and setup:

  • Intel i5-7200U (2 cores, 4 threads), 8 GB RAM, Windows 10
  • 2× Tapo C200C RTSP cameras
  • OpenVINO CPU inference, ByteTrack, Flask

We need to detect eight vehicle classes: car, motorcycle, truck, bus, bicycle, EVs like golfcar which is commonly called ebike in philippines, philippine jeepney, and philippine tricycle.

Our current approach
We’re using two object detection models:

  • YOLO26n (COCO pretrained): Detects standard vehicle classes.
  • Custom YOLOv8n: Detects Philippine-specific vehicles (e-bikes, jeepneys, and tricycles).

We use two models because COCO-pretrained YOLO doesn’t include dedicated classes for these Philippine vehicles.

Both models are quantized to INT8. However, since both cameras share one CPU and run both detectors, we’re only achieving around 1.5–2.4 detection FPS per camera during live RTSP testing.

The main problem
We also trained a single unified YOLO26n model covering standard and Philippine vehicles.
It ran significantly faster, but missed more vehicles during our tests, making it less reliable for our use case.
So we’re stuck with a trade-off:

  • One model: Faster inference, but more missed detections.
  • Two models: Covers the required vehicle classes, but inference is too slow on our CPU.

Hardware upgrades aren’t an option because of our limited thesis budget.

My questions:

What would be the best way to reliably detect all eight vehicle classes using only one lightweight model on this hardware?

Should we improve our unified YOLO training
approach, try a different lightweight architecture, or optimize OpenVINO inference differently?

We’re aiming for around 10 detection FPS per camera, although we understand that might be unrealistic for an i5-7200U.

I’d really appreciate recommendations from anyone who has worked with CPU-only vehicle detection or similar low-resource setups.

Help is so much appreciated Thank you!


r/computervision • • 1d ago

Commercial I indexed 90M movie & TV frames into a 768-d vector database so you can identify any scene via a single REST call

130 Upvotes

Hey everyone,

Over the past months, I’ve been building a frame-level cinema & TV scene recognition engine (**Visual DNA API**). The goal was to make an API that works like Shazam, but for single movie/TV screenshots—including cropped, mirrored, or subtitled frames.

**What it does:**

- `POST /v1/identify`: Send any frame (PNG/JPG/WEBP) → get back the movie/series title, release year, episode code, `tmdb_id`, and confidence score in JSON.

- `POST /v1/search`: Describe a scene in plain English (e.g., *"rainy neon cyberpunk street at night"*) → retrieve matching titles.

**Privacy & stack highlights:**

- 90M+ 768-d CLIP vectors indexed in Qdrant on EU servers.

- Volatile RAM processing: uploaded frames are vectorized in memory and immediately discarded (zero disk storage, zero biometric recognition).

**Try it without signing up:**

I’ll drop the live scanner (5 free daily scans + raw JSON output) and the open-source quickstart repo (Python, Node.js, Telegram & Discord bots) in the comments below.

If you test it with a tricky screenshot and it misses, drop the movie name here—edge cases help me tune the indexing pipeline!


r/computervision • • 9h ago

Help: Project I made a tutorial covering some really cool Computer Vision architectures

Thumbnail
youtu.be
7 Upvotes

Video covers:

  1. Photo library search, text removal, label generation
  2. Document layout detection models and RT-DETR (D-Fine) architectures
  3. Tennis ball, player, courtmap tracking system with CenterMap-style heatmap prediction
  4. And my favourite: World Models/Neural Computers with VQ-VAE + Spatio-temporal Transformers

Project repo: https://github.com/avbiswas/computer_vision_recipes

Video: https://youtu.be/mPmdI-0ISx4


r/computervision • • 8m ago

Help: Project Best approach to instance segmentation of thousands of hand-drawn tattoo flash sheets? Classical CV vs. fine-tuning SAM or YOLO

Thumbnail
gallery
0 Upvotes

I'm a professional tattoo artist, rather than a computer vision engineer, and I've been working on a personal project to digitize and organize many years' worth of my original artwork.

I've already identified over 4,000 unique Procreate documents across several storage devices, with another 90 GB of artwork still on my iPad.

Many of these canvases contain anywhere from one to 40 individual tattoo designs. My goal is to automatically isolate each design, preserve its original pixels, generate accurate segmentation masks, and build a searchable library of individual designs.

What makes this challenging:

  • Individual designs often contain disconnected linework, dots, sparkles and shading.
  • Adjacent designs can be extremely close together or physically overlapping.
  • Some sheets contain titles, handwritten notes, decorative borders and reference images.
  • The archive contains everything from rough sketches and fine line drawings to fully rendered color paintings.

What I've tried so far:

  1. Classical image processing: Using Python, NumPy, SciPy and Pillow, I've developed an AI-assisted implementation using thresholding, connected components, morphological operations, distance transforms and Voronoi-based mask partitioning. This has performed surprisingly well, particularly on clean flash sheets.
  2. Vision-language models: I've experimented with agents that visually analyze individual canvases, generate segmentation coordinates and independently verify the results. A recent 300-canvas run produced 690 candidate designs. The results were generally excellent, but processing time and model usage limits make scaling expensive.
  3. SAM and Grounding SAM: I've experimented with automatic and manually prompted segmentation using X-AnyLabeling. The results have been inconsistent on drawings, particularly when disconnected strokes belong to the same design.
  4. Human verification: I've also built a desktop application, with AI coding assistance, to review segmentation results, correct masks, compare duplicates and organize different versions of the same artwork.

I'm currently considering a hybrid pipeline where classical CV handles straightforward images, with more difficult cases routed to a specialized segmentation model.

I'd really appreciate some advice from people with experience in computer vision:

  • Are there established approaches or research papers specifically addressing instance segmentation of hand-drawn illustrations or line art?
  • How would you approach grouping disconnected strokes that semantically belong to one illustration?
  • Would fine-tuning SAM or training a YOLO segmentation model on a carefully annotated dataset be worthwhile for this particular domain?
  • Are there other architectures or classical CV techniques that might be better suited to this problem?

My priority is segmentation accuracy and preservation of the original artwork, rather than image generation.

I'm happy to invest time into creating a high-quality annotated dataset if it would significantly improve the results.

Any suggestions, criticism, research references or alternative approaches would be greatly appreciated.


r/computervision • • 18h ago

Showcase Full Visual SLAM & 3D Hand Tracking on a DIY AR mask: MediaPipe plus triangulation in C++, driving a full desktop UI

27 Upvotes

This is my solo-built system, VIOX. VIOX is the computer you wear. It's an AR personal computer that renders its own entire computer setup in an AR mask. VIOX doesn't use an external phone or laptop connection, because it isn't made to extend another computer's display, it IS the computer, with a hip-mounted LattePanda Sigma computing module.


r/computervision • • 8h ago

Discussion i built a free 36M-param tool that lets LLMs “watch” videos locally (MCP server, runs on laptop cpu, apache-2.0)

Thumbnail reddit.com
1 Upvotes

r/computervision • • 13h ago

Help: Project Replacing expensive polarized dome lighting with real-time Fresnel specular separation in PyTorch — is metallic glare still a major bottleneck on your lines?

0 Upvotes

I'm a computational optics researcher building lightweight wave-optics/Fresnel inverse solvers (ShivInterferentia) to strip specular glare from shiny metal parts and PCBs using a single standard camera at 35+ FPS.

Before I optimize this into a deployable C++/TensorRT SDK, I want to sanity-check with folks running real factory or robotics vision pipelines:

  1. How much time/budget do you currently lose fighting specular glare or bad lighting on shiny surfaces?
  2. What edge hardware (Jetson, x86 CPU, industrial smart camera) would this need to run on to actually be useful in production?

If anyone has a tricky, glare-ruined test dataset they want me to run through our solver for free as a benchmark, drop a comment or DM—happy to share the restored output and benchmark metrics.


r/computervision • • 21h ago

Discussion 24 AI Camera Streams on 1* RTX 5090: Zero Dropped Frames, but Still Missing Deadlines

4 Upvotes

Imagine you're building an AI monitoring system for a warehouse with dozens of cameras. You need to detect people, track their movements, and process incoming video without falling behind.

How many cameras can you assign to a single GPU before you need another?

If your object detector processes 300 frames per second and each camera needs 10 FPS, supporting 30 cameras sounds reasonable. But that calculation doesn't account for video decoding, object tracking, or delays between receiving a frame and finishing its processing.

I tested this on an RTX 5090 using YOLO11n and ByteTrack. At 24 simulated cameras, the pipeline sustained 240 FPS without skipping any scheduled frames during repeated 60-second tests. However, some batches still finished after their deadlines.

Offline demonstration of 24 simulated streams using time-offset segments from one source video, with YOLO detections and ByteTrack IDs.

Building the experiment

Rather than loading a separate detection model for each camera, I grouped frames into batches and processed them through a shared YOLO11n model. Each stream maintained its own ByteTrack instance.

ComponentConfiguration
GPURTX 5090, 32 GB
CPU / RAM allocation25 core equivalents / 90 GiB
DetectionYOLO11n, FP16
TrackingIndependent ByteTrack instances
Video processingOpenCV, CPU decoding
Target10 FPS per camera

I created the simulated streams using multiple readers of the same local 768 × 576 video. There were no physical cameras or RTSP connections involved.

The benchmark included decoding, detection, and tracking, but excluded network transport, rendering, and output video encoding.

From model inference to video processing

I started by benchmarking YOLO11n with preloaded frames. At a batch size of eight, it reached approximately 404 FPS.

Based on that number, processing 32 streams at 10 FPS each seemed feasible. Once I added video readers and ByteTrack, however, the available processing margin became much smaller.

The initial 24-camera tracking test skipped 11.5% of scheduled frames, even though processing times at lower camera counts suggested the pipeline should have been able to keep up.

I suspected that initialization overhead was affecting the results. After moving detector and tracker warm-up outside the timed window, the 24-camera test processed every scheduled frame.

I kept this warm-up procedure for the remaining steady-state benchmarks.

Finding the camera limit

I increased the camera count and extended the tests to 60 seconds per configuration.

CamerasAchieved FPSSkipped framesP95 latency
242400%92.98 ms
262600%108.74 ms
28274.871.83%191.09 ms
Camera capacity and latency graph

At 24 cameras, the pipeline processed all 14,400 scheduled frames with a P95 completion latency of 92.98 ms.

Adding just two more cameras pushed P95 latency above 100 ms, even though no frames were skipped. At 28 cameras, scheduler skips appeared and P95 latency nearly doubled relative to the frame interval.

Since each stream was targeting 10 FPS, a new synchronized batch was scheduled every 100 ms. A batch could finish late without necessarily causing the scheduler to skip subsequent frames.

I added a deadline-miss counter and repeated the 24-camera test three times. All 43,200 scheduled frames were processed, but 77 of the 1,800 batches finished after their 100 ms deadline (4.28%).

An individual 60-second RTX 5090 benchmark showing 14,400 processed frames, zero skipped frames, and a 2.33% late-batch rate. Across three separate trials, the late-batch rate was 4.28%.

An individual 60-second benchmark with 14,400 processed frames, zero skipped frames, and a 2.33% late-batch rate. The separate three-trial deadline test recorded a combined rate of 4.28%.

An application doing periodic video analytics might tolerate these occasional delays. For a monitoring system that needs consistently timely detections, I'd want to account for them when deciding how many streams to assign to each GPU.

What happened when I parallelized decoding

Video decoding was taking up a noticeable part of the processing interval, so I tried distributing frame reading across multiple CPU threads.

In an initial 28-camera test, four decoder workers reduced mean decoding time from 24.40 ms to 14.84 ms.

To check whether this improved the entire pipeline, I ran nine 60-second trials at 24 cameras, with three runs each for sequential decoding, two workers, and four workers.

MetricSequential2 workers4 workers
Aggregate FPS240210.53240
Skipped frames0%12.28%0%
Late batches4.00%95.28%18.17%
Mean P95 latency99.02 ms213.22 ms111.89 ms
Decoder comparison chart

With four workers, mean decoding time decreased from 22.77 ms to 13.05 ms. However, detection time increased from 49.91 ms to 57.88 ms, and tracking time increased from 16.39 ms to 20.34 ms.

The time saved during decoding was lost in the subsequent stages. The four-worker configuration maintained 240 FPS but completed considerably more batches late.

The two-worker configuration performed poorly across all three trials, skipping an average of 12.28% of scheduled frames.

I haven't isolated the cause of these regressions. Thread contention, CPU scheduling, and interactions between OpenCV and the inference pipeline are possible contributors. I'd need more detailed profiling to determine what's happening.

For this implementation, I'd stick with sequential decoding.

Would I deploy 24 cameras on one GPU?

Under the conditions I tested, 24 simulated streams at 10 FPS were manageable, although the deadline misses suggest limited processing headroom.

For a real monitoring system, I wouldn't assign 24 live cameras to a GPU based on these results alone. Independent RTSP feeds, different codecs, varying scene complexity, and output encoding could all change the workload. I'd also need longer-duration tests and measurements of detection and tracking quality.

The experiment doesn't establish whether an RTX 5090 is necessary, either. I only tested one GPU. Running the same pipeline on an RTX 4080 or a less expensive GPU would help determine whether the additional hardware investment is justified.

I'd also like to try asynchronous frame ingestion and per-camera buffering. The current implementation processes synchronized batches sequentially, which may be contributing to the latency behavior.

For someone planning a multi-camera system, I'd benchmark the complete application before deciding how much GPU capacity to rent or purchase. The 240 FPS figure alone would have missed a problem that became visible as soon as I started counting late batches.

For those running multi-camera CV systems, how do you determine the number of streams to assign to each GPU? Do you work with a latency target, a deadline-miss threshold, or leave a fixed amount of processing headroom?


r/computervision • • 21h ago

Showcase tamserve update: track anything you name on live cameras, at 95% of SAM 3's accuracy and 2.5x less GPU time (open source)

3 Upvotes

Last week I shared tamserve, an open-source serving engine for SAM 2-class video trackers. One copy of the model tracks every object on every camera, so a GPU handles far more cameras and memory stays flat. Since then more than 3,000 people have visited the repo and over 260 have starred it. Thank you.

The project has grown with more support this past week:

  • Track by name. Give each camera phrases such as "dog" or "the red mug". SAM 3 looks every few frames to find matches, and a small tracker follows them in between. On Meta's own SAM 3 benchmark it scores 56.3 HOTA against 59.4 for SAM 3's own tracker, in 41 ms of GPU time a frame against 101 on an RTX 4090.
  • People. A tracker tuned for people scores 89.6 HOTA against 87.8 on 116 home videos, in 8.9 ms a frame against 126.
  • Several GPUs and machines. A router spreads cameras over GPUs and keeps going when a camera or a machine drops. Two RTX 4090s tracked 476 frames a second from 32 cameras.
  • A look rate per camera. Suggested in the comments last time: a quiet camera can be looked at less often, which frees budget for a busy one.
  • Apple silicon. It runs on M-series Macs, with part of the model on the Neural Engine while the GPU does the rest. That is about twice as fast as the models' own code on the same Mac, and it keeps up with a live 30 fps camera.

What it is not good at:

  • In a crowd it only matches SAM 3's own tracker if it looks at every frame, which costs more.
  • Tracking by name tops out at about 30 frames a second in total on one RTX 4090, however many cameras share it.
  • A Mac is about a fifth of an RTX 4090, and I have only measured an M4 Max.
  • SAM 3's weights are gated by Meta, so you accept their licence to track by name.

Code, and every measurement with the command that reproduces it: https://github.com/aryamandarda/tamserve

If you run the Mac benchmark on a different chip, I would like to see the numbers.


r/computervision • • 20h ago

Help: Project How do you actually investigate the synthetic-to-real gap in CV, beyond tuning by intuition? Looking for practitioners' experience

2 Upvotes

I'd like to hear from people who have worked with synthetic or cross-domain training data in practice: How do you investigate what's actually wrong with your data, instead of tweaking it on intuition until the numbers improve?

Most advice I find stops at "close the domain gap" or "use domain randomization." It rarely explains how people found out which factors mattered for their problem. I'm after the real workflow: what you checked first, which analyses told you something useful, which were a waste of time, and what surprised you.

My case, as a concrete example

I'm working on UAV detection (fixed-wing vs. quadcopter) in MWIR thermal imagery. The general question isn't specific to this, but it helps to have something concrete.

  • Model: YOLO26m
  • Target data: small annotated MWIR set, split into K folds by video (no frames from one video in both train and test)
  • Public-data baseline: trained on MMFW-UAV + Anti-UAV-300 (LWIR, 5k images per class), evaluated on the MWIR held-out folds
  • In-domain baseline: trained on the MWIR training folds, evaluated on the same held-out folds

The public-data model is much worse: [X] vs. [Y] mAP50-95 (mean ± std over folds). A large public LWIR dataset loses to a small amount of in-domain data.

My target domain doesn't exist publicly, so the plan is to generate synthetic MWIR data, either physics/3D-rendered or learning-based (image translation, diffusion). I can list the obvious suspects: waveband, sensor noise, UAV models, backgrounds. What I don't have is a systematic way to tell which of them, or which unknown factors, actually drive the gap, or to check whether a change to the generator fixed the right thing.

What I'd like to hear about, from your experience on any CV task

  1. Locating the gap. How do you work out which factors cause the drop between training data (synthetic or another domain) and real target data? Did any metrics, controlled ablations or experiment designs actually give you an answer?
  2. Seeing what the model learned. What do you use in practice to understand what a model relies on and where it behaves differently across domains? For example, feature-space comparisons, attribution maps, error breakdowns by size/contrast/background, or probing individual factors. Which of these led you to a real fix?
  3. Turning findings into better data. How do you feed those findings back into the data or generator so each iteration is driven by evidence? What did that loop look like on your project?

r/computervision • • 1d ago

Discussion 80 downloads on my tiny vision model so i made v1.0, now it lets claude “watch” videos (36M params, runs on laptop cpu)

2 Upvotes

so few days back i posted my tiny vision model that i built just for learning (v0.1), and 80 people downloaded it on huggingface which honestly shocked me lol. people in comments gave really good feedback so i actually worked on all of it, and now v1.0 is out.

main thing: claude (and many other assistants) can look at images but cant open a video file. so tiny-vision runs locally next to it, breaks the video into scenes and gives the assistant a scene by scene breakdown. then you just ask “what happens in this video” and it actually answers. nothing gets uploaded, all on your machine. i tested it with a 10-15 min youtube video and it still works fine.

works through MCP so it runs in claude desktop and claude code (tested both), and should work in cursor, vs code, lm studio etc (havent tested those yet). if your app dont have MCP you can just pipe the text into ollama or paste it anywhere.

what changed from v0.1 (thanks to the comments):

  • someone said the “a man in a blue shirt” thing is probably the encoder not the decoder. i tested it and yaar they were right. my from-scratch encoder only saw 29k images, so i switched to a frozen pretrained DINOv2-small (meta’s) and trained the adapter + decoder myself (14M params) on COCO
  • it picks frames on scene changes now instead of one every 2 sec, like someone suggested. so the timestamps land on the actual cuts
  • real benchmarks this time, not vibes

numbers (COCO test):

  • v0.1: CIDEr 0.06 💀
  • v1.0: CIDEr 1.16

vs florence-2-base (231M) on the same 1000 images:

  • ~89% of its caption score
  • 6x smaller
  • 8x faster on gpu
  • 75x faster on cpu (54ms per image vs 4 sec)

not saying its better than florence, its not. it just does a lot with very little.

honest stuff:

  • encoder isnt mine anymore, its DINOv2. i tried my own and it just cant compete without millions of images
  • no audio yet, so it describes what it sees, not what is said. speech is coming next
  • captions are COCO style, so no OCR, no names, no counting
  • and like last time, i took a lot of help from claude to build this, so its not entirely mine

previous post: https://www.reddit.com/r/computervision/comments/1xyuaia/built_a_tiny_vision_model_from_scratch_just_for/
code: https://github.com/amar683/tiny-vision
model: https://huggingface.co/amartomar683/tiny-vision

anything you want me to try next? still learning so all feedback welcome


r/computervision • • 1d ago

Showcase Real-world inference benchmarks: Linux CUDA EP vs Windows DirectML on mobile GTX 1650 Ti (i5-10300H)

2 Upvotes

Lobo-to-lobo comparison on the same serviced laptop hardware (Intel i5-10300H + discrete mobile GTX 1650 Ti with GDDR6). Bypassing heavy Python dependencies via a lightweight native C-API pipeline.

The Pipeline:

  1. Frame capture to RAM (NV12 format).
  2. Localized detection: YOLOX-nano (640x640) inside a targeted 3x3 control zone. Pure FP32 to ensure bounding box geometry stability.
  3. Feature extraction: 256x256 ROI crop processed for 44 distinct texture/geometry metrics.
  4. Classification: Downstream CPU-bound LightGBM classifier evaluating extracted features.

Key Technical Outcomes (Part 6): Windows 11 + DirectML backend: Hitting a stable 106–111 FPS (~5.0 ms inference latency). Standard DX12 compute shader translation introduces slight frame-pacing variance (t_max up to 11 ms). Linux + Native NVIDIA CUDA EP backend: Delivering a constant 114–116 FPS (~4.1 ms inference latency) with a strictly bounded t_max limit. Shifting from DirectML to the native CUDA driver layer reduces raw execution overhead on the Turing architecture. Preprocessing overhead: Isolated at 1.5–2.2 ms inside OS threads, remaining consistent across both environments.

https://medium.com/@olesha-ai/bypassing-the-avx2-tax-part-6-same-hp-on-linux-cuda-wins-again-db4c95673a2a

Best regards.

olesha-ai


r/computervision • • 1d ago

Help: Project Removing light lettering from a colored ribbon after OCR: how would you make the glyph mask more reliable?

1 Upvotes
The full editor after OCR, tracing and text reconstruction. The selected title is a live text object.

I’m testing editable text recovery on a flattened 1122 × 1402 Korean festival poster shared by Design Company Eun. EasyOCR gives me line boxes and text. In this case it found six regions, misread a venue character and missed a hand-lettered headline that I boxed manually. The question is about the pixel mask after the line is known.

In a padded crop around pale lettering on a red banner, a lightness threshold can include pale background outside the banner. My fallback runs Otsu on grayscale, splits the light pixels into 8-connected components, and drops components touching the padded crop boundary. I intersect the result with the approved text box, dilate the mask slightly and run OpenCV Telea inpaint. This kept the pale exterior out in that sample. The crop-boundary rule can discard a genuine stroke, and textured edges can leave visible residue.

Downstream, VTracer WASM traces the cleaned art. I render candidate fonts at the fitted size in Canvas 2D and score their mask overlap and aspect ratio. The renderer then restores actual editable text. ClusterGlyph and Distill-SVG were references for the pipeline design.

Would you start with a local Lab foreground/background color model seeded inside the OCR box or a learned text matte for this failure mode? I’m especially interested in evaluating anti-aliased edges when the only original is a flattened JPEG.


r/computervision • • 1d ago

Discussion Real-time alternatives to YOLO for detecting human-like figures in live video? (Desktop & Laptop GPUs & CPUs/TensorRT/low latency)

5 Upvotes

I'm interested in moving beyond traditional YOLO versions (v8/11/26) and investigating alternatives for real-time detection of human like figures in a live video stream (high framerates).

What I'm trying to detect: Human-like figures (various shapes, often in outfit/equipment, and in scenes that could be stylized/videos rather than very conventional CCTV footage)

Real-time requirements: Ideally, I'd want to get sub-5ms or sub-10ms inference times on desktop CPUs/GPUs (at 960x960, 640x640, 320x320 resolution)

Deployment requirements: Can export to ONNX format easily and deploy to NVIDIA TensorRT (with some degree of stability in, say, FP16 or FP8 quantization and without getting bogged down in custom plugins)

Primary challenges that I'd like to address:

- Heavy partial occlusions (only part of a figure's head/shoulder/torso may be visible)

- Rapid camera pans and motion blur

- Bounding box jitter between frames in a video stream

- Detecting small, distant figures that grow rapidly

I've been tinkering with NMS-free real-time transformers such as RF-DETR, D-FINE, DEIMv2, RT-DETRv2 and RT-DETRv4, as well as single-stage human/pose/keypoint detectors such as RTMO or EdgeCrafter.

For those of you using other detectors (other than YOLO variants) in production for live video streams:

- What's the best tradeoff you've experienced in terms of real-world inference latency (TensorRT or CPU) vs detection performance (i.e., recall) for human(-like) figures in presence of occlusions?

- How bad are the ONNX export and TensorRT FP16/FP8 quantization experiences for other detectors (compared to CNNs)?

- Should I adopt a simple single-stage pose/keypoint detector instead for better localization in cases with occlusions? Or is a fine-tuned DETR-like model still the way to go?

- Any insights, benchmarks, repo links, or TensorRT deployment experiences you can share?


r/computervision • • 2d ago

Showcase My iPhone filmed our amateur football match by itself

770 Upvotes

A fine-tuned ball detector + players decide where to look, a cheap gimbal moves the camera

I play amateur football, and nobody ever wants to stand behind the camera. So I made my iPhone do it, on a simple phone stabilizer, the kind people use for vlogs (mine is a DJI Osmo Mobile 8P, €159: https://www.dji.com/osmo-mobile-8p):

  1. Find the ball: YOLO26s fine-tuned on amateur footage, where the ball is often under 16 px, at night.
  2. Pick where to film: a hybrid rule follows the ball, and the players when the ball is hidden.
  3. Move the camera: the phone drives the gimbal's motors, like a cameraman panning.

All on the phone, about 30 times a second. This clip is from our match last Wednesday.

I've made the model, the dataset and the training code open source:

On a match the model never saw, fine-tuning takes it from 51 to 99 balls found out of 219. Tiny balls are still hard (under half are found), which is why the players matter. The app itself is still in development.

Tell me what you think, and if you'd like to get involved, I'd be glad to hear from you!

Edit: thanks for all the questions about the app! I’ve open-sourced it too: https://github.com/Thibault-GILLARD/box2boxcam-ios (gimbal control over DockKit, the real-time Core ML pipeline, the camera logic). It’s still in development, and my bottleneck is testing on real matches, so feedback from anyone who films a game with it would be gold.


r/computervision • • 1d ago

Help: Theory help me!!!monocular depth estimation

0 Upvotes

我正在寻找一个想法,以解决在包含透明物体的场景中进行单眼深度估计所面临的挑战,并希望将其发展成一个研究项目。你有什么建议吗?


r/computervision • • 2d ago

Showcase I taught my phone to count my reps and catch bad form

19 Upvotes

I’ve been building Repchek, a web app that counts reps and checks exercise form using your phone camera.

It runs entirely in the browser using MediaPipe.

A few things I learnt while building it:

My initial assumptions about joint angles were wrong. Real movements don’t always match what you expect, so I built a debug mode to record and analyse metrics from actual workout sessions.

Pose detection struggles with occlusion. For example, during push-ups, shoulders can appear closer together and wrists can disappear. I had to account for this using measurements captured at the start of a set.

Tracking the biggest person in the frame doesn’t always work. In a busy gym, that person might be the spotter. I added logic to identify and follow the person actually exercising.

Counting reps before someone is in position creates false positives. A readiness check helped solve this.

Repchek currently supports 13 exercises, including holds like planks and stretches.

Some limitations I’m still working on:
- Handling occlusion more reliably.
- Personalizing thresholds for different users.
- Testing newer exercises with real workout footage.

I’d appreciate feedback, especially from anyone who’s worked with pose estimation models.

GitHub: https://github.com/frankndungu/repchek
Try it live: https://repchek.fit


r/computervision • • 1d ago

Discussion If someone can help as mentor, or buddy that would be awesome

5 Upvotes

Looking for someone perhaps bachelor degree or master in AI who is willing to connect one hour per week to go through computer vision.I am following this book: Szeliski, R. (2022). Computer Vision: Algorithms and Applications, 2nd ed. Springer.

Need some coaching or some buddy to study together, any expert or someone who is studying and can hep


r/computervision • • 1d ago

Help: Project Looking to connect with Computer Vision researchers and professors

8 Upvotes

I'm exploring Computer Vision, particularly Visual SLAM, 3D perception and reconstruction, through research-oriented projects and open-source contributions OpenCV and Kornia.

Looking to connect with professors, researchers, PhD students and research groups working in these areas for potential research collaborations, contributing to ongoing projects or exchanging ideas.

I bring a software engineering background and hands on experience building and experimenting with these systems. Also open to research guidance from people in the field.

If you're working in this space, I'd be glad to connect.


r/computervision • • 1d ago

Help: Theory Prof explains and I don’t get it

0 Upvotes

Lecturer explaining resnet, vggnet, googlelenet and alexnet in cnn class.

I have no clue what is that ? I am concentrating 100% and still can’t catch up.

And how resnet is better than human being ?

And what is convolution is local connection plus weight sharing ? What does this mean

And what is the inductive bias : translation equivalent


r/computervision • • 1d ago

Discussion [D] What the Hell is going on at ICLR this year?

Thumbnail reddit.com
0 Upvotes

r/computervision • • 1d ago

Research Publication Hierarchical detection of neonatal brain pathologies using a two-stage deep learning framework from magnetic resonance imaging

Thumbnail
frontiersin.org
1 Upvotes

r/computervision • • 2d ago

Discussion after tiny vision, i wanted to see if i can build a tiny language model from scratch too (36M, NCM paper, trained on macbook) 😅

3 Upvotes

so few days back i posted my tiny vision model here(https://huggingface.co/amartomar683/tiny-vision-flickr8k) and you guys actually checked it out, its at 74 downloads now 😅 thanks for that. after that one worked (kinda), i started wondering what else i can build from scratch on my mac, and a language model was the obvious next thing to try.

i was reading the Next-Concept Modeling (NCM) paper by Kumbong, Khare et al. (Stanford / Together AI, arXiv:2610.14162) and the idea really clicked for me. normal llms write one token at a time, but in NCM a VQ-VAE turns text into levels of codes, coarse to fine, and the model fills one full level at once. so i decided to build it myself from the paper, just to understand how it works from inside. all credit for the idea goes to them, i just implemented my own version.

its called tiny-ncm. its around 36M params (21M generator + 15M VQ tokenizer), trained completely from scratch, no pretrained weights. same macbook pro m5, but this time it took like 12 hours total instead of 42 min 💀

basic idea:

text → split in 32-token blocks → each block made coarse to fine in 5 levels (2 → 4 → 8 → 16 → 32 codes) → 6 passes per block instead of 32

i trained it as a small daily assistant. alarms, reminders, timers, calendar, gmail (read only), weather, notes, web search, simple maths and little bit of small talk. there is a --show-levels flag which shows each block going from rough guess to final text, honestly this was the most satisfying part:

level 1: Hello! How can I help you today?
level 3: Hello Jordan, How I I you you today today?
level 5: Good morning! What can I do for you today?

some honest stuff:

  • it picks the right action most of time, but copying details is very bad. “London” became “Singapore”, “5pm” became “5pmpm”, “23 times 47” became 23*10 😂 cause all codes in a level are predicted same time, so two positions can both write the same word.
  • so i added app-side checks that fix the tool arguments using your actual words. with checks it gets 490/500 on my test set, but model alone gets every detail right only around half the time. so ya, the checks are doing lot of the work tbh.
  • training longer on same data didn’t help. what helped was making the data copy-focused, with new random names and times in every example so it cant just memorise. same lesson as tiny-vision (“a man in a blue shirt” for everything lol).
  • i tried fine-tuning it to summarise web results also. it learned the format but wrote gibberish inside it (“Stir in cheeseaksil sl fresh biil” 😭), so i reverted it.
  • no coding, no multi-step maths. “8 slices, 3 friends eat 2 each” and its gone.

random thing i learned on mac (MPS):

  • attention was the main bottleneck. letting each level only look at the last 128 tokens of context made training around 2x faster.
  • bf16 matmul was way faster than fp32 on MPS (~15 vs ~3.4 TFLOPS in my test).
  • manual attention was again faster than pytorch’s sdpa for small sizes, same as tiny-vision.

next i am thinking to export it to onnx so it can run on phone also, and maybe connect tiny-vision with it. lets see what will happened.

code: https://github.com/amar683/tiny-ncm
model: https://huggingface.co/amartomar683/tiny-ncm

would love any feedback or tips, especially if someone played with NCM or coarse-to-fine text models