r/reinforcementlearning • • 8h ago

Drone quadcopter RL pilot

5 Upvotes

I have been working on training a model (the pilot) to fly my drone. I have a simulator and the pilot has learned to fly, hover, go to colored pads, do orbits, maneuver over hills. It’s not a perfect pilot, it kinda looks like a kid flying the drone. I have the pilot running on a raspberry pi and connected to sensors and a UART line into the flight computer so the pilot can fly in the real world but haven’t proved the sim training will transfer. So I’m deep into the project but nowhere near done.

I’m looking for a community of like minded people taking on the RL flying of drones. Any discords or groups that are working on something similar? Mostly I am interested in having the drone fly through woods on its own but I think any RL flying will be close enough for some good conversations


r/reinforcementlearning • • 4h ago

P Ataraxos-UNO: Imperfect-info RL agent for 1v1 UNO (VRPO + EMAgnet + autoregressive belief net + 192-world WebGPU search in browser)

2 Upvotes

I adapted the Ataraxos framework (Sokota et al., Nature 2026) from Stratego to 1v1 UNO, including same-type draw stacking. The interesting part was getting the entire test-time search loop (belief sampling + 192 parallel rollouts) running in the browser, with no backend, using ONNX Runtime Web.

How it works

  1. C++17 game engine: 7.8M steps/sec across 512 parallel environments on Apple Silicon. Exact 10,000-game parity with uno-gym, including reshuffle provenance tracking.
  2. Blueprint: A 100M-step RL model (~2.45M games, ~5 hours on Apple Silicon MPS). Hybrid SwiGLU ResMLP, dilated-conv history and cross-attention, trained using VRPO Expected SARSA(λ), with symmetry handling for UNO's four colours.
  3. Belief network: Trained on 20M positions to predict opponents' hidden cards. Uses a hard-masked autoregressive decoder and history dropout. It outperformed an exact hypergeometric prior by 1.19 nats/hand on validation. You can see its predictions live using X-RAY.
  4. Test-time search: Blueprint-only search initially looked good against itself but struggled against greedy opponents. Turns out, it had learned to expect opponents to hoard Wilds, just like itself. I ended up using three opponent styles (Blueprint / Aggressive / Unpredictable) with entropic soft-minimax to reduce this overfitting.

Results (Mirrored Duplicate Decks)

  • vs. ExpertHeuristicBot: 66.0% (132–68, 200 games) vs. Blueprint's 61.0%
  • vs. 3-opponent suite: 58.3% vs. Blueprint's 55.3% (+3.0 percentage points, 600 duplicate games)

I'd be interested in feedback, especially on handling non-alternating turn chains in imperfect-information games and reducing rollout overfitting against opponents that don't necessarily play like the agent.


r/reinforcementlearning • • 2h ago

QMIX in METIS-Core v0.5.0 (example PredatorPreyQMIX)

1 Upvotes

METIS-Core v0.5.0-alpha now implements the **QMIX** algorithm! 🚀

QMIX is a **MARL** (Multi-Agent Reinforcement Learning) algorithm of the **CTDE** type (Centralized Training, Decentralized Execution). This means training is centralized, but once trained, the agents can be deployed individually: each one makes its own decisions based on what it observes locally.

**Original paper:** *"QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning"* by Tabish Rashid, Mikayel Samvelyan, Christian Schroeder de Witt, Gregory Farquhar, Jakob Foerster and Shimon Whiteson (ICML 2018). [arXiv](https://arxiv.org/abs/1803.11485)

**PredatorPreyQMIX** is a simple example, a sort of "Hello World" for QMIX. The environment is a 7×7 grid with one prey and two predators, drawn as drones:

* To catch the prey, both predators must execute the "Capture" action **at the same time**, while both are one cell away from it. If a predator tries to capture it alone, it gets penalized.

* Each predator can only "see" up to 2 cells away.

* There is no reward per drone, only **a single team reward**.

So the two drones have to learn to cooperate: one waits next to the prey while the other approaches, and they capture it together. Nobody programs this strategy; they discover it during training.

This example opens up some really nice challenges. The next step on the roadmap is to bring these examples closer to real-world scenarios and to prepare METIS-Core for distributed reinforcement learning with an actor-learner architecture: several machines generating episodes in parallel and another one dedicated only to training. More realistic scenarios will need much more compute, and this architecture will make it possible to spread it across several machines, for example on a cluster. That's the plan, anyway. Fingers crossed I get there! 🙂

Graphics made with [raylib](https://www.raylib.com/).

**Links:**

* METIS-Core: https://github.com/felixromo314/METIS-Core

* PredatorPreyQMIX tutorial: https://github.com/felixromo314/METIS-Core/blob/main/tutorial/Tutorial-QMIX-PredatorPrey.md

* Multi-Head tutorial (SARSwarm): https://github.com/felixromo314/METIS-Core/blob/main/tutorial/Tutorial-MultiHead-SARSwarm.md (a good candidate for QMIX: with it, the learned policy could be deployed on each drone individually)

Feedback is very welcome! I've also opened a Discussions section on GitHub: https://github.com/felixromo314/METIS-Core/discussions


r/reinforcementlearning • • 16h ago

DL, M, R, Robot "SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model", Liao et al 2026

Thumbnail
arxiv.org
7 Upvotes

r/reinforcementlearning • • 1d ago Bravo

How Difficult Is a Master’s Thesis in Quantum AI and Reinforcement Learning?

7 Upvotes

I’m currently pursuing an MS in Artificial Intelligence and considering a thesis combining Quantum Computing and Reinforcement Learning (QRL).

How difficult is this domain for an MS-level thesis, particularly for someone with a computer science and AI background?

Would selecting a recently published base paper, reproducing its results, and introducing incremental improvements be a feasible approach to completing an MS thesis and potentially publishing a research paper?

I’d appreciate recommendations for relatively straightforward, recent papers that could serve as a starting point, along with insights into the difficulty, computational requirements, and realistic publication prospects


r/reinforcementlearning • • 18h ago

D, MF, Safe "Models are grader modelling not score seeking; grader-modelling is much scarier"

Thumbnail
lesswrong.com
0 Upvotes

r/reinforcementlearning • • 1d ago

P CAIRN - World first 50%+ win rate slay the spire 2 AI.

19 Upvotes

been working on this for a several months. it plays slay the spire 2 as the ironclad on ascension 10 (the hardest difficulty), in the actual game, nobody touching anything. it's won 124 of its 222 real runs so far. 56%, it’s likely more now but not had the time to run in game, only sims. for reference gpt-6 gets 40% on the same setup in epoch ai's spireclimb benchmark. that's 15 runs on an older game version, so take it as rough. no llm anywhere. it's a neural net trained with reinforcement learning, only ever inside a simulator of the game i wrote, over a billion decisions on a couple of gaming pcs at home. then it gets plugged into the real game and has to hold up.

So pooled over all 222 runs it's 55.9% real vs 61.9% what the simulator predicts. gap -6 points (95% ci -12.6 to +0.4). acts 1 and 3 line up almost exactly. the whole gap is act 2: 13% of real runs die there vs 6% in sim. still hunting that one and the classic: it found bugs in my simulator and farmed them. a relic handing out a free card, a cost bug, a replay bug. worth about 2-3 points of fake win rate until i fixed them 🙃
also started a defect run this week: 0% to 13.6% in 33M decisions. So I’m curious if I can get it to use data from in game to find new bugs….
not sharing the exact setup yet, but happy to talk results and the sim-to-real stuff. For now I have named the model CAIRN. We will see if it sticks:)


r/reinforcementlearning • • 1d ago

BalatroNET: JAX-Native Reimplementation of Balatro's Mechanics for Reinforcement Learning [P]

9 Upvotes

Github

I've been working on this for a few months now and finally got to a full recreation of the base game (antes 1-8 any deck or stake, but no endless or challenge runs). The main artifact is the environment: jitted, vmapped implementation that can run in parallel on GPUs. I have also worked on a simple agent and a pygame front end where you can step through a run and observe the agent's choices.

I've done some training on an RTX 2070S, the agent can pass white stake red deck about 50% of the time after 6.6 billion training moves. This approach is faster than other implementations such as balatro-gym, which can only run on a CPU, and in my testing (AMD 9800X3D), hits about 1,700 steps/s (per core) while running random actions. On this 2019 GPU, training runs at 16,000 steps/s, all while holding the environment, observation, and performing the network's forward pass and gradient updates.

I'm now opening this up to the public to share progress and see what insights people are willing to give.


r/reinforcementlearning • • 1d ago

DL MCTS over LLM coding agents: the value function is the bottleneck (small negative result)

0 Upvotes

I'm an undergrad in RL. I built agent-mcts, which treats each Claude Code run as a node in a search tree. Expansion starts a new agent session, either a fresh approach from the root or a revision of a parent that sees its test output. The value is the visible test pass ratio, optionally averaged with an LLM judge score. Selection is UCT, and the top candidates get a final head-to-head comparison.

To measure it I used tasks with hidden test suites the agent never sees, and compared the chosen answer with the first attempt of the same run, which is the no-search baseline. Two tasks, haiku, 5-node budget, 10 runs per setting.

  • Tests-only value: 0 better / 9 same / 1 worse. The first attempt already passed the visible tests in 9 of 10 runs, so the reward had nothing left to say.
  • Tests + judge: 4 better / 4 same / 2 worse, mean change -0.1 pt.
  • For pairs of attempts that both pass the visible tests but differ on the hidden ones, the judge ordered them correctly 38 times out of 62 (4 ties).
  • Revisions under the judge were mostly neutral: 7 better / 12 same / 1 worse.

So the search does what the reward tells it to; the reward just isn't informative once the visible tests saturate. Small n, so take it as one data point.

I'd be interested in ideas for a better value estimate here: pairwise comparisons instead of absolute scores, generated tests, ensembles of judges, a learned verifier?

Code and benchmark: https://github.com/natsu0529/agent-mcts

Update (v0.4): I changed the value function and it now beats a single run on my benchmark.

Before the search starts, an agent writes extra pytest files from the task description only (it never sees an attempt). Each attempt is scored half on the project's own tests and half on these. The generated tests also go into the revision prompt, so a failing test tells the next attempt what to fix.

Same two tasks, haiku, 5-node budget, 20 runs: the final answer beat the first attempt 16 times, tied 2, lost 2. Mean hidden score went from 88% to 93%. Most of the gain comes from revisions (32 of 40 raised the hidden score), not from picking among first attempts.

One way it fails: on a vague bug report, the test writer made the same mistake as the bug, so the unfixed code scored highest and got returned every time. v0.4.1 only returns the unchanged code if every attempt does worse on the project's own tests.

Still small (three tasks, one model). Numbers are in bench/README.md.


r/reinforcementlearning • • 1d ago

Would an online competitive game where players train RL agents in real time be feasible?

Thumbnail reddit.com
3 Upvotes

r/reinforcementlearning • • 1d ago

Safe AI plays Quake Live / Quake 3 [NeuralFPS Framework]

Thumbnail
youtu.be
5 Upvotes

r/reinforcementlearning • • 2d ago

How transferable do you think RL post-training really is?

10 Upvotes

Around a year ago I was convinced the labs were doing a bait-and-switch with the claims that the scaling laws would apply to RL like they did to pre-training.
Now I’m not so sure. Clearly their plan is to march down the verifiability ladder RLing the whole world. It seems to me that all the extreme superhuman-at-every-task projections come down to this central question.
It does stand to reason that if you RL enough related tasks and your representations are good enough, new tasks should topple much faster.


r/reinforcementlearning • • 2d ago

Ent v0.1 - a general-purpose RL agent that doesn't need tuning

Post image
30 Upvotes

My lab is building a MBRL agent from scratch that is able to robustly perform in any environment out of the box without per task tuning. We decided to open up a bit with this research preview on our current progress.

Ent was tested on 35 very different environments, from control and mazes to arcades and multi-agent games, alongside with DreamerV3, MuZero, PPO, SAC and DQN - one single tuning for each. Ent is reliably good on 29 of the 35 and never fails outright.

Like probably many in the field, we were inspired back in the day by RL successwes in games like Chess, Go, Atari, DotA etc. It felt like RL could be this steamroller that goes and solves everything. Years passed, people seem to have abandoned the idea of general-purpose zero-knowledge agent and settled instead on tuning a very specific version of PPO for every single task or do imitation learning and hope to solve robotics that way.

To us, this feels like a giant gap. An agent that can't reliably do various different simple tasks, shouldn't be able to perform in a messy real world scenarios. If your algorithm needs you to very carefully tune it for every single task, is it still RL or a sort of meta supervised learning?

Thus we decided to breach this gap and develop an RL agent that doesn't need all that and could be deployed out of the box, build a robust world model, plan and continuously learn. We're starting with small but very diverse environments and gradually increase the difficulty without loosing generalist abilities.

Full blog post with detailed results: https://regularlab.ai/blog/work-in-progress-ent-v0-1/

Curious what you think of this result.

P.S. it's a bit outdated already, internally it's 33/35 by solving exploration issue. 35/35 is the goal though


r/reinforcementlearning • • 2d ago Bravo

Independent researchers who published at ICLR, ICML, or NeurIPS, how did you actually pull it off?

Thumbnail reddit.com
5 Upvotes

r/reinforcementlearning • • 3d ago

DL, MF, R, Safe "Gradient Routing (GRAFT) Localizes and Removes Unintended Behaviors in RL"

Thumbnail
lesswrong.com
18 Upvotes

r/reinforcementlearning • • 2d ago

Training a neural network to play the trick-taking game Sheepshead via self-play RL and ISMCTS search distillation

7 Upvotes

As a fun side project over the last year and a half or so I've been working on training an agent to play a trick-taking card game called Sheepshead using self-play reinforcement learning. This game is my favorite card game and is well known in Wisconsin, but virtually unknown elsewhere. It has unfixed, asymmetric teams and separate bidding and play phases, which make it an interesting RL testbed.

In order to get to the skill level I wanted, I combined 3 separate policy improvement stages:

  1. Pure self-play PPO
  2. League-based CTDE
  3. Expert Iteration with Information Set MCTS as the expert teacher

The final ExIt phase required what I believe is a novel search distillation technique. It borrows the Fay-Herriot estimator from small area estimation to blend a learned advantage model with the search committee's values, weighting each by the inverse of its estimated noise. It then uses the result to tilt the network's own policy into the distillation target.

Full write-up: https://bender.io/2026/11/17/building-an-expert-sheepshead-ai-with-reinforcement-learning/

The model is available to play against and view some internal details at https://sheepshead.ai (the "analyze AI" section shows action probabilities, critic outputs, etc)

Code is on GitHub: https://github.com/adamanthil/sheepsheadai

I'm pretty proud of how the project came together. Hope you check it out.


r/reinforcementlearning • • 3d ago

R, DL, Multi, Safe, MetaRL "Models Can Communicate via Inference Servers Despite Being Sandboxed"

Thumbnail
lesswrong.com
12 Upvotes

r/reinforcementlearning • • 3d ago

R Binary-gated GRPO on GDPval: 79% of GRPO groups had no passing rollout, and pass@1 rose from 31.9% to 40.7% while plain GRPO barely moved (Qwen3.8-27B)

13 Upvotes

I work at Snorkel AI, and we wanted to share new training work from my colleague that may be of interest here. The write-up is linked at the end of the post.

The setting is RL on long-horizon professional tasks, where a single attempt averages about 50,000 response tokens and success is rare. Qwen3.8-27B was trained on 1,105 tasks from the GDPval+ series. Each task has an expert-built rubric and reference solution, and each group has eight rollouts, about 102K rollouts in total. An attempt passes when at least 90% of rubric items are satisfied.

With a binary signal, GRPO gives zero advantage to any group where every rollout fails. Here, 79% of generated groups had no passing rollout. Dynamic sampling skips those groups and refills the batch, but it does not create new signal. In the groups GRPO discarded, the median best rollout scored 78% on the rubric, and 66% of them had a rollout scoring at least 70%. The attempts in these groups differed substantially.

Binary-gated GRPO works as follows.

  • If a group has at least one pass, the advantages are the standard GRPO advantages on pass/fail.
  • If no rollout passes, the advantages come from a fallback on the rubric score. Scores at or below 40% map to zero, and scores above that scale linearly up to the 90% pass threshold. The fallback is weighted by 0.85.

The fallback is used only when there is no pass in the group, so partial credit never competes with a pass.

On the 185 GDPval gold tasks, with eight attempts per task, the results were as follows.

  • Groups carrying a training signal rose from 21% under GRPO to 95% under binary-gated GRPO.
  • Pass@1 rose from 31.9% to 40.7%. GRPO on the same data and a similar rollout count barely moved from the base model.
  • In LLM-judged head-to-head comparisons against human experts, deliverables were rated better in 44.0% of cases, up from 36.4%.
  • Attempts that exhausted the response budget fell from 17.2% to 7.0%.

Each method was trained once. The judge is an LLM and is noisy, which is why the pass threshold is set at 90%. The results cover GDPval-style tasks only.

Write-up: https://snorkel.ai/blog/gdpval-agents-rare-success/

Earlier post on partial credit and completion: https://snorkel.ai/blog/when-partial-credit-opposes-completion/


r/reinforcementlearning • • 2d ago

Want a teammate for a Robotics RL Research Project

Thumbnail reddit.com
1 Upvotes

As all of u suggested https://discord.gg/MsNWJqBRc This is link to my server everyone interested join nd if possible spread in other RL and robotics community


r/reinforcementlearning • • 3d ago

I mapped 39 companies selling RL environments for LLM agents. 29 do coding, 2 publish a price

9 Upvotes

I've been tracking the companies that sell RL environments (tasks, sandboxes, verifiers) to labs doing post-training. Disclosure: this is my comparison site.

By environment type (most vendors do several): coding/SWE 29, tool use 21, domain experts 21, app replicas 19, computer use 16, data major 12, platform/hub 6, reasoning 4.

Only two publish a price: HUD (Pro plan $50 a month; hourly rates only after login) and Zero-Trust Physics ($9,500 per pack of 30 physics tasks). Open options: Prime Intellect's Environments Hub (2,500+ community environments), OpenEnv (BSD-3).

Full table with sources: https://toprlenvironments.com/

For those training agents: are you building environments in-house or buying them? What made the call?


r/reinforcementlearning • • 3d ago

Research paper explanation Found out this JEPA - LE WORLD paper explanation. What are ur views

Thumbnail
youtube.com
0 Upvotes

I was searching for some good explanation on the JEPA models research papers. Found out this explanation on youtube. I felt it made me skim through the papers really well. Incase it helps, you can watch it too. This channel also has another video which explains the inner mathematical functions and base too.


r/reinforcementlearning • • 3d ago

D, Safe "Managing frontier AI risk: From training to deployment", OpenAI (defense-in-depth for RL training runs)

Thumbnail
openai.com
2 Upvotes

r/reinforcementlearning • • 3d ago

P websandbox: the WebArena sites as small SQLite + Node servers for web-agent RL (ms resets, template-disjoint splits)

4 Upvotes

websandbox rebuilds the five WebArena sites as one SQLite DB plus a small Node server each, for RL on web agents. Resets in milliseconds, dozens of copies per machine, 807 of 812 tasks verified plus 12,283 generated, template-disjoint splits. Apache-2.0: https://github.com/junfengzh/websandbox


r/reinforcementlearning • • 3d ago

DL, MF, I, R, Exp Rethinking "RL Generalizes, SFT Memorizes": The Role of SFT Data, Hou et al. 2026 [SFT matches generalization of RL on the same data (no. of rollouts); diversity in teacher models is beneficial]

Thumbnail
arxiv.org
1 Upvotes