r/MachineLearning • • 1h ago

Discussion Where does your robot manipulation training data come from? [D]

1 Upvotes

For people training manipulation policies or world models: teleop, sim, public video like Ego4D, or do you pay a vendor?

If you've tried human egocentric video, did it help or did the embodiment gap kill it?

I'm working on egocentric data collection, so I'm curious what's working for people.


r/MachineLearning • • 5h ago

Project We got tired of debugging robot policies across 5 windows so we are building Kepler [P]

2 Upvotes

anyone who has trained a robot policy knows the workflow. VS Code open, ROS 2 terminal running, Isaac Sim in another window, LeRobot script somewhere, Rerun for visualization. five tools, one task.

the real cost is what happens at the boundary between tools. a policy fails and you spend an afternoon correlating logs across three applications with no shared concept of time.

we are building Kepler to fix this. one workspace where you record demonstrations, train policies, visualize trajectories, and trace failures without leaving the tool. built specifically for people working with LeRobot and similar stacks.

waitlist is open: kyroslabs.tech

also wrote about why we think this problem is bigger than it looks if you want to read: kyroslabs.tech/blog

happy to answer questions from anyone who works in this space.


r/MachineLearning • • 12h ago

Project PhiloLM: a 112M language model trained from scratch on philosophy texts [P]

Post image
20 Upvotes

I trained PhiloLM, a 112M-parameter language model from scratch using only openly available philosophy texts.

The training corpus, PhiloText, contains 0.5B tokens from philosophy encyclopedias (SEP, IEP), public-domain books, reviews, textbooks, and educational resources.

The model was trained on 1.5B tokens in 15 hours on a single Intel Arc A770 (16 GB). The architecture uses grouped-query attention, sliding-window attention, bigram embeddings, and the Muon optimizer.

Model and technical details: https://huggingface.co/gustavecortal/PhiloLM

The training dataset and interactive demo are also publicly available. Happy to discuss the architecture, training setup, or results.


r/MachineLearning • • 15h ago

Discussion ICLR 2027 - Deleted Reviews Are Back [D]

7 Upvotes

Has anyone else noticed that the deleted AI-generated reviews are visible again on OpenReview for ICLR 2027?


r/MachineLearning • • 16h ago

News ICLR 2027 Rebuttal Score Ranked by University [N]

Post image
60 Upvotes

Can't believe this is the third time posting it

Crawled from OpenReview, titles matched to arXiv, then affiliated with OpenAlex


r/MachineLearning • • 17h ago

Discussion Is ultra-sparse MoE (200B-A3B style) a bad idea for inference? The Hy team thinks so [D]

5 Upvotes

Pulling this out of the Hy AMA thread for its own discussion, because it argues against something this sub asks for a lot. Someone asked them to push sparsity way further, something like 200B-A3B with engrams. They said no, and explained why:
• overly sparse MoE is inference-unfriendly: it eats GPU memory without using the FLOPs
• they ran scaling ladders on sparsity (they compared it to the MAI-Thinking-1 paper) and found that at fixed active params, more sparsity does improve compute efficiency
• but keeping total params fixed and cutting active params (their example was 156B-A8B or A5B) should hurt quality a lot, because active params are basically your compute budget
Their line was "increasing sparsity recovers some efficiency, but not all."
I mostly buy it for serving on datacenter GPUs. What I'm less sure about is local. A lot of us are already offloading experts to system RAM, where memory is cheap and compute is the bottleneck, so the trade-off kind of flips. They did say on-device models would get a different design, so maybe they agree.
Also a fun detail: they went back and forth between GQA and MLA for a while. MLA has better quality at the same KV cache size but was making spec decoding compute-bound, and DSA is what tipped it toward MLA.
Anyone here have numbers on ultra-sparse models running from RAM vs denser ones at the same total size?


r/MachineLearning • • 1d ago

Project Curated list of resources: Theory, Code and math for Reinforcement Learning for beginners [P]

Post image
16 Upvotes

[Curated list: Beginners]

Personal favorite resources on learning Reinforcement Learning from Scratch:

  1. Deep RL Course Amazing course by HuggingFace to understand the basics of classic and deep RL
  2. DeepMind x UCL Reinforcement Learning (2015) Give intuition and more towards theory of what you'll learn above
  3. Spinning Up in Deep RL Code and math for each of algorithm you'll learn above from classic to Deep RL
  4. CleanRL A neat library with 1 page implementations for all the algorithms above from scratch to run algorithms s and test the waters!
  5. NeatRL My own very notes and from scratch implementations!

Reading order and link here

If you want something to be added/removed or reordered then you can do it here


r/MachineLearning • • 1d ago

Research Can LLM-guided evolution optimize the wrong objective? An empirical study of OpenEvolve, evaluator failures, and independently verified algorithm quality [R]

0 Upvotes

There is an interesting problem at the intersection of LLM-guided program synthesis, evolutionary computation, and automated scientific discovery.

What happens when an AI system successfully discovers a better algorithm, but the fitness function responsible for selecting it is incomplete?

Systems inspired by FunSearch and AlphaEvolve use an iterative process:

  1. An LLM proposes or modifies a program.
  2. An evaluator executes it and assigns a numerical score.
  3. High-scoring candidates influence subsequent generations.
  4. The system eventually selects the best-scoring candidate.

The assumption underlying this process is that improving the evaluator's score corresponds to improving the actual solution.

But that relationship depends entirely on what the evaluator measures.

If a task has requirements that are missing from the fitness function, evolutionary search may discover implementations that optimize the measured objective while violating the original specification.

This raises a broader question: Does increasing the search budget improve the quality of the final solution, or does it sometimes increase the probability of exploiting weaknesses in the evaluation process?

Experimental setup

A recent empirical study investigated this question using OpenEvolve 0.4.0 on two bounded algorithmic tasks.

Circle packing: Place 26 non-overlapping circles inside a unit square while maximizing the sum of their radii.

Approximate k-nearest neighbors: Return approximate nearest neighbors efficiently while satisfying requirements for neighbor indices and their corresponding squared distances.

The experiments used four Gemma 4 configurations: E4B, 12B, 26B A4B MoE, and 31B.

Three search conditions were considered.

Evolutionary search (L): Candidate programs are generated using an evolutionary population and information from previous evaluations.

Best-of-N (B): Independent candidate programs are generated from the original seed without evolutionary history, then ranked by the working evaluator.

Cloud-assisted evolution (LC): The evolutionary process is preserved, but approximately 10% of mutation calls are routed to Gemini 4 Argon rather than the local Gemma model.

Search budgets included 100, 500, and 1,000 LLM calls, depending on the experimental condition.

Crucially, the evaluation process had two separate components:

Working evaluator: The score used during evolutionary search to rank candidates and select a final program.

Independent re-check: A separate post-run evaluation that verifies task requirements and measures the selected program on additional test inputs.

The independent re-check was not used to guide candidate generation or working-score selection.

This separation makes it possible to distinguish the best program discovered during search from the program the system actually selects.

Finding 1: The search discovered valid solutions but selected invalid programs

This is arguably the most interesting observation.

Three evolutionary circle-packing runs were performed with a budget of 1,000 LLM calls each.

The best valid solutions discovered during these runs achieved scores of approximately:

RunBest valid solution foundSelected program's verified score
12.4322.432
22.2040
32.2580

In two of the three runs, the search successfully discovered a valid improvement.

However, the program selected by the working evaluator failed independent verification.

Because the initial program was also invalid, the registered evaluation protocol assigned those failed selections a verified score of zero.

The important distinction is between:

Discovery quality: How good was the best valid candidate found anywhere in the search?

Selection quality: How good was the program the system actually chose?

These are not necessarily the same.

The approximate selection regrets across the three runs were 0.000, 2.204, and 2.258.

The sample is too small to establish a general relationship between search budget and reliability. Nevertheless, it demonstrates a concrete failure mode: increasing the number of generated candidates can produce strong valid solutions without ensuring that the selection mechanism retains them.

Finding 2: Higher fitness scores frequently failed independent verification

The approximate kNN experiments revealed another type of evaluator mismatch.

The working evaluator rewarded a combination of neighbor recall and relative execution speed.

However, it did not fully verify that the returned squared distances were correct for the corresponding neighbor indices.

This allowed some programs to receive favorable scores while violating part of the task's required API contract.

In the cloud-assisted kNN condition:

60 of 213 candidate evaluations (28.2%) ranked above the initial program did not represent independently verified improvements.

Of those 60 evaluations, four involved invalid candidates. The other 56 passed validity checks but achieved a verified composite score no higher than the seed.

These are candidate evaluations, not necessarily unique programs. Repeated evaluations of the same code hash are included.

The broader implication is that an evaluator can reward improvements in the properties it measures while overlooking violations of properties it does not measure.

This matters particularly when the search process is capable of discovering implementation shortcuts that were not anticipated when the evaluator was designed.

Finding 3: Hybrid local/cloud evolution produced promising, but preliminary, results

The study also compared Gemma-only evolution against a hybrid configuration in which approximately 10% of mutation calls used Gemini 4 Argon.

In approximate kNN, the cloud-assisted condition achieved a higher independently verified score in eight of nine matched runs.

The median paired difference was approximately +0.776.

In circle packing, the cloud-assisted condition performed better in seven of nine pairs, with one tie and one lower result.

These observations suggest that introducing a second model into the evolutionary search process may be useful in some settings.

However, this was an exploratory comparison involving small samples, incomplete planned conditions, and additional cloud-model computation.

It does not establish a general causal advantage for hybrid evolution.

More importantly, improved generation capability did not remove the need for independent verification.

Finding 4: Adding a correctness gate did not automatically solve the problem

A separate follow-up experiment investigated whether strengthening the working evaluator would improve the validity of the final selected program.

This experiment used six paired kNN runs with two evaluator configurations.

V1-matched: A timing-matched evaluator without a pre-fitness distance-correctness gate.

V2: The same timing framework, but with a distance-correctness gate applied before fitness evaluation.

Both conditions used the same local Gemma 4 12B model configuration and 20 LLM calls per run.

Independent verification produced the following results:

EvaluatorValid selected programs
V1-matched6/6
V25/6

The median paired score difference (V2 minus V1-matched) was +0.0290, with a range from −0.2241 to +0.7690.

The comparison is sensitive to individual runs and does not establish superiority or equivalence.

The interesting detail is the one invalid V2-selected program.

It achieved perfect neighbor recall on the holdout evaluation, but the independent checker identified 140 failures involving the returned squared distances.

In other words, the program could identify the correct neighbors while still violating the required distance contract.

A correctness gate had been added, but the final independent test still detected a failure.

This illustrates why adding a validation step and achieving complete specification coverage are different problems.

The measurement problem

A related difficulty emerged during kNN performance evaluation.

When several Docker sandboxes were benchmarked concurrently, the identical seed program received scores ranging from approximately 0.02 to 1.0.

Because runtime was part of the fitness function, system load could substantially affect candidate rankings.

The study therefore repeated timing measurements serially, using an otherwise idle machine.

This improved comparability, but measurable variability remained.

This raises an additional challenge for evolutionary program optimization: when performance is noisy, the fitness function can become a source of misleading feedback even if the correctness checks are well designed.

What can actually be concluded?

The results do not demonstrate that OpenEvolve is generally unreliable.

They do not show that evolutionary search is inherently worse than independent sampling.

They do not establish universal scaling relationships between model size, search budget, and algorithm quality.

The experiments cover two relatively small algorithmic tasks, with several exploratory comparisons and documented execution limitations.

What they demonstrate is narrower, but operationally important:

The highest fitness score observed during an evolutionary search is not necessarily a reliable measure of the final program's independently verified quality.

A system can discover a strong valid candidate and still select a weaker or invalid one.

This distinction becomes increasingly relevant as LLM-guided evolutionary systems move from benchmark optimization toward autonomous scientific and engineering work.

Questions for discussion

1. Should program-evolution systems use separate evaluators for search guidance and final candidate selection?

2. How can correctness constraints be expressed in a way that remains robust against unexpected implementation shortcuts?

3. Should algorithm-discovery benchmarks report both best-valid-discovered performance and independently verified selected-program performance?

4. How should noisy performance measurements be incorporated into evolutionary fitness without rewarding measurement artifacts?

5. Could larger search budgets systematically increase exposure to evaluator weaknesses, even when the population contains increasingly strong valid candidates?

The broader issue is not limited to OpenEvolve or any particular LLM.

It concerns the reliability of systems that use executable feedback to automate algorithm discovery.

Finding a high-scoring program is one achievement. Establishing that it is actually a better program is another.


r/MachineLearning • • 1d ago

Research How to answer this AI-review for AAMAS? [D]

0 Upvotes

Hi, so AAMAS reviews out. Out of the 3 reviews in my paper, I got 3 for both proceeding and finding with a 4000+ word review with, like, 3 strengths, 8 weaknesses, and 8 suggestions.

One of the 8 weaknesses was, "Your delta in figure 8 shows 2.34, but your table is showing 2.335. Please fix the inconsistency." - Personally, I don't know how to answer this - or Am I wrong here?

Any help

Here is the pangram of the review


r/MachineLearning • • 1d ago

Project BalatroNET: JAX-Native Reimplementation of Balatro's Mechanics for Reinforcement Learning [P]

Post image
4 Upvotes

Github

I've been working on this for a few months now and finally got to a full recreation of the base game (antes 1-8 any deck or stake, but no endless or challenge runs). The main artifact is the environment: jitted, vmapped implementation that can run in parallel on GPUs. I have also worked on a simple agent and a pygame front end where you can step through a run and observe the agent's choices.

I've done some training on an RTX 2070S, the agent can pass white stake red deck about 50% of the time after 6.6 billion training moves. This approach is faster than other implementations such as balatro-gym, which can only run on a CPU, and in my testing (AMD 9800X3D), hits about 1,700 steps/s (per core) while running random actions. On this 2019 GPU, training runs at 16,000 steps/s, all while holding the environment, observation, and performing the network's forward pass and gradient updates.

I'm now opening this up to the public to share progress and see what insights people are willing to give.


r/MachineLearning • • 1d ago

Discussion Do we have AI Bots submitting the rebuttal as well [D]

9 Upvotes

Reviewed around 5 papers for AAMAS, and on 2 papers I see a rebuttal within less than 10 hours. Pretty weird. The rebuttal is also not clear and does not addressed the concerns and is super hard to read.

Man these AI-Slop folks are indeed discouraging at least from engaging with the community.

PS: Even at ICLR had one paper with a pure AI Slop rebuttal


r/MachineLearning • • 1d ago

News Levent Alpöge claims to have found a proof of the Riemann Hypothesis [N]

Thumbnail alpo.ge
18 Upvotes

r/MachineLearning • • 1d ago

Discussion ICLR: Reviewer inactive during rebuttal [D]

0 Upvotes

One of our reviewers didn’t participate in the rebuttal discussion or acknowledge our responses. The AC also hasn’t left any comments.
We didn’t explicitly tag the AC in our rebuttal either.
Is this normal? Should we assume the reviewer simply ignored the rebuttal, or is the AC expected to take the discussion and our responses into account when making the final decision?
Would appreciate hearing from people familiar with the ICLR reviewing process.


r/MachineLearning • • 1d ago

Discussion Why don't optimizes like Shampoo, SOAP, and Adam measure the actual per sample second moments like the Fischer Information Matrix does? [d]

31 Upvotes

It always felt unintuitive to me that Muon matched the performance of Shampoo despite dropping the actual second moments L and R.

But I then I realized, Shampoo itself isn't actually measuring the true second moments.

For each sample i in the batch, we have a gradient G_i.

And the gradient we use during SGD is:

G = E(G_i)

The factors of Shampoo are:

L = EMA GG^T

R = EMA G^TG

Whereas, the true second moments are:

L = EMA E(G_i G_i^T)

R = EMA E(G_i^T G_i)

We can also write them as:

D_i = G_i - G

L = EMA (GG^T + E(D_i D_i^T))

R = EMA (G^TG + E(D_i^T D_i))

What shampoo is doing is dropping the covariance terms like E(D_i D_i^T) from the factors.

It's an approximation.

And, Muon does the same thing. It takes things one step further:

M = EMA G

L = MM^T

R = M^TM

Shampoo's factors are

L_C = EMA (G - M) (G - M)^T
R_C = EMA (G - M)^T (G - M)

L = MM^T + L_C

R = M^TM - R_C

Shampoo drops the covariance across the samples in the batch.

Muon drops the covariance across the batches in the optimization path.

Adam does the same thing except it's doing it to the diagonal of the Fischer. It drops the per sample variance terms from its second moment.

These approximations are sufficient when the covariance terms are small enough.

But the strange thing is, when I tried using the actual second factors for Shampoo and Adam, they underperformed compared to their approximate versions...

I don't understand this at all.


r/MachineLearning • • 2d ago

Discussion ICLR is deleting AI generated reviews and DR AI reviewers' papers [D]

98 Upvotes

Seems like many people have reviews under their papers deleted (likely because of AI) and people whose co-author used AI to review has their paper rejected.


r/MachineLearning • • 2d ago

Discussion ICLR 2027 Desk Rejection [D]

66 Upvotes

Just got news that our paper got DR just because some co-authors failed to serve as a responsible reviewer. The email doesn’t state who that reviewer is, and there have been no emails warning about that co-author’s negligent reviews sent to me. Just instantly DR my paper. All of my efforts so far have been wasted and it is out of my control :)


r/MachineLearning • • 2d ago

Discussion AAMAS 2027 reviews will be out today [D]

22 Upvotes

Creating this thread for review and score discussion. The rebuttal period is 20-24 November. The decision will be out on 21st December.

Good luck everyone!


r/MachineLearning • • 2d ago

Discussion Reviewer AC discussion period [D]

4 Upvotes

I am serving as a reviewer for ICLR this time and I was wondering if reviewers know the actual dates for the Reviewer AC discussion. The website says it's from today up to December 16. However, an SAC sent a message for a specific paper today saying the discussion is till Nov 25. I hope people atleast engage in the discussion if not the rebuttal.


r/MachineLearning • • 2d ago

Project Looking for Collaborators on an AI Safety Project [P]

0 Upvotes

Hi! I'm a PhD student in the UK working on AI safety and evals, with multiple A* conference papers. I'm currently working on an AI safety project about online monitoring of LLMs and looking for collaborators to help with the coding side of this project. The method and idea are already well-scoped.

Happy to share equal contribution. Do reach out to me in DMs with some information about your background if this is of potential interest!


r/MachineLearning • • 2d ago

Project Zusia: ask about the paper open in Zotero and get proofs back as linked theorem boxes, with drawings [P]

0 Upvotes

Zusia is a free, MIT plugin for Zotero 7 to 10 that I wrote: a chat beside the PDF that reads the whole paper.

The video is a real answer, sped up. Asked whether the proof of Adam's Theorem 4.1 is correct, Opus 5.5 finds the step that fails between pp. 14 and 15 (the telescoping sum assumes √v̂_t/α_t never decreases), states that assumption as a lemma, and gives a counterexample to it. It also says what this does not show: the theorem itself is a separate question, the one Reddi et al. settled in 2018, and it marks that part as coming from outside the paper.

WHAT IT DOES

• LaTeX made for the model: KaTeX rendering, definitions, lemmas and theorems as numbered boxes, every \ref a link with Back.

• Drawings in the same answer as the maths: diagrams, and plots like the softmax gradient against the score gap.

• Images: attach the page you're on or a screenshot to ask about a figure or a table.

• Page-level reading: the full text split by page with the printed page labels, the outline, and where each equation, figure and table is. Ask about "page 18" and it reads the page printed 18. Answers cite pages as (p. N).

• A per-paper memory of what you understood and what is still unclear.

It runs on what you already have: your Claude Code, Codex or Antigravity subscription, any ACP agent, your own API key, or a local model through Ollama. No server of mine in between.

https://github.com/firekern/zusia


r/MachineLearning • • 2d ago

Discussion What if we created a Chinese-language AI conference? [D]

0 Upvotes

China has already caught up with, or even surpassed, the English-based AI research. Chinese researchers also account for a significant share of the global AI research community.

If that's the case, wouldn't establishing a Chinese-language AI conference and creating an exclusive pool of Chinese-speaking researchers help further consolidate our hegemony in the field?

What if Chinese researchers collectively decided to exclusively submit our papers only to Chinese-language AI conferences?

This would also free Chinese researchers from the AI slop and low-quality peer reviews that have swallowed up mainstream ML conferences such as ICLR, NeurIPS, ICML.


r/MachineLearning • • 2d ago

Discussion ARR Oct '26 Reviewer Discussion [D]

5 Upvotes

With the reviewing period of this ARR cycle concluding, I wanted to see how others felt about reviewing this cycle.

Also, did you notice any difference in assigned submissions, potentially due to the sustainable reviewing pilot?


r/MachineLearning • • 2d ago

Discussion Can we still see the update of score in Openreview? ICLR 2027 [D]

0 Upvotes

For instance, if the reviewer or AC changes the score for some reason during their discussion, can we see the scores are changing in Openreview?

So frustrating!


r/MachineLearning • • 2d ago

Research Is ICLR a scam or worst organizers? [R]

0 Upvotes

This is quite disappointing not receiving any response from the reviewer even after multiple reminders. It seems like organizers failed to make the conversation interactive or they didn't warn reviewers properly that their paper would desk reject if not actively participate in discussion/rebuttal.