Comparison The battle of the medium efforts in pure reasoning: Sol 6.2 vs Opus 5.5 vs Gemini 4 Neon
I tested GPT-6.2 Sol, Claude Opus 5.5, and Gemini 4 Neon, all at medium reasoning effort, using three problems from AtCoder Grand Contest 078 (AGC078). Why these problems? Because the agents spent 99% of the tokens purely for reasoning. I want to verify the most intelligent models.
Why Medium? Because I wanted to see whether we can get the best price/performance in term of reasoning with medium.
Each problem is worth 1,500 points, is challenging for humans to solve, even experienced competitive programmer. The comparison focuses on five measurements: solution time, code size, execution time, memory consumption, and correctness.
Here are the results.
Problem A: Rearrange ABC
| Model | Solve time | Code size | Run time | Memory |
|---|---|---|---|---|
| GPT-6.2 Sol | 2m 44s | 6,532 B | 323 ms | 4,904 KiB |
| Claude Opus 5.5 | 6m 18s | 3,370 B | 1,574 ms | 4,660 KiB |
| Gemini 4 Neon | 14m 30s | 6,607 B | 802 ms | 12,072 KiB |
Observation: Sol completed the problem fastest, approximately 2.3× faster than Opus and 5.3× faster than Gemini. It also produced the fastest-running solution, while Opus generated the smallest code.
Problem B: L Robust IS
| Model | Solve time | Code size | Run time | Memory |
|---|---|---|---|---|
| GPT-6.2 Sol | 10m 34s | 5,372 B | 460 ms | 31,212 KiB |
| Claude Opus 5.5 | 8m 4s | 2,180 B | 441 ms | 69,444 KiB |
| Gemini 4 Neon | 21m 21s | 6,657 B | 329 ms | 37,212 KiB |
Observation: Opus won on solution time and code size. Gemini produced the fastest-running code, despite taking the longest to solve the problem. Sol used the least memory.
Problem C: AB vs. BA
| Model | Solve time | Code size | Run time | Memory |
|---|---|---|---|---|
| GPT-6.2 Sol | 6m 50s | 2,219 B | 13 ms | 2,636 KiB |
| Claude Opus 5.5 | 36m 46s | 7,981 B | Wrong Answer | — |
| Gemini 4 Neon | 46m 00s | 4,710 B | 17 ms | 3,624 KiB |
Observation: Sol dominated this problem, solving it approximately 6.7× faster than Gemini and 5.4× faster than Opus. It also generated the shortest and most resource-efficient solution. Opus returned a Wrong Answer after nearly 37 minutes.
Conclusion
Across these three problems, GPT-6.2 Sol delivered the strongest overall results, winning on solution time in two out of three problems and producing particularly impressive results on Problem C.
Opus 5.5 showed its strengths in concise code generation and won Problem B on solution time, but Wrong Answer on Problem C was a huge blow. Still, this should be resolved at higher reasoning.
Gemini 4 Neon produced competitive execution times, even achieving the fastest runtime on Problem B, but consistently required the longest time to reach a solution.
A model that takes longer and consume more tokens to solve a problem does not necessarily produce better code—and more generated code does not necessarily mean better reasoning. Anyway, from the result, if you have problems with models like Sol 6.2 for operating in your projects, one way is to limit it to purely problem solving as much as you can, by delegating repo investigation to other models, e.g., Terra or Luna, and Sol only reads the report to start solving the problems.
EDIT: Here is the prompt:
<copy and paste the problem statement here>
Solve this problem. No internet. Work in this dir C:\Temp\<model>\<effort>\<promblem_id> and only this dir, do not look around. Save the solution file there.
Lang: C++.
Save a timestamp before working on, and save a timestamp after your save the solution file, then calculate the total solve time.
Record this information at the top of the comment, along with: "<model> medium".