r/mlscaling • • 46m ago

ZipGate v2.2 Spatial Reasoning Benchmark Pilot Results - Can your model pass the 5-task challenge?

Thumbnail
gallery
1 Upvotes

Title: IKIN ZipGate v2.2 Pilot Benchmark: Chain-of-Thought vs. Zero-Shot Performance on Spatial Reasoning
Body:
I just wrapped up the pilot evaluation of ZipGate v2.2, a new benchmark designed specifically to test complex spatial reasoning on 2D triangular lattices.
We tested 5 frontier models across 4 tiers of difficulty:
Bullet Tier 1: Axial Kinematics
Bullet Tier 2: Internal State Tracking
Bullet Tier 3: Gate Arbitration (Rule following)
Bullet Tier 4: Long-Horizon Rollout
Key Takeaway: Explicit Chain-of-Thought (CoT) or extended thinking modes are mandatory for high performance here. While CoT models (Claude 3.5, o1, Gemini 1.5 Pro) cleared the 5-task pilot set with 100% accuracy, standard zero-shot models struggled significantly with Tier 3 (rule tie-breakers) and Tier 4 (multi-step rollout drift).
Full scorecard attached. The 5-task pilot dataset is available on GitHub if you want to test your own models.
🔗 GitHub: https://github.com/jadeikin-ZIPGATE/ikin-zipgate-benchmark


r/mlscaling • • 9h ago

N, Hardware, NV, Econ "Nvidia Said to Face SEC Probe Into Its Accounting Practices: Questions around the company’s accounting had surfaced amid revelations about its questionable chip sales in Asia", NYT

Thumbnail
nytimes.com
4 Upvotes

r/mlscaling • • 4h ago

Kapso: Evolving harness by learning from experimentation

Thumbnail reddit.com
1 Upvotes

r/mlscaling • • 8h ago

N, Hardware, NV, Econ "Alice Huang’s links to the chip underworld: Chinese executive in romantic relationship with former Nvidia CEO Jensen Huang had connections to the lucrative trade moving chips to China", FT

Thumbnail
ft.com
0 Upvotes

r/mlscaling • • 9h ago

N, Hardware, NV, Econ "The Alice Huang Associate [Li Ming] Accused of Funneling Nvidia Chips to China: Defendant in case involving chip smuggling worked with Jensen Huang’s romantic partner" (Megaspeed/Century Linktone)

Thumbnail wsj.com
1 Upvotes

r/mlscaling • • 10h ago

N, Hardware, NV, Econ "Treasury secretary Bessent says no further restrictions are needed for Nvidia chip exports to China"

Thumbnail
apnews.com
1 Upvotes

r/mlscaling • • 10h ago

OP, D, Econ, RL "Trump on superintelligence: “It may be too super.” What is China saying?"

Thumbnail
lesswrong.com
0 Upvotes

r/mlscaling • • 2d ago

OA, Emp, M-L Research acceleration: The jagged frontier [OpenAI November 2026 update]

Thumbnail
openai.com
9 Upvotes

r/mlscaling • • 2d ago

N, NV, Hardware, Econ "China’s Moonshot AI Scraps $150 Million Deal to Access Nvidia Chips in Indonesia"

Thumbnail theinformation.com
8 Upvotes

r/mlscaling • • 2d ago

N, NV, Hardware, Econ "Nvidia Stops Selling [H200] AI Chips to China Despite US Approval"

Thumbnail
bloomberg.com
25 Upvotes

r/mlscaling • • 2d ago

N, Hardware, Econ "TSMC tightens China AI chip curbs as new US rules close manufacturing loophole, sources say"

Thumbnail reuters.com
2 Upvotes

r/mlscaling • • 3d ago

N, A, Econ "Anthropic IPO raises record $81.3 billion on strong AI demand"

Thumbnail
npr.org
10 Upvotes

r/mlscaling • • 3d ago

Emp, M-L Singularity-Bench: Can Agents Build Agents? [Benchmarking full-stack AI agent development: pre + mid + post-training + inference engine + multi-GPU deployment + harness]

Thumbnail
infini-ai-lab.github.io
1 Upvotes

r/mlscaling • • 3d ago

N, NV, Econ, Hardware "Inside Tencent’s deal to use Nvidia’s best AI chips in Japan: Contracts with Chinese tech giant have rapidly turned Datasection into one of Asia’s biggest ‘neoclouds’", FT

Thumbnail
ft.com
4 Upvotes

r/mlscaling • • 3d ago

OP, Econ, Hardware "The end of startups: the coming credit crunch" due to compute shortages, Evan Conrad 2026-10-22 (SF Compute)

Thumbnail evanjconrad.com
4 Upvotes

r/mlscaling • • 2d ago

A, Econ, D Claude max for $29 instead of $200 ... seriously how are they doing this?

0 Upvotes

i got a Claude max account from a reseller for essentially nothing compared to what I used to pay . It works flawless. since 3 months No issues yet i paid only for 1 month . Now I can't stop thinking about their business model. If resellers can sell it this cheap and still profit, how are they acquiring these accounts at scale?


r/mlscaling • • 3d ago

N, NV, Hardware, Econ "Nvidia Introduced Alibaba to Alice Huang for Workaround on US Export Controls", TI

Thumbnail
theinformation.com
2 Upvotes

r/mlscaling • • 3d ago

N, OA GPT-6.2 Astra & Sol

Thumbnail openai.com
1 Upvotes

r/mlscaling • • 3d ago

R, T, A, RL, Econ, Code "Building agents: When to fold subagents into your main agent loop", Anthropic ("a single agent with skills frequently outperformed both the one-prompt-for-everything design and the subagent design on quality, and often at a lower cost and latency per task")

Thumbnail
claude.com
1 Upvotes

r/mlscaling • • 3d ago

N, NV, Hardware, Econ "Trump Administration Moves to Close AI Chip Loophole Ahead of China Trip: The rules have been a topic of fierce debate in the administration and would cut back on a booming data center business serving Chinese companies"

Thumbnail
nytimes.com
2 Upvotes

r/mlscaling • • 3d ago

N, NV, Econ "Wall Street Pledged to Mobilize $500 Billion for Nvidia—But Hasn’t Put Up a Dime: A wave of money that was supposed to help customers keep buying Nvidia chips hasn’t materialized", WSJ

Thumbnail wsj.com
1 Upvotes

r/mlscaling • • 3d ago

N, NV, Hardware, Econ "How Nvidia Chips Ended Up Within Reach of China’s Military: Alibaba, the Chinese internet giant behind the company at the center of the Nvidia spy saga, has used its AI computing technology to aid a lab with links to China’s military" (NV→Megaspeed→Alibaba (Zhejiang Lab)→PLA)

Thumbnail
nytimes.com
0 Upvotes

r/mlscaling • • 3d ago

N, NV, Hardware, Econ "NVIDIA Board of Directors Establishes National Security and Export Control Oversight Committee", Nvidia 2026-11-18

Thumbnail
nvidianews.nvidia.com
1 Upvotes

r/mlscaling • • 3d ago

Prompt Engineering and LLM-as-a-Judge (2026)

Thumbnail
youtube.com
0 Upvotes