r/mlscaling • u/gwern • 10h ago
N, Hardware, NV, Econ "Nvidia Said to Face SEC Probe Into Its Accounting Practices: Questions around the company’s accounting had surfaced amid revelations about its questionable chip sales in Asia", NYT
r/mlscaling • u/gwern • 10h ago
r/mlscaling • u/PotentialFirm7721 • 2h ago
Title: IKIN ZipGate v2.2 Pilot Benchmark: Chain-of-Thought vs. Zero-Shot Performance on Spatial Reasoning
Body:
I just wrapped up the pilot evaluation of ZipGate v2.2, a new benchmark designed specifically to test complex spatial reasoning on 2D triangular lattices.
We tested 5 frontier models across 4 tiers of difficulty:
Bullet Tier 1: Axial Kinematics
Bullet Tier 2: Internal State Tracking
Bullet Tier 3: Gate Arbitration (Rule following)
Bullet Tier 4: Long-Horizon Rollout
Key Takeaway: Explicit Chain-of-Thought (CoT) or extended thinking modes are mandatory for high performance here. While CoT models (Claude 3.5, o1, Gemini 1.5 Pro) cleared the 5-task pilot set with 100% accuracy, standard zero-shot models struggled significantly with Tier 3 (rule tie-breakers) and Tier 4 (multi-step rollout drift).
Full scorecard attached. The 5-task pilot dataset is available on GitHub if you want to test your own models.
🔗 GitHub: https://github.com/jadeikin-ZIPGATE/ikin-zipgate-benchmark
r/mlscaling • u/alirezamsh • 5h ago
r/mlscaling • u/gwern • 10h ago
r/mlscaling • u/gwern • 10h ago
r/mlscaling • u/gwern • 11h ago
r/mlscaling • u/gwern • 12h ago
r/mlscaling • u/StartledWatermelon • 2d ago
r/mlscaling • u/gwern • 2d ago
r/mlscaling • u/gwern • 2d ago
r/mlscaling • u/gwern • 2d ago
r/mlscaling • u/gwern • 3d ago
r/mlscaling • u/StartledWatermelon • 3d ago
r/mlscaling • u/gwern • 3d ago
r/mlscaling • u/gwern • 3d ago
r/mlscaling • u/mehdiweb • 2d ago
i got a Claude max account from a reseller for essentially nothing compared to what I used to pay . It works flawless. since 3 months No issues yet i paid only for 1 month . Now I can't stop thinking about their business model. If resellers can sell it this cheap and still profit, how are they acquiring these accounts at scale?
r/mlscaling • u/gwern • 3d ago
r/mlscaling • u/gwern • 3d ago
r/mlscaling • u/gwern • 3d ago
r/mlscaling • u/gwern • 3d ago
r/mlscaling • u/gwern • 3d ago
r/mlscaling • u/gwern • 3d ago
r/mlscaling • u/Neurosymbolic • 4d ago