r/backgammon • • 15h ago

I built a new backgammon engine, Vulcan. I believe it's the best engine in the world. Details inside, and you can play and analyze with it for free.

0 Upvotes

Hi all,

I've built a backgammon engine from scratch, programmed in Rust, Vulcan, with its own neural networks. I believe it's now the strongest backgammon engine in the world, and I'd like to make that case as simply as I can: two results, then the site where you can use it for free, then all the detail for anyone who wants to check the work.

1. The strongest network we could find to test against

Take away all search and let each engine play with its neural network alone. In cubeful money games, Vulcan's network beats every other engine's network we could get our hands on, by a wide margin:

  • vs XG 2.10's network: +0.058 points per game
  • vs GNU Backgammon 0-ply: +0.084
  • vs HedgeHog FOX: +0.058
  • vs HedgeHog Aureus: +0.037

(20,000 game pairs each; every 95% interval is about ±0.003, so none of these is close to level.)

And Colossus 2.0? It isn't publicly playable, so nobody outside HedgeHog can play it. But HedgeHog published how Colossus 2.0's network scores against these same four opponents, with the same settings, and Vulcan's margin is bigger against every one of them. Against Aureus, HedgeHog's own second-best network, Vulcan wins by +0.037 and Colossus 2.0 by +0.021: Vulcan beats HedgeHog's No. 2 by almost twice as much as HedgeHog's No. 1 does.

HedgeHog also published 25,000 games of Colossus 2.0 against XG. On the 74,639 decisions in those games where Vulcan, looking as far ahead as Colossus did, would have played differently, rollouts favour Vulcan's play (+0.0004 per disagreement; Vulcan's play was clearly better in 17,160 decisions, Colossus's in 15,323). Those rollouts were played by Vulcan's own networks, so treat this as supporting evidence rather than proof.

2. Vulcan's 2-ply beats XG's 4-ply, even though XG looks one ply further ahead

Plies count differently between programs: Vulcan's 2-ply looks as far ahead as XG's 3-ply. XG's 4-ply, the setting strong players use to check their games, searches one ply deeper than Vulcan's 2-ply. I played them against each other anyway, with full cube decisions on both sides and XG using its opening book:

  • Money games: Vulcan +0.0049 points per game over 200,000 game pairs (luck-adjusted, 95% interval +0.0038 to +0.0059; the raw result is level).
  • 7-point matches: Vulcan wins 50.27% of 80,000 matches (interval 50.22-50.32%).
  • Vulcan thinks a quarter to a third as long per move as XG 4-ply.

At equal lookahead (Vulcan 2-ply vs XG 3-ply) the gap is much wider: +0.0125 points per game.

That's the case: a better network than any engine I could test, more than Colossus 2.0's published margins against the same opponents, and a 2-ply that holds its own against XG one ply deeper. Every run was planned and written down before it started, and raw results are reported beside the adjusted ones (details below).

Use it free on Gammon Forge

Vulcan runs everything on gammonforge.com, and all of it is free: no ads, no subscription, one Google sign-in (you can play one game without an account).

  • Play Vulcan at five levels, from Beginner (about 1100) to Full strength (about 1950; the ratings are rough estimates), in single games, matches or money sessions with the cube. Climb a ladder of 5-point matches, or play rated games against people, live or by correspondence.
  • Analyse any position. Paste an XGID or GNU BG ID and Vulcan ranks every play, going deeper while you watch. Close calls are settled with free rollouts. Here's a 1-3 backgame where the top four plays are within 0.01 of each other at 3-ply; a quick rollout keeps 8/4 on top and drops the hit 13/12*/9 to an inaccuracy:
  • Every game you finish is reviewed: PR for both players, every mistake with the better play drawn on the board, a win-chance graph and the luck of each roll. You can also upload .mat files from XG or GNU BG.
  • 84,233 puzzles from Vulcan's games against itself, with a puzzle rating, a daily puzzle, streaks and sprints, plus lessons on the rules, the opening moves and the cube.

Verification

I've done my best to validate and second-guess myself before making this post. The project has been a year in the making. There's detailed results showing engine-vs-engine analysis that I conducted personally, so there's an element of trust here. I would be happy to work with someone who wants to set up a third party engine evaluation system, as I think that would be the best way to prove the claims I am making. Also, I encourage strong players (not me!) to test it out, share feedback, critique flaws you find. I'd love positions where you think Vulcan plays badly, and questions about the testing.

More detail, for those interested

Every headline run

Vulcan 2-ply vsFormatGamesLuck-adjusted [95% CI]Raw [95% CI]
XG 2.10, 4-ply checker + cube, XG's opening bookMoney200,000 pairs+0.0049 [+0.0038, +0.0059] ppg-0.0039 [-0.0112, +0.0033]
XG 2.10, 4-ply checker + cube7-point matches80,000 matches50.27% [50.22, 50.32]50.21% [49.90, 50.52]
GNU Backgammon 1.08.003, 2-ply checker + cubeMoney100,000 pairs+0.0214 [+0.0197, +0.0230] ppg+0.0150 [+0.0040, +0.0261]
GNU Backgammon 1.08.003, 2-plyCubeless200,000 pairs+0.0119 [+0.0114, +0.0124]+0.0099 [+0.0065, +0.0133]

Every equal-lookahead run, with Colossus 2.0's published cells

Cubeful money, HedgeHog's benchmark settings (Jacoby and beavers off, no opening books).

Vulcan depthOpponent (same lookahead)PairsVulcan, ppg [95% CI]Colossus 2.0, same opponent (published)
0-plyGNU BG 0-ply20,000+0.0837 [+0.0811, +0.0864]+0.0708 [+0.0689, +0.0728]
0-plyHedgeHog FOX20,000+0.0580 [+0.0554, +0.0606]+0.0388 [+0.0368, +0.0407]
0-plyXG 2.10 1-ply20,000+0.0577 [+0.0550, +0.0603]+0.0499 [+0.0479, +0.0518]
0-plyHedgeHog Aureus20,000+0.0371 [+0.0346, +0.0396]+0.0209 [+0.0190, +0.0229]
1-plyXG 2.10 2-ply20,000+0.0360 [+0.0336, +0.0385]+0.0327 [+0.0232, +0.0422] (level with Vulcan's)
1-plyGNU BG 1-ply20,000+0.0506 [+0.0482, +0.0529]not published
2-plyXG 2.10 3-ply4,000+0.0125 [+0.0082, +0.0169]not published
2-plyGNU BG 2-ply3,000+0.0292 [+0.0131, +0.0452]not published

Depth names: Vulcan's 0-ply = GNU BG 0-ply = XG / HedgeHog "1-ply" (the network alone); Vulcan's 1-ply = XG 2-ply; Vulcan's 2-ply = XG 3-ply. Colossus's cells are HedgeHog's raw results over 10M games per pair (its XG 2-ply cell is against real XG 2.10 on HedgeHog's own harness). Vulcan's are luck-adjusted with Vulcan's networks as the reference; on the GNU BG row, GNU BG's networks give the same answer (+0.0841 vs +0.0837).

5-point matches with public dice. One run took its dice from a public randomness beacon (drand) whose round I named in a public gist before that round existed. Vulcan 1-ply vs XG 2.10 2-ply, 10,000 5-point matches: 51.29% match wins [51.15, 51.43] (raw 50.81%). HedgeHog's published Colossus 2.0 vs XG 2.10 in the same format: 50.68% [50.44, 50.93]. Every match file replays and its dice can be checked.

Colossus 2.0's own decisions. Rollouts of the decisions in Colossus's 25,000 published games vs XG where Vulcan would play differently: at equal lookahead (Vulcan 1-ply, an earlier network from Nov 1), +0.0004 [+0.0002, +0.0006] per disagreement over 74,639 decisions; with Vulcan's 2-ply (one ply deeper than Colossus played), +0.0051 per disagreement over 70,091. Rollouts: cubeful, 1-ply policy, played by Vulcan's own networks.

Speed

Vulcan's ply costs more than XG's or GNU BG's: in cube-aware play it runs its full network on every reply it considers, where they screen replies with a small network first. Its 2-ply still thinks a quarter to a third as long as XG's 4-ply. The neural nets behind Vulcan are larger than the networks in XG, with numerical optimizations applied that compress the network for speed, while retaining as much intelligence as possible.

How it was measured

  • Paired games. Each pair of games uses one dice sequence twice with the seats swapped.
  • Luck adjustment. Before each roll a fixed reference network values all 21 rolls; the roll's luck is its value minus the average. Subtracting it can't shift the expected result (the average luck is zero by construction); it removes noise, about 7x less spread per pair here, so 200,000 pairs measure what would otherwise take millions. In the headline runs and the 5-point run the reference is GNU Backgammon's networks, not Vulcan's. Raw results are shown beside every adjusted one.
  • Pre-registered. Sample size, settings and the decision rule were written down before each run; runs weren't stopped early, and the one extension (a second 100,000-pair money run against XG 4-ply on fresh dice) was registered as its own run and both are reported.
  • The XG opponent. Engine-vs-engine play needs an automated XG, so these runs use a native port of XG 2.10's engine: its networks, move filters, cube logic and opening book. To check it, real XG 2.10 analysed the port's games: at 2-ply it flagged 0 of 14,233 decision records in 100 matches, and at 4-ply 0 of 7,384 in 40 matches from the 7-point run above. XG agrees with every move and cube decision the port made.
  • GNU BG is the real program (1.08.003), with Jacoby off and cubeless checker play for the cubeless run (its defaults undervalue gammons there).

Looking forward

A bigger research version of Vulcan's network (too slow to serve today) beats the current one by a further +0.012 points per game head to head. So, I'm confident that there is more room for Vulcan (or others) to improve further.

If you've made it this far, thank you! Please give it a spin and let me know what you think.

-Caleb Wetherell


r/backgammon • • 14h ago

My free browser backgammon game now shows you the best move, step by step

Post image
0 Upvotes

Turn on Show Correct Move when you play the computer, and a "Recommended turn" panel (GNU Backgammon 2-ply) draws the best move as numbered arrows on the board, then you tap Continue game. Free, no email, just pick a player name. No clutter, no gimmicks, easy for beginners. Completely FREE. No frills, pure enjoyment. Beginners learn by seeing, and experienced players can check their own plays. Feedback welcome from players at every level. https://thegammonlounge.com/


r/backgammon • • 3h ago

I made a free backgammon (nard) game from Azerbaijan — looking for Android testers

Thumbnail
gallery
0 Upvotes

Hi r/backgammon! I've been building NərdBaz, a free nard/backgammon game in the style of an Azerbaijani tea house: a 3D walnut board, four computer levels (the top one is quite strong), online games with friends through a link, and a tutorial inside the game. No ads, no real money.

It already runs in the browser (nardbaz.az). The Android app is in Google Play's closed test, which needs real players for 14 days. If you'd like to help:

  1. Join the tester group: https://groups.google.com/g/nardbaz-testers
  2. Become a tester and install: https://play.google.com/apps/testing/az.nardbaz.app

Play a few games over the next two weeks and tell me what you think — how the strongest level plays, and anything that feels off in the rules or the controls. Comments here or on the Play test page are very welcome. Thanks!


r/backgammon • • 16h ago

Precision dice for tournament use

4 Upvotes

Hello! I have a question for anyone who's played in tournaments. I'm finally getting around to buying a good board so I can play in clubs and tournaments, but it didn't come with precision dice. Are clubs and tournaments usually picky about particular brands? As in, will I probably be fine if I get the cheap Bescor precision dice from Amazon, or should I spring for the more expensive ones from Backgammon Galaxy or Ace Point? Any other tips for first-timers at a tournament would be great too!


r/backgammon • • 8h ago

New free web based FIBS client

3 Upvotes

I have been working a new web based FIBS client that I hope will work well on both mobile and desktop devices. Give it a try and let me know what you think.

https://pocketfibs.fly.dev/


r/backgammon • • 4h ago

I made a free backgammon (nard) game from Azerbaijan — looking for Android testers

0 Upvotes

Hi r/backgammon! I've been building NərdBaz, a free nard/backgammon game in the style of an Azerbaijani tea house: a 3D walnut board, four computer levels (the top one is quite strong), online games with friends through a link, and a tutorial inside the game. No ads, no real money.

It already runs in the browser (nardbaz.az). The Android app is in Google Play's closed test, which needs real players for 14 days. If you'd like to help:

  1. Join the tester group: https://groups.google.com/g/nardbaz-testers
  2. Become a tester and install: https://play.google.com/apps/testing/az.nardbaz.app

Play a few games over the next two weeks and tell me what you think — how the strongest level plays, and anything that feels off in the rules or the controls. Comments here or on the Play test page are very welcome. Thanks!