Leaderboard — Elo & head-to-head win ratios
Every policy plays every other in a round robin; Elo starts at 1500, K=32. The matrix shows each row method's win rate against each column method.
Elo ratings
| Method | Elo |
|---|---|
| V4 Ordered List (Play coins first) | 1669.7 |
| AlphaStar (under-trained, 140g) | 1626.9 |
| V2 Ordered List | 1591.7 |
| AlphaStar (well-trained, 500g) | 1528.0 |
| V1 Choose randomly, excluding end turn | 1523.3 |
| V3 Ordered List (cautious scraping) | 1333.0 |
| V0 Choose completely randomly | 1227.3 |
Win-ratio matrix (row vs. column)
| V0 Choose completely randomly | V1 Choose randomly, excluding end turn | V2 Ordered List | V3 Ordered List (cautious scraping) | V4 Ordered List (Play coins first) | AlphaStar (under-trained, 140g) | AlphaStar (well-trained, 500g) | |
|---|---|---|---|---|---|---|---|
| V0 Choose completely randomly | — | 0% | 0% | 0% | 0% | 0% | 17% |
| V1 Choose randomly, excluding end turn | 100% | — | 33% | 100% | 33% | 50% | 33% |
| V2 Ordered List | 100% | 67% | — | 100% | 33% | 33% | 67% |
| V3 Ordered List (cautious scraping) | 100% | 0% | 0% | — | 17% | 17% | 0% |
| V4 Ordered List (Play coins first) | 100% | 67% | 67% | 83% | — | 83% | 33% |
| AlphaStar (under-trained, 140g) | 100% | 50% | 67% | 83% | 17% | — | 67% |
| AlphaStar (well-trained, 500g) | 83% | 67% | 33% | 100% | 67% | 33% | — |
The two AlphaStar entries are the same architecture at two training depths, included to show Elo rising with training. See the policies page for what each method is.
How the fully-trained agents' games end
A game can end three ways: decisively (a player's authority is driven to 0), by the engine's 50-turn cap (whoever has more authority wins), or by our 200-action move cap (a safety bound we impose so a single game can't run away). We halved the move cap from 400 to 200; the counts below — from 30 games of the well-trained models head-to-head at each authority — check we did not overdo it.
| Starting authority | Decisive (lethal) | Turn cap (50 turns) | Move cap (200 actions) |
|---|---|---|---|
| 10 authority | 30 (100%) | 0 (0%) | 0 (0%) |
| 50 authority | 1 (3%) | 25 (83%) | 4 (13%) |
4 game(s) hit the 200-action move cap — the cap is ending games that had not yet resolved, so it may be set too low. See the engineering challenge for why high-authority games are long and expensive.