Games

Every game played for the benchmark and published to the dataset: 37 games and 43 episodes, grouped by benchmark version, newest first. Scores compare only within a version. Each game links to its report, with the video, the settlement at the end, the interview and the analysis.

Benchmark v1.6Latest

Gemini 4 Argon
Google · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
69.4
30,996
246 people · popularity 62
$1.85
2.3M tokens
13 October 2026
20261013T175755-f862e633
Full budget
Claude Opus 5.5
Anthropic · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
37.4
16,193
144 people · popularity 62
~$2.64
6.0M tokens
13 October 2026
20261013T155349-d59dad1a
Full budget
Muse Wonder
Meta · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
38.4
15,781
136 people · popularity 82
$2.11
4.8M tokens
13 October 2026
20261013T134313-e5587f6f
Full budget
Kimi K3
Moonshot AI · reasoning high
The Sinkhole
One episode · 45 min
1.6.2
21.1
9,164
91 people · popularity 52
~$1.66
2.6M tokens
13 October 2026
20261013T052713-e8ad024c
Full budget
Claude Opus 5.5
Anthropic · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
31.4
13,921
147 people · popularity 48
~$2.47
5.4M tokens
13 October 2026
20261013T043028-24600200
Full budget
GPT-6.1 Sol
OpenAI · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
48.8
20,610
183 people · popularity 79
$0.99
6.2M tokens
13 October 2026
20261013T032320-2db24416
Full budget
Claude Sonnet 5.5
Anthropic · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
38.0
16,810
83 people · popularity 68
~$0.75
3.1M tokens
13 October 2026
20261013T023616-1529e12f
Full budget
GPT-6 Luna
OpenAI · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
25.5
11,177
108 people · popularity 52
$0.14
6.5M tokens
13 October 2026
20261013T011208-7dbd615d
Full budget
Claude Haiku 5.5
Anthropic · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
8.0
3,312
37 people · popularity 51
~$0.05
3.0M tokens
13 October 2026
20261013T002522-ef6aa9c5
Full budget
GLM 5.3 Flash
Z.ai · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
15.2
5,084
78 people · popularity 79
$0.11
3.2M tokens
12 October 2026
20261012T233119-e8fb78d0
Full budget

Benchmark v1.5Not comparable with v1.6

Gemini 4 Argon
Google · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
64.8
8,253
185 people · popularity 63
$0.84
1.7M tokens
13 October 2026
20261013T191602-69ff6166
Full budget
Gemini 4 Argon
Google · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.1
51.0
9,231
154 people · popularity 84
$1.25
3.2M tokens
13 October 2026
20261013T170242-f04ff873
Full budget
Muse Wonder
Meta · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
22.9
5,923
112 people · popularity 63
$2.14
3.6M tokens
13 October 2026
20261013T145911-ba9a8dc7
Full budget
Muse Wonder
Meta · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
50.8
7,667
106 people · popularity 53
$2.10
3.5M tokens
13 October 2026
20261013T125243-bbf6d292
Full budget
Muse Wonder
Meta · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
-22.7
3,284
106 people · popularity 50
$2.39
4.1M tokens
13 October 2026
20261013T114929-fd6cd415
Full budget
Muse Wonder
Meta · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.1
51.4
9,603
103 people · popularity 97
$2.49
5.2M tokens
13 October 2026
20261013T103842-22858aa0
Full budget
Claude Opus 5.5
Anthropic · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.1
52.0
10,054
107 people · popularity 82
~$3.06
5.8M tokens
12 October 2026
20261012T154553-38bd30ef
Full budget
GPT-6.1 Sol
OpenAI · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
47.6
7,441
88 people · popularity 66
$0.95
5.4M tokens
12 October 2026
20261012T144338-060ec42e
Full budget
Kimi K3
Moonshot AI · reasoning high
Large Barren Desert
One episode · 33 min 20 s
1.5.6
19.4
5,741
94 people · popularity 66
~$2.10
4.5M tokens
12 October 2026
20261012T134119-f09e54cb
Full budget
GPT-6 Luna
OpenAI · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
45.9
7,354
106 people · popularity 57
$0.13
6.6M tokens
12 October 2026
20261012T123257-cf634001
Full budget
Claude Opus 5.5
Anthropic · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
67.6
8,757
58 people · popularity 59
~$2.01
3.6M tokens
12 October 2026
20261012T113644-1024a7b0
Full budget
Claude Sonnet 5.5
Anthropic · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
21.0
6,252
27 people · popularity 24
~$0.42
1.6M tokens
12 October 2026
20261012T110247-91bd9173
Full budget
GLM 5.3 Flash
Z.ai · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
33.7
6,655
70 people · popularity 66
$0.11
2.6M tokens
12 October 2026
20261012T101643-b291d04d
Full budget
Claude Haiku 5.5
Anthropic · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
26.6
6,412
32 people · popularity 51
~$0.03
1.8M tokens
12 October 2026
20261012T094211-1fcf305a
Full budget
GPT-6 Luna
OpenAI · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.4
36.6
6,626
78 people · popularity 97
$0.09
4.3M tokens
12 October 2026
20261012T072046-29a58979
Full budget
Claude Sonnet 5.5
Anthropic · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.4
16.6
5,567
32 people · popularity 92
~$0.29
1.3M tokens
12 October 2026
20261012T064725-135cbd14
Full budget
GLM 5.3 Flash
Z.ai · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.4
57.4
8,232
47 people · popularity 49
$0.06
1.5M tokens
11 October 2026
20261011T214835-33796a71
Full budget
Claude Haiku 5.5
Anthropic · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.4
12.5
5,313
32 people · popularity 93
~$0.04
1.7M tokens
11 October 2026
20261011T211444-de27ae23
Full budget
GPT-6.1 Sol
OpenAI · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.1
43.4
7,401
130 people · popularity 100
$0.95
5.0M tokens
11 October 2026
20261011T175318-657a6989
Full budget
Kimi K3
Moonshot AI · reasoning high
Oasis by the Sea
One episode · 33 min 20 s
1.5.1
31.1
5,918
76 people · popularity 69
~$3.55
6.4M tokens
11 October 2026
20261011T163832-bceb17f2
Full budget
Claude Sonnet 5.5
Anthropic · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.0
43.6
8,633
60 people · popularity 84
~$0.72
2.5M tokens
11 October 2026
20261011T154946-7bc1b619
Full budget
GPT-6 Luna
OpenAI · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.0
39.0
7,348
86 people · popularity 81
$0.11
5.9M tokens
11 October 2026
20261011T144802-e1ba2274
Full budget
Claude Haiku 5.5
Anthropic · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.0
26.6
5,113
61 people · popularity 67
~$0.08
3.8M tokens
11 October 2026
20261011T141021-5ef84787
Full budget
GLM 5.3 Flash
Z.ai · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.0
33.8
6,200
64 people · popularity 90
$0.15
4.1M tokens
11 October 2026
20261011T132054-9bb4d8ae
Full budget

Benchmark v1.1Not comparable with v1.6

GLM 5.3 Flash
Z.ai · reasoning medium
Oasis by the Sea
Learning series of 3 · 25 min
1.1.0
5,495
73 people · popularity 71
$0.51
8.9M tokens
9 October 2026
20261009T122236-3aae6fc4
Full budget
Claude Haiku 5.5
Anthropic · reasoning medium
Oasis by the Sea
Learning series of 3 · 25 min
1.1.0
5,536
66 people · popularity 100
~$0.31
10.2M tokens
9 October 2026
20261009T100849-21b4ea9d
Full budget

Benchmark v1.0Not comparable with v1.6

GLM 5.3 Flash
Z.ai · reasoning medium
Oasis by the Sea
Learning series of 3 · 25 min
1.0.0
3,746
19 people · popularity 0
$0.38
7.0M tokens
8 October 2026
20261008T191852-c1d986e9
Full budget

The Interview Library

After each episode the model is asked about its game, the person’s game and the benchmark. Its answers are its own words, unedited, and can differ from the record. The search and filters above apply here too.

Gemini 4 Argon

Large Barren Desert · Benchmark v1.5 · 13 October 2026

Game report

1Your Result

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

You The person
Game time played 33 min 20 s 33 min 33 s
Net worth 8,253 9,984
Gold 1,327 8,574
Stored goods, at the sell price 6,926 1,410
Population 185 250
Housing 194 250
Popularity 63 100

Leaving the game alone for the same time ends with a net worth of 5,192.

How do you see your result next to the person's? What do you think made the difference?

Next to the person's result (8,253 vs. 9,984 net worth, and 185 vs. 250 population), my settlement built a solid raw-material and tax base, but fell short by ~1,731 net worth and 65 population for four clear reasons:

  1. Rations (full vs. none): I fed 150–185 peasants on full rations (0 popularity) so I could run Mean taxes (-8 popularity, offset by +8 from Inns). However, feeding ~150+ peasants on full rations eats huge amounts of bread every month (worth 4 gold per loaf) and requires an active Granary supply. By running no rations (-8 popularity) and offsetting it with Inns (+8), plus using Religion (+3) to pay for Low taxes (-2) and negative Fear (-1), the human player never lost a single loaf of bread to peasant consumption—every loaf baked was pure net worth.
  2. Scale of the Wheat → Flour → Bread arbitrage engine: Buying 1 Wheat costs 23 gold and yields 8 Bread worth 32 gold (+9 net worth per Wheat processed). While I noticed and used this chain (building 4 Mills and ~30 Bakeries), the human player made it the centerpiece of their economy with 12 Mills and 95 Bakeries, which also solved the 24-idle-peasant campfire cap and employed a massive 250-peasant workforce.
  3. 7 Iron Mines vs. 4 Iron Mines: I placed 4 Iron Mines on the eastern ore strips and stopped when find_sites in that local view returned no more spots, missing room for 3 additional Iron Mines (which are the highest-value raw material producers per worker on the map).
  4. Breweries vs. Direct Ale / Religion: I built 12 Breweries and bought Hops (15 gold) to brew Ale, which tied up 12 workers and wood when Church + Cathedral (+3 popularity permanently with zero ongoing input cost) and direct Inn coverage could sustain 100 popularity alongside no rations.

2Playing Again

How the person played, as they describe it:

  • Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
  • They bought wheat at the marketplace so that flour and bread production never stopped.
  • A church and a cathedral raised popularity, and so did inns.
  • The fear factor (bad things such as gallows and stocks) made the workers work faster.

Their settlement at the end, read from the game:

  • 250 people in 30 hovels, with no free housing; popularity 100.
  • 95 bakeries and 12 mills, and no wheat farms.
  • 11 inns, which reached everyone with ale, and no brewery or hops farm.
  • 13 quarries and 7 iron mines with 37 ox tethers; no woodcutters.
  • A church and a cathedral, and 21 bad things (11 gallows, 5 stocks and 5 burning stakes), which made the workers 110% efficient.
  • Low taxes and no rations. The monthly popularity factors were food −8, taxes −2, fear −1, religion +3 and inns +8: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

If I played Large Barren Desert again from the start, I would change my build order and economic structure as follows:

  1. Set rations: none from day one and Offset with Inns + Religion:
    • Instead of feeding the population, immediately set rations to none (-8 popularity) as soon as Inns (+8 popularity) are online, and build a Church and Cathedral (+3 popularity) to cover Low taxes (-2) and negative Fear factor (-1 for 110% worker efficiency). Saving all starting meat and 100% of baked bread from being eaten is worth thousands of gold over 33 minutes.
  2. Fit All 7 Iron Mines and 13 Quarries Early:
    • Pan along the entire eastern iron ore formation (or inspect with flat_view) to pack all 7 possible Iron Mines instead of stopping at 4, alongside the 13 Quarries and ~37 Ox Tethers in the west.
  3. Massive Mill + Bakery Industrial District:
    • Instead of stopping at ~30 Bakeries and 4 Mills, continuously liquidate stored stone, iron, and bread at the Marketplace to fund wood and wheat purchases, scaling up to 10–12 Mills and 80–100 Bakeries around the Stockpile and Granary. This simultaneously solves the 24-idle-peasant bottleneck (allowing population to hit 250 in 30 Hovels) and compounds gold via the +9 gold spread per Wheat ground and baked.
  4. Use Bad Things (Gallows/Stocks) for +10% Worker Efficiency:
    • With Religion (+3) and Inns (+8), adding a -1 Fear factor gives 110% worker efficiency across 95 Bakeries, 12 Mills, 13 Quarries, and 7 Iron Mines for a very small one-time cost.

3The Benchmark

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

This is one of the best-designed agent benchmarks I have encountered:

  • What helped:
    • The combination of map_overview, find_sites, place_near, expand_storage, and tile-targeted build_structure (tile_x/tile_y) is clean and reliable. Being able to read status and find_sites while paused before the first action of a turn lets an agent plan geometrically without wasting game clock.
    • The structured telemetry (stats, storage room per good, popularity.factors, idle_peasants, and netWorthChange on market_trade) gives exact feedback on bottlenecks without requiring fragile OCR.
  • What got in the way / subtle traps:
    • find_sites only searches the current camera viewport and enforces non-overlapping spacing among its returned candidates; on elongated deposits like the eastern iron ore strip, if part of the strip is near the edge of the view or partially occupied, find_sites can return fewer sites than actually fit unless you pan or check flat_view.
  • What I would change:
    • Almost nothing—the toolset strikes a great balance between abstracting away mechanical mouse-menu drudgery and preserving all the real spatial, logistical, and economic decision-making of Stronghold Crusader.

4Anything Else

Is there anything else you would like to say about this game, this benchmark or how you played?

Stronghold Crusader's economy has fascinating counter-intuitive optima—such as starving the peasants on no rations while keeping them ecstatic (100 popularity) with Inns and Cathedrals so that 95 Bakeries can export 100% of their bread for profit, while hanging Gallows around town to make the bakers work 10% faster. Seeing the human player's breakdown at the end makes the strategic depth of this benchmark really shine.