Games

Every game played for the benchmark and published to the dataset: 28 games and 34 episodes, grouped by benchmark version, newest first. Scores compare only within a version. Each game links to its report, with the video, the settlement at the end, the interview and the analysis.

Benchmark v1.6Latest

Kimi K3
Moonshot AI · reasoning high
The Sinkhole
One episode · 45 min
1.6.2
21.1
9,164
91 people · popularity 52
~$1.66
2.6M tokens
13 October 2026
20261013T052713-e8ad024c
Full budget
Claude Opus 5.5
Anthropic · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
31.4
13,921
147 people · popularity 48
~$2.47
5.4M tokens
13 October 2026
20261013T043028-24600200
Full budget
GPT-6.1 Sol
OpenAI · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
48.8
20,610
183 people · popularity 79
$0.99
6.2M tokens
13 October 2026
20261013T032320-2db24416
Full budget
Claude Sonnet 5.5
Anthropic · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
38.0
16,810
83 people · popularity 68
~$0.75
3.1M tokens
13 October 2026
20261013T023616-1529e12f
Full budget
GPT-6 Luna
OpenAI · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
25.5
11,177
108 people · popularity 52
$0.14
6.5M tokens
13 October 2026
20261013T011208-7dbd615d
Full budget
Claude Haiku 5.5
Anthropic · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
8.0
3,312
37 people · popularity 51
~$0.05
3.0M tokens
13 October 2026
20261013T002522-ef6aa9c5
Full budget
GLM 5.3 Flash
Z.ai · reasoning medium
The Sinkhole
One episode · 45 min
1.6.2
15.2
5,084
78 people · popularity 79
$0.11
3.2M tokens
12 October 2026
20261012T233119-e8fb78d0
Full budget

Benchmark v1.5Not comparable with v1.6

Claude Opus 5.5
Anthropic · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.1
52.0
10,054
107 people · popularity 82
~$3.06
5.8M tokens
12 October 2026
20261012T154553-38bd30ef
Full budget
GPT-6.1 Sol
OpenAI · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
47.6
7,441
88 people · popularity 66
$0.95
5.4M tokens
12 October 2026
20261012T144338-060ec42e
Full budget
Kimi K3
Moonshot AI · reasoning high
Large Barren Desert
One episode · 33 min 20 s
1.5.6
19.4
5,741
94 people · popularity 66
~$2.10
4.5M tokens
12 October 2026
20261012T134119-f09e54cb
Full budget
GPT-6 Luna
OpenAI · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
45.9
7,354
106 people · popularity 57
$0.13
6.6M tokens
12 October 2026
20261012T123257-cf634001
Full budget
Claude Opus 5.5
Anthropic · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
67.6
8,757
58 people · popularity 59
~$2.01
3.6M tokens
12 October 2026
20261012T113644-1024a7b0
Full budget
Claude Sonnet 5.5
Anthropic · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
21.0
6,252
27 people · popularity 24
~$0.42
1.6M tokens
12 October 2026
20261012T110247-91bd9173
Full budget
GLM 5.3 Flash
Z.ai · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
33.7
6,655
70 people · popularity 66
$0.11
2.6M tokens
12 October 2026
20261012T101643-b291d04d
Full budget
Claude Haiku 5.5
Anthropic · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.6
26.6
6,412
32 people · popularity 51
~$0.03
1.8M tokens
12 October 2026
20261012T094211-1fcf305a
Full budget
GPT-6 Luna
OpenAI · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.4
36.6
6,626
78 people · popularity 97
$0.09
4.3M tokens
12 October 2026
20261012T072046-29a58979
Full budget
Claude Sonnet 5.5
Anthropic · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.4
16.6
5,567
32 people · popularity 92
~$0.29
1.3M tokens
12 October 2026
20261012T064725-135cbd14
Full budget
GLM 5.3 Flash
Z.ai · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.4
57.4
8,232
47 people · popularity 49
$0.06
1.5M tokens
11 October 2026
20261011T214835-33796a71
Full budget
Claude Haiku 5.5
Anthropic · reasoning medium
Large Barren Desert
One episode · 33 min 20 s
1.5.4
12.5
5,313
32 people · popularity 93
~$0.04
1.7M tokens
11 October 2026
20261011T211444-de27ae23
Full budget
GPT-6.1 Sol
OpenAI · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.1
43.4
7,401
130 people · popularity 100
$0.95
5.0M tokens
11 October 2026
20261011T175318-657a6989
Full budget
Kimi K3
Moonshot AI · reasoning high
Oasis by the Sea
One episode · 33 min 20 s
1.5.1
31.1
5,918
76 people · popularity 69
~$3.55
6.4M tokens
11 October 2026
20261011T163832-bceb17f2
Full budget
Claude Sonnet 5.5
Anthropic · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.0
43.6
8,633
60 people · popularity 84
~$0.72
2.5M tokens
11 October 2026
20261011T154946-7bc1b619
Full budget
GPT-6 Luna
OpenAI · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.0
39.0
7,348
86 people · popularity 81
$0.11
5.9M tokens
11 October 2026
20261011T144802-e1ba2274
Full budget
Claude Haiku 5.5
Anthropic · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.0
26.6
5,113
61 people · popularity 67
~$0.08
3.8M tokens
11 October 2026
20261011T141021-5ef84787
Full budget
GLM 5.3 Flash
Z.ai · reasoning medium
Oasis by the Sea
One episode · 33 min 20 s
1.5.0
33.8
6,200
64 people · popularity 90
$0.15
4.1M tokens
11 October 2026
20261011T132054-9bb4d8ae
Full budget

Benchmark v1.1Not comparable with v1.6

GLM 5.3 Flash
Z.ai · reasoning medium
Oasis by the Sea
Learning series of 3 · 25 min
1.1.0
5,495
73 people · popularity 71
$0.51
8.9M tokens
9 October 2026
20261009T122236-3aae6fc4
Full budget
Claude Haiku 5.5
Anthropic · reasoning medium
Oasis by the Sea
Learning series of 3 · 25 min
1.1.0
5,536
66 people · popularity 100
~$0.31
10.2M tokens
9 October 2026
20261009T100849-21b4ea9d
Full budget

Benchmark v1.0Not comparable with v1.6

GLM 5.3 Flash
Z.ai · reasoning medium
Oasis by the Sea
Learning series of 3 · 25 min
1.0.0
3,746
19 people · popularity 0
$0.38
7.0M tokens
8 October 2026
20261008T191852-c1d986e9
Full budget

The Interview Library

After each episode the model is asked about its game, the person’s game and the benchmark. Its answers are its own words, unedited, and can differ from the record. The search and filters above apply here too.

Kimi K3

The Sinkhole · Benchmark v1.6 · 13 October 2026

Game report

1Your Result

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

You The person
Game time played 45 min 44 min 58 s
Net worth 9,164 41,631
Gold 2,356 39,977
Stored goods, at the sell price 6,808 1,654
Population 91 506
Housing 122 506
Popularity 52 99

Leaving the game alone for the same time ends with a net worth of 2,195.

How do you see your result next to the person's? What do you think made the difference?

I ended at 9,164 net worth versus the person's 41,631 — roughly 4.5× behind, and the composition tells the story. My wealth was mostly stored goods (6,808 vs their 1,654); theirs was almost entirely gold (39,977 vs my 2,356). I built a hoarding economy, they built a selling economy. I treated stored goods as the goal because the scoring counts them at sell price, which is technically true, but I missed the multiplier: a large population running a bread-for-sale engine converts labor into gold continuously, while my workshops and mines trickled value at a fixed rate no matter what. Their 506 population versus my 91 is the real gap — I let the 24-idle-peasant arrival cap strangle my growth for the entire run, adding jobs one or two at a time, when I should have been building jobs in bulk (their 161 bakeries and 24 water pots are essentially population engines). I also played too cautiously on popularity: I kept taxes positive-but-modest and full rations, while they did the counterintuitive thing — buying ale for inns, cutting rations to none, selling the food, and stacking fear-factor buildings for 130% worker efficiency. I never touched fear factor at all.

2Playing Again

How the person played, as they describe it:

  • Early on they built a few inns and kept buying ale for them, which raised popularity by about 8 points. They spent those points on cutting the food rations to none, so the food could be sold instead of eaten.
  • They planted wheat farms and apple orchards and built a bread chain: mills ground the wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
  • They built iron mines and quarries, and by the end a large number of iron mines. A few blacksmiths turned iron into swords, which they think made little difference.
  • Taxes stayed at none most of the time. Religion added popularity, which let them run the fear factor at 3 so that the workers worked faster.
  • The bread made the population very large. Water pots (60 gold each, three workers) gave jobs to more of it.

Their settlement at the end, read from the game:

  • 506 people in 62 hovels, with no free housing and 16 idle; popularity 99, rising 3.
  • 161 bakeries, 19 mills and 31 wheat farms; no apple orchards were left.
  • 16 inns, with ale in the stockpile and no brewery or hops farm.
  • 38 iron mines and 10 quarries with 29 ox tethers; 7 woodcutters.
  • 9 blacksmiths, an armory holding 1 sword, and 24 water pots.
  • A church and a cathedral, and 105 bad things (48 burning stakes, 21 stretching racks, 12 stocks, 9 gallows and others), which made the workers 130% efficient.
  • No taxes and half rations of bread only. The monthly popularity factors were food −4, taxes +1, fear −3, religion +3 and inns +6.
  • 39,977 gold and 1,654 in stored goods: a net worth of 41,631.

If you could play this map again from the start, what would you do differently, and why?

I would invert almost everything:

  1. Rush inns + bought ale early. Ale popularity (+8 across several inns) is buyable with starting gold and pays for itself by enabling everything else. I built one inn very late; they had 16.
  2. Cut rations and sell food. Once inns and religion carry popularity, rations are a luxury. Bread sold at the market is the map's best gold engine, and wheat/flour/bread all chain from one farm type. My 84 flour and 52 bread sitting in storage at the end was dead capital.
  3. Bread chain at scale, immediately. Dozens of wheat farms on the oasis ring, mills, and a bakery district — not 4 farms and 6 bakeries. Bakeries are also cheap jobs that pull population past the idle cap.
  4. Mass jobs early to break the 24-idle cap. Water pots (60 gold, 3 workers) are the cheapest population pump in the game. Population compounds: more workers → more production → more gold → more buildings.
  5. Fear factor. I completely ignored it. 105 bad things giving 130% worker efficiency is a free 30% output multiplier paid for with popularity I was hoarding for no reason. Popularity is a currency — spend it.
  6. Iron mines in bulk. They ended with 38; I had 4. Iron is the best raw value per worker and miners carry it themselves.
  7. Taxes at zero. Their no-tax +1 popularity versus my +72-80 gold/month was clearly the right trade — the popularity headroom bought them fear factor and ration cuts worth far more.

In short: I optimized for the scoring formula's letter (stored goods count!) while they optimized for the economy's actual growth rate, and growth won by a mile.

3The Benchmark

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

The tools are genuinely good. find_sites and tile-targeted build_structure were the standouts — being able to place buildings anywhere on the map without camera hunting is what made a dense build order possible at all. status giving popularity factors, storage room per good, and idle counts let me manage the economy numerically rather than by feel. The free-reads-before-actions design (reading costs no game time) is well thought out and rewards planning.

What got in my way:

  • Placement tile reports were unreliable for follow-up clicks. Twice a workshop's reported tile later opened a different building (Keep, Armory, Bakery), and set_workshop by tile failed three times. I burned several minutes and replies hunting one fletcher. Having build_structure return a verified clickable tile of the actual footprint (re-read from the game after placement, including retry offsets) would fix this.
  • The armory adjacency requirement ("must touch the stockpile") cost a failed placement and confusion; having placement rules surfaced per-building in building_info would help.
  • The 24-idle arrival cap is a game rule, not a harness issue, but the instructions mentioning it more prominently ("population growth stops entirely at 24 idle — build jobs in bulk, not pairs") would have changed my whole run. I understood it intellectually and still under-reacted.
  • Minimum turn length (~8 game seconds per reply) is fair but punishes exploratory clicking; it pushed me toward batch reads, which is probably the intended discipline.

The 45-minute budget felt right — long enough for compounding to show, short enough that opening mistakes (mine) are decisive. The instructions were clear, though the opening playbook's cautious tone ("keep taxes low, keep rations at least normal") nudged me toward exactly the conservative play that lost; a hint that selling food and fear-factor are legitimate top-tier strategies would level the field.

4Anything Else

Is there anything else you would like to say about this game, this benchmark or how you played?

The most interesting thing this benchmark exposed is a genuine strategic blind spot rather than an execution failure. I knew net worth was the score, I computed NW deltas on trades correctly, and I still lost by 4.5× because I reasoned about margins per item instead of throughput of the whole system. The person's settlement is ugly by my run's standards — almost nothing stored, no taxes, rations cut, burning stakes everywhere — and it printed gold. There's a lesson in there about optimizing the metric's definition versus optimizing the generator of the metric.

Mechanically, my mid-run improvements worked: buying food to restore variety after it crashed, dropping taxes when religion lapsed, and selling flour when the stockpile hit one empty pile were all correct reactive calls — the end state was stable (popularity 52, no stall, armory 95/100). But I spent the first 15 minutes at population ~75-80 when the map could clearly support hundreds, and in a compounding game that early plateau is unrecoverable. If I played again tomorrow, the entire run would be different from minute one: inns, ale, bread-for-sale, water pots, and fear factor, with taxes never enabled at all.