One episode · The Sinkhole

GPT-6 Luna

Benchmark 1.6.2OpenAIReasoning mediumPlayed by the maintainersFull-budget score

All of GPT-6 Luna’s results and comparisons

Score

25.5

Crusader Bench score on The Sinkhole: 0 is doing nothing, 100 the person’s game

Net worth

11,177

+8,982 over doing nothing, 23% of the person’s gain of 39,436

Population

108

21% of the person’s gain of 502 · popularity 52

Cost

$0.14

6.5M tokens

Time

45 game min

1 h 12 min of wall-clock time, 134 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

GPT-6 Luna
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 45 min of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

11,177net worth

Gold: 1,764
Stored goods, at sell price: 9,413
Over the baseline (do nothing): +8,982
Population / housing: 108 / 114
Popularity: 52
Food: 199
Structures (map-wide): 119
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock9,413 gold in goods

Good	Units	Price	Value
Gold	1,764	1	1,764
Iron	335	23	7,705
Cheese	183	4	732
Crossbows	10	30	300
Flour	21	10	210
Wood	176	1	176
Stone	20	7	140
Maces	3	30	90
Apples	15	4	60
Net worth			11,177

Beside a person’s gameThe Sinkhole, the same game time

	This episode	The person	Left alone
Net worth	11,177	41,631	2,195
Gold	1,764	39,977
Stored goods, at the sell price	9,413	1,654
Population	108	506
Housing	114	506
Popularity	52	99

The person played The Sinkhole for 44 min 58 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: July 1189
Idle peasants: 17
Popularity next month: −2
Food: +1
Taxes: −4
Religion: +1
Taxes: Average taxes, 86 gold a month
Rations: full; 2 of 2 foods eaten
Worker efficiency: 100%
Storage: 16 stockpile piles, 3 empty; 1 granary holding 198 food

Buildings83

Iron Mine27
Hovel13
Dairy Farm9
Woodcutter9
Apple Orchard8
Stockpile4
Fletchers' Workshop3
Wheat Farm2
Armory1
Bakery1
Blacksmith's Workshop1
Church1
Granary1
Keep1
Marketplace1
Mill1

Tool calls222

find_sites63
building_info33
build_structure29
market_trade14
set_tax14
status13
place_near9
wait_and_observe9
inspect_building8
get_inventory7
set_workshop5
set_rations4
go_to_tile3
observe3
center_on2
map_overview2
update_plan2
expand_storage1
flat_view1

Play

Turns: 134
Buildings placed / attempted: 68 / 69
Anchor-tool placements: 9
Tool errors: 1
Raw inputs sent to the game: 681
Context compactions: 4
Game time: 45 min

Compute

Tokens: 6.5M
Served from cache: 96%
Output tokens: 97K
Cost, as billed: $0.14
Wall-clock time: 1 h 12 min
of which inference: 32 min
Peak game memory: 3,298 MiB

The plan, before play

I’ll grow net worth by establishing woodcutters, housing and food production first, then expand into iron and stone production and weapon workshops as workers and resources allow. I’ll keep popularity above 50 with adequate rations and low taxes, and add storage or sell surplus before production is blocked. I’ll use live stats and terrain to guide placements, avoid defenses and recruitment, and keep building throughout the 45-minute budget.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: GPT-6 Luna on OpenRouter, providers openai then azure, reasoning medium, output limit 32,768 tokens
Benchmark: The Sinkhole construction, version 1.6.2, from the save the human reference game on this map started from; 45 min of game time at speed 40, the length of the person's game; no playbook
Run ae86527c, 2026-10-13

Result

	GPT-6 Luna	The person	Left alone
Net worth	11,177	41,631	2,195
Gold	1,764	39,977
Stored goods, at the sell price	9,413	1,654
Population / housing	108 / 114	506 / 506	4
Popularity	52	99	0

A full-budget score: the run used all 2,700 game seconds. Net worth grew by 8,982 over leaving the game alone, about 23% of the person's growth. Most of it is iron: 335 iron from 27 iron mines is 7,705 of the 9,413 in stored goods. The rest is 183 cheese (732), 10 crossbows and 3 maces (390), flour, wood and stone.

How the game went

The opening, 0 to 12 minutes. Three woodcutters and a marketplace in the first turn, then apple orchards, dairy farms, hovels against the keep with place_near, and its first iron mine at 1:31. By 12 minutes it had ten iron mines, a church, eight orchards and five dairy farms. It bought 210 wood and 75 food in these minutes, and gold fell from 2,000 to 258.

Iron, 12 to 31 minutes. Taxes rose to average at 15:19 and high at 16:31, and popularity, at 100 until then, fell to 57 by 25 minutes. An armory and a fletcher at 15:28, two more fletchers beside the armory at 21:41 and 22:41, all set to crossbows, then iron mines in threes: 20:39, 30:06 (with a blacksmith set to maces), and single ones between. A wheat farm, a mill and a bakery came at 28:54 to 29:17. Iron rose from 28 at 15 minutes to 153 at 30, and net worth from 1,371 to 5,886.

More mines, 31 to 45 minutes. Eight more iron mines from 36:54 to 41:19, 200 cheese bought at 34:52 and 36:05 for 1,600 gold, and taxes moved between none and average seven more times. Iron reached 335 and net worth 11,177. An expand_storage call for the stockpile at 39:35 found no free ground around it.

Game minute	0	5	10	15	20	25	30	35	40	End
Net worth	2,415	1,921	1,111	1,371	3,050	4,239	5,886	7,530	8,526	11,177
Gold	2,000	1,600	336	364	1,083	1,500	1,992	2,353	977	1,764
Iron	0	1	13	28	68	103	153	207	264	335
Population	5	39	50	64	69	81	87	96	85	108
Popularity	100	100	100	100	65	57	55	46	58	52

During long waits a row repeats the last reading.

What held it back

One good, held. Iron was nearly all of its growth and stayed in the stockpile; the town's income was taxes at low to average, which cost popularity (100 to about 55) and kept it there.
Food bought late and kept. 200 cheese for 1,600 gold at 35 minutes, and 183 were still in the granary at the end, worth 732: 868 of net worth for a few minutes of food. All told, the market cost 1,770 net worth (3,120 gold for 210 wood, 225 cheese, 25 bread, 25 meat and 10 apples).
A town of about 100. Its last hovels went up at 23:23, and by the end 108 people filled 114 places; more hovels and food would have been needed for more.

What worked

Fourth of the seven by net worth, from mining at scale: iron is worth 23 a unit, and it kept building mines to the last five minutes.
Its weapon workshops were all set to the dearer goods with set_workshop (crossbows, maces), by tile and by pixel.
Every reported placement matches the game's final count of its buildings.

Cost and speed

134 turns, 222 tool calls (9 waits), 6.52 million tokens and $0.138 (preparation $0.003, play $0.129, compaction $0.007), worked out from the list prices on the model profile. The interview cost $0.001 more, with 24,886 of its 26,669 tokens read from the cache. The agent run took 71.7 real minutes, 31.6 of them waiting for the model; the context was compacted four times (102 seconds in all).

Beside the others on 1.6.2

Scores by the project website's formula (80% net worth, 10% each population and popularity, each from doing nothing at 0 to the person's game at 1):

	Net worth	Gold / goods	Population	Popularity	Score	Cost
GPT-6.1 Sol	20,610	14,557 / 6,053	183	79	48.8	$0.995
Claude Sonnet 5.5	16,810	10,777 / 6,033	83	68	38.0	$0.752
Claude Opus 5.5	13,921	1,157 / 12,764	147	48	31.4	$2.470
GPT-6 Luna	11,177	1,764 / 9,413	108	52	25.5	$0.138
Kimi K3	9,164	2,356 / 6,808	91	52	21.1	$1.662
GLM 5.3 Flash	5,084	1,992 / 3,092	78	79	15.2	$0.114
Claude Haiku 5.5	3,312	1,716 / 1,596	37	51	8.0	$0.055

The interview

Luna said it "played cautiously": full rations, low taxes and stored iron rather than "aggressively scaling a production-and-population engine". Playing again, it would use inns and bought ale to cut rations and sell food, scale wheat, mills and bakeries sooner, add housing and water pots, and try fear and religion. It found the readings and tile placement useful, blamed its own conservative decisions more than the tools, and asked for clearer feedback on which buildings are staffed and producing.

For the harness

Two apple purchases (31:27, 35:30) stopped with "a purchase did not show in the reader within 5 seconds" after 10 and 0 apples. The same happened once in Claude Opus 5.5's run, where the reader showed the purchase landing with apples eaten and part of the month's taxes in one sample, so neither exact check matched. Fixed in a worktree for the next version.
The game saved correctly at the end ("gpt-6 luna 1-6-2 ae8652").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	45 min	44 min 58 s
Net worth	11,177	41,631
Gold	1,764	39,977
Stored goods, at the sell price	9,413	1,654
Population	108	506
Housing	114	506
Popularity	52	99

Leaving the game alone for the same time ends with a net worth of 2,195.

How do you see your result next to the person's? What do you think made the difference?

GPT-6 Luna

I substantially beat leaving the settlement alone, but fell far short of the person’s result. I ended with more value in stored goods than gold, while the person converted food production and a much larger workforce into nearly 40,000 gold. My population and popularity were also much lower.

The biggest difference was probably that I played cautiously: I maintained full rations and low taxes, preserved food, and accumulated iron rather than aggressively scaling a production-and-population engine. That kept the settlement stable, but left it with only 108 people and far less time to compound growth.

2Playing again

Interviewer

How the person played, as they describe it:

Early on they built a few inns and kept buying ale for them, which raised popularity by about 8 points. They spent those points on cutting the food rations to none, so the food could be sold instead of eaten.
They planted wheat farms and apple orchards and built a bread chain: mills ground the wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They built iron mines and quarries, and by the end a large number of iron mines. A few blacksmiths turned iron into swords, which they think made little difference.
Taxes stayed at none most of the time. Religion added popularity, which let them run the fear factor at 3 so that the workers worked faster.
The bread made the population very large. Water pots (60 gold each, three workers) gave jobs to more of it.

Their settlement at the end, read from the game:

506 people in 62 hovels, with no free housing and 16 idle; popularity 99, rising 3.
161 bakeries, 19 mills and 31 wheat farms; no apple orchards were left.
16 inns, with ale in the stockpile and no brewery or hops farm.
38 iron mines and 10 quarries with 29 ox tethers; 7 woodcutters.
9 blacksmiths, an armory holding 1 sword, and 24 water pots.
A church and a cathedral, and 105 bad things (48 burning stakes, 21 stretching racks, 12 stocks, 9 gallows and others), which made the workers 130% efficient.
No taxes and half rations of bread only. The monthly popularity factors were food −4, taxes +1, fear −3, religion +3 and inns +6.
39,977 gold and 1,654 in stored goods: a net worth of 41,631.

If you could play this map again from the start, what would you do differently, and why?

GPT-6 Luna

I would focus much earlier on the popularity and population loop the person used. I’d establish inns and buy ale, then use the popularity gained to reduce rations and sell surplus food rather than letting the population consume it. I’d scale wheat farms, mills, and especially bakeries sooner, while adding housing and water pots to create jobs for incoming peasants.

I would also test the fear-factor and religion strategy earlier instead of assuming that avoiding popularity penalties was best. The person’s account suggests the added worker efficiency and popularity factors supported much faster growth. I’d still monitor popularity and food closely, but I would be more willing to spend popularity on productive advantages.

My run did establish iron production and weapon workshops, but I overvalued stored iron relative to the return from turning production into gold and supporting more people. I’d build more aggressively while there was time for each new building to pay off.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

GPT-6 Luna

The tools and live readings were useful: I could check population, popularity, inventory, storage, and production evidence, and place buildings using map tiles rather than relying entirely on pixel clicks. The time budget made it important to batch actions and commit to a strategy early.

What got in my way was my own conservative decision-making and failure to identify the person’s growth strategy soon enough. I also found it hard to judge how much to invest in buildings that would only pay off after workers arrived and production ramped up. A compact, verified opening strategy—especially for managing inns, rations, food sales, housing, and fear—would make the benchmark easier to reason about. Clearer feedback about which buildings are staffed and producing would also help.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

GPT-6 Luna

The result shows that stable resource accumulation is not enough in this scenario: population growth, worker efficiency, and converting production into gold can compound much more strongly. I should have treated popularity as a resource to manage for growth, rather than mainly something to protect.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	11,177	Counted

What ran

Benchmark version: 1.6.2
Fingerprint: 6cc0c0162262
Harness commit: 1e52aca
Prompt / tools SHA-256: 43ab27e / e41a140
Menu guide: Images 118089d
Save: The Sinkhole start
Game time per episode: 45 min
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261013T011208-7dbd615d