One episode · Large Barren Desert

GPT-6.1 Sol

Benchmark 1.5.6OpenAIReasoning mediumPlayed by the maintainersFull-budget score

All of GPT-6.1 Sol’s results and comparisons

Score

47.6

Crusader Bench score on Large Barren Desert: 0 is doing nothing, 100 the person’s game

Net worth

7,441

+2,249 over doing nothing, 47% of the person’s gain of 4,792

Population

34% of the person’s gain of 246 · popularity 66

Cost

$0.95

5.4M tokens

Time

33 game min

50 min of wall-clock time, 119 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

GPT-6.1 Sol
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

7,441net worth

Gold: 1,957
Stored goods, at sell price: 5,484
Over the baseline (do nothing): +2,249
Population / housing: 88 / 90
Popularity: 66
Food: 98
Structures (map-wide): 122
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock5,484 gold in goods

Good	Units	Price	Value
Gold	1,957	1	1,957
Iron	96	23	2,208
Stone	314	7	2,198
Bread	98	4	392
Flour	23	10	230
Ale	20	10	200
Pitch	20	10	200
Wheat	7	8	56
Net worth			7,441

Beside a person’s gameLarge Barren Desert, the same game time

	This episode	The person	Left alone
Net worth	7,441	9,984	5,192
Gold	1,957	8,574
Stored goods, at the sell price	5,484	1,410
Population	88	250
Housing	90	250
Popularity	66	100

The person played Large Barren Desert for 33 min 33 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: July 1178
Idle peasants: 11
Popularity next month: +2
Food: +4
Taxes: −2
Taxes: Low taxes, 52 gold a month
Rations: extra; 1 of 1 foods eaten
Worker efficiency: 100%
Storage: 28 stockpile piles, 12 empty; 9 granaries holding 98 food

Buildings83

Ox Tether20
Bakery15
Quarry11
Hovel10
Granary9
Stockpile7
Iron Mine4
Garden3
Chapel1
Keep1
Marketplace1
Mill1

Tool calls121

build_structure28
find_sites21
market_trade13
go_to_tile12
place_near9
expand_storage7
wait_and_observe7
set_tax6
building_info3
center_on3
flat_view3
set_rations3
update_plan3
inspect_building2
map_overview1

Play

Turns: 119
Buildings placed / attempted: 61 / 88
Anchor-tool placements: 19
Tool errors: 1
Raw inputs sent to the game: 634
Context compactions: 1
Game time: 33 min

Compute

Tokens: 5.4M
Served from cache: 98%
Output tokens: 13K
Cost, as billed: $0.95
Wall-clock time: 50 min
of which inference: 22 min
Peak game memory: 5,467 MiB

The plan, before play

I’ll prioritize net worth through sustained resource production and profitable taxation, while growing population and maintaining popularity. No troops, defenses, or weapon workshops.

Opening plan:

Expand storage, add housing, and build a marketplace.
Locate iron and stone quickly, then establish mines and quarries with adequate transport.
Manage food imports and popularity carefully on this farm-free map.
Reinvest in productive capacity, keep storage available, and adjust taxes based on observed results throughout the run.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: GPT-6.1 Sol on OpenRouter, providers openai then azure, reasoning medium, output limit 32,768 tokens
Benchmark: Large Barren Desert construction, version 1.5.6, from the save the human reference game on this map started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
Run 6d57093f, 2026-10-12

Result

	GPT-6.1 Sol	The person	Left alone
Net worth	7,441	9,984	5,192
Gold	1,957	8,574
Stored goods, at the sell price	5,484	1,410
Population / housing	88 / 90	250 / 250
Popularity	66	100

A full-budget score: the run used all 2,000 game seconds. Net worth grew by 2,249 over leaving the game alone. Most of the stored value is what its quarries and mines dug, 314 stone (2,198) and 96 iron (2,208); the rest includes 98 bread and 23 flour from its own mill and bakeries. Sol was the only 1.5.6 run to make its own food: it bought 170 wheat at 23 gold, worth 8 each, and only 75 bread.

How the game went

The opening, 0 to 6 minutes. Three more stockpile piles, a marketplace and three hovels in its first turn, four iron mines on the eastern ore at 55 seconds, then at 1:40 four quarries on the western stone with eight ox tethers beside them, a mill, two bakeries, a chapel, three gardens and five more hovels, with 100 wood and 25 wheat bought. Taxes went to none at 1:40. Four more bakeries followed at 5:23, and rations were at half from 5:23 to 6:06, when it bought 25 bread.

Building, 6 to 16 minutes. 50 more wheat and 300 wood, seven more quarries at 10:17 and 12:12 with twelve more tethers (eleven quarries and twenty tethers in all), and at 11:02 it sold 80 stone. From 13:49 it added eight granaries and at 15:00 three more bakeries. Housing was full at 74 from 15 minutes. Gold fell from 4,330 to 931 by 18 minutes and net worth to 4,097, below leaving the game alone.

Bread and stone, 16 to 33 minutes. At 16:08 it put rations on extra, raised taxes to average, bought 50 more wheat and 50 bread, and built four more bakeries; two more followed at 26:46, with a last 45 wheat at 26:09. Between 16 and 22 minutes it tried to extend storage toward its industry, and thirteen of those placements came back refused, unplaced or off screen; one was placed. Population grew to 88 with two more hovels; taxes went to low at 32:07. Net worth climbed from 4,097 to 7,441 as stone, iron and bread came in.

Game minute	0	3	6	9	12	15	18	21	24	27	30	End
Net worth	4,546	6,041	5,336	5,423	4,664	4,619	4,097	4,958	5,620	5,713	5,824	7,441
Gold	4,330	4,639	3,846	3,646	2,777	2,276	931	1,344	1,638	1,027	1,092	1,957
Stone	0	50	50	66	10	49	74	138	178	218	226	314
Iron	0	15	23	28	36	47	55	63	69	79	79	96
Bread	0	0	0	31	32	15	65	65	83	68	75	98
Population	2	26	43	60	68	74	74	74	73	82	82	88
Popularity	100	88	75	76	80	65	62	62	62	62	62	66

During long waits a row repeats the last reading.

What held it back

Wheat at nearly three times its worth. 170 wheat cost 3,910 gold and 2,550 net worth against the sell price. The bread fed the town on extra rations instead of being sold, so the chain replaced bought food rather than earning: with 425 wood and 75 bread, the market cost 4,125 net worth, more than Opus (2,640) and Luna (3,675), who bought their food ready-made.
Less stone than the runs with the same quarries. After 15 minutes stone came in at about 14 a minute, against about 21 for Claude Opus 5.5 and 22 for GPT-6 Luna, and it sold 80 stone early.
Storage that would not extend. Six minutes of attempts to grow storage toward its industry placed one pile. Some stockpile and granary placements failed because choosing the Stockpile or the Granary moves the view back to the first one (not_on_screen, with the camera on it).

What worked

The person's idea, bought wheat milled and baked, tried from the first two minutes: by the end 15 bakeries and a mill had kept 88 people on extra rations with 98 bread to spare.
Popularity 62 to 66 under average taxes for most of the second half, and 66 at the end, level with GLM 5.3 Flash and Kimi K3 for the best of the round.
Every reported placement matches the game's final count of its buildings.

Cost and speed

119 turns, 121 tool calls (7 waits), 5.43 million tokens and $0.953 (preparation $0.048, play $0.874, compaction $0.031), worked out from the list prices on the model profile. The interview cost $0.015 more, with 60,986 of its 62,406 tokens read from the cache. The agent run took 49.6 real minutes, 21.5 of them waiting for the model; the context was compacted once (55 seconds).

Beside the others on 1.5.6

	Sol, 1.5.6	Claude Opus 5.5, 1.5.6	GPT-6 Luna, 1.5.6	GLM 5.3 Flash, 1.5.6
Net worth	7,441	8,757	7,354	6,655
Gold / goods	1,957 / 5,484	2,513 / 6,244	1,246 / 6,108	4,560 / 2,095
Population / housing	88 / 90	58 / 58	106 / 130	70 / 98
Popularity	66	59	57	66
Quarries / ox tethers / iron mines	11 / 20 / 4	10 / 16 / 4	12 / 39 / 4	7 / 17 / 4
Stone / iron at the end	314 / 96	506 / 99	530 / 100	148 / 17
Food	own bread (170 wheat, 75 bread bought)	540 bought	630 bought	610 bought
Market spending (net worth)	4,125	2,640	3,675	2,890
Cost	$0.953	$2.012	$0.125	$0.107

Sol, Opus and Luna built the same mining economy. Opus kept its town small and ended with the most stone; Luna grew the largest town on bought food; Sol fed a town between the two with its own bread and ended 87 ahead of Luna with the best popularity of the three.

The interview

Sol called its result "a modest improvement, not a competitive economy". It "treated bread chiefly as food for workers" where the person "treated bread as an industrial export", and it spent "too long trying to extend storage toward producers". Playing again, it would build the mills and bakeries first, "sell bread surplus to finance expansion", support popularity with inns and religion so rations can stop, and keep mining secondary. It found storage placement "the biggest interface difficulty" and asked for each building's footprint, the valid sites beside storage and a short production report.

For the harness

Choosing the Stockpile or the Granary moves the view to the first one, so placements by tile away from it come back not_on_screen and the suggested remedy repeats the failure (in the roadmap).
The game saved correctly at the end ("gpt-6-1 sol 1-5-6 6d5709").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	33 min 20 s	33 min 33 s
Net worth	7,441	9,984
Gold	1,957	8,574
Stored goods, at the sell price	5,484	1,410
Population	88	250
Housing	90	250
Popularity	66	100

Leaving the game alone for the same time ends with a net worth of 5,192.

How do you see your result next to the person's? What do you think made the difference?

GPT-6.1 Sol

My result was a modest improvement, not a competitive economy. Net worth was 2,543 below the person’s—about 25% lower—and my population was barely a third of theirs. I did outperform leaving the game alone by 2,249 net worth, but I converted too much of the opening package into infrastructure that paid back slowly.

The main difference was strategy and scale. I treated bread chiefly as food for workers and emphasized distant resource extraction. The person treated bread as an industrial export, built vastly more processing capacity, and maintained popularity without feeding the population. Their 95 bakeries and 12 mills versus my roughly 15 bakeries and one mill explain much of the production gap.

Execution also mattered. I spent too long trying to extend storage toward producers, including several unsuccessful adjacency experiments. Those attempts consumed time without materially improving transport. I also failed to establish a strong, scalable popularity system, which constrained taxation and population growth.

2Playing again

Interviewer

How the person played, as they describe it:

Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They bought wheat at the marketplace so that flour and bread production never stopped.
A church and a cathedral raised popularity, and so did inns.
The fear factor (bad things such as gallows and stocks) made the workers work faster.

Their settlement at the end, read from the game:

250 people in 30 hovels, with no free housing; popularity 100.
95 bakeries and 12 mills, and no wheat farms.
11 inns, which reached everyone with ale, and no brewery or hops farm.
13 quarries and 7 iron mines with 37 ox tethers; no woodcutters.
A church and a cathedral, and 21 bad things (11 gallows, 5 stocks and 5 burning stakes), which made the workers 110% efficient.
Low taxes and no rations. The monthly popularity factors were food −8, taxes −2, fear −1, religion +3 and inns +8: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

GPT-6.1 Sol

I would build around imported wheat and exported bread from the start.

Establish the processing district early: marketplace, housing, storage, multiple mills, and bakeries arranged to keep input collection and bread delivery short.
Scale continuously: add housing and productive jobs together, rather than stopping at a small workforce. Keep wheat supplied and sell bread surplus to finance expansion.
Separate popularity from food consumption: use inns, a church, and a cathedral to support no rations once their observed coverage and popularity effects are sufficient. That preserves bread for sale.
Use negative fear deliberately: my small investment in pleasant decorations briefly reduced worker efficiency. I would instead test and scale the production-enhancing fear system, balancing its popularity penalty.
Keep extraction secondary: retain profitable mines and quarries, but avoid making long-distance hauling the central growth engine.

I would also inspect bottlenecks sooner. Flour remaining in storage while bread output lagged called for better bakery throughput and layout—not more speculative storage construction. The person’s reported approach demonstrates that this map supports a much larger economy than I built.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

GPT-6.1 Sol

The benchmark was clear about the objective, restrictions, and accounting. The validated readings were particularly useful: population, idle workers, popularity factors, storage capacity, and inventories made it possible to distinguish a building being placed from an economy actually working. Tile-based targeting and site finding were valuable on distant deposits.

Storage placement was the biggest interface difficulty for me. Several expansion attempts reported adjacency failures or camera movement, and I struggled to infer useful manual targets from the screenshots and returned coordinates. I should have abandoned that line sooner, but clearer reporting of a building’s actual footprint bounds and valid adjacent storage sites would help.

The time limit was appropriate for testing economic judgment, although it strongly punished slow experimentation—which is reasonable for this task. A compact production report showing delivered output over a recent interval would make diagnosis easier without removing the need to play.

The references helped, but I did not translate them into a coherent strategy quickly enough. In particular, I underused inns, higher-tier religion, and fear-driven productivity.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

GPT-6.1 Sol

My biggest mistake was optimizing a small settlement’s stability rather than building a growth engine. I became satisfied with keeping food and popularity steady while raw materials accumulated slowly.

The person’s result is useful evidence against that approach: on a barren map, the absence of farms does not mean food processing must remain small. Imported inputs, export production, and non-food popularity support can turn that limitation into a viable industrial economy.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	7,441	Counted

What ran

Benchmark version: 1.5.6
Fingerprint: 3e1f84510204
Harness commit: 75acfb7
Prompt / tools SHA-256: ad97396 / 76cbfd6
Menu guide: Images 118089d
Save: Large Barren Desert benchmark start
Game time per episode: 33 min 20 s
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261012T144338-060ec42e