One episode · Large Barren Desert

GPT-6.1 Sol

Benchmark 1.6.3OpenAIReasoning mediumPlayed by the maintainersFull-budget score

All of GPT-6.1 Sol’s results and comparisons

Score

98.3

Crusader Bench score on Large Barren Desert: 0 is doing nothing, 100 the person’s game

Net worth

10,289

+5,097 over doing nothing, 106% of the person’s gain of 4,792

Population

114

45% of the person’s gain of 246 · popularity 87

Cost

$1.00

4.9M tokens

Time

33 game min

48 min of wall-clock time, 113 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

GPT-6.1 Sol
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

10,289net worth

Gold: 3,998
Stored goods, at sell price: 6,291
Over the baseline (do nothing): +5,097
Population / housing: 114 / 114
Popularity: 87
Food: 123
Structures (map-wide): 136
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock6,291 gold in goods

Good	Units	Price	Value
Gold	3,998	1	3,998
Stone	530	7	3,710
Iron	41	23	943
Bread	123	4	492
Flour	43	10	430
Spears	26	10	260
Pitch	20	10	200
Wheat	20	8	160
Ale	6	10	60
Wood	36	1	36
Net worth			10,289

Beside a person’s gameLarge Barren Desert, the same game time

	This episode	The person	Left alone
Net worth	10,289	9,984	5,192
Gold	3,998	8,574
Stored goods, at the sell price	6,291	1,410
Population	114	250
Housing	114	250
Popularity	87	100

The person played Large Barren Desert for 33 min 33 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: July 1178
Idle peasants: 22
Popularity next month: ±0
Taxes: −8
Ale from the inn: +8
Taxes: Mean taxes, 136 gold a month
Rations: full; 1 of 1 foods eaten
Worker efficiency: 100%
Storage: 36 stockpile piles, 16 empty; 1 granary holding 123 food

Buildings92

Ox Tether31
Bakery13
Hovel13
Quarry10
Stockpile9
Inn4
Iron Mine4
Poleturner's Workshop3
Armory1
Granary1
Keep1
Marketplace1
Mill1

Tool calls117

find_sites28
build_structure25
market_trade11
place_near9
game_action7
set_tax6
wait_and_observe6
expand_storage5
building_info4
update_plan4
go_to_tile3
flat_view2
inspect_building2
map_overview2
observe2
center_on1

Play

Turns: 113
Buildings placed / attempted: 71 / 80
Anchor-tool placements: 18
Tool errors: 1
Raw inputs sent to the game: 634
Context compactions: 1
Game time: 33 min

Compute

Tokens: 4.9M
Served from cache: 97%
Output tokens: 14K
Cost, as billed: $1.00
Wall-clock time: 48 min
of which inference: 19 min
Peak game memory: 3,555 MiB

The plan, before play

I’ll maximize final net worth, with population and popularity as secondary goals, without recruiting troops or building defenses.

I’ll open with housing, a marketplace and storage expansion, then locate iron deposits and establish mining. Since the map has no farmland, I’ll compare imported food processing with other production options, manage food and taxes carefully, and reinvest in productive jobs while checking staffing, access and storage capacity.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: GPT-6.1 Sol (openai/gpt-6.1-sol) on OpenRouter, providers openai then azure, reasoning medium, output limit 32,768 tokens
Benchmark: Large Barren Desert construction, version 1.6.3, from the save the human reference game started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
Run 0608bbd7, series 20261014T112858-49a607da, 2026-10-14: Sol's second game on this map, a new series with no memory of the first, played on the latest version (the first game, series 20261012T144338-060ec42e, played 1.5.6). The map's score for Sol is the mean of the two.

Result

	This game	First game	The person	Left alone
Net worth	10,289	7,441	9,984	5,192
Gold	3,998	1,957	8,574
Stored goods, at the sell price	6,291	5,484	1,410
Population / housing	114 / 114	88 / 90	250 / 250
Popularity	87	66	100

A full-budget score: the run used all 2,000 game seconds. Net worth grew by 5,097 over leaving the game alone, 106% of the person's 4,792 and the first game on this map to end above the person's net worth; the first game grew 2,249. The stored goods are 530 stone (3,710), 41 iron (943), 123 bread (492), 43 flour (430), 26 spears (260), 20 pitch (200), 20 wheat (160), 6 ale (60) and 36 wood (36).

How the game went

Bread and mines, 0 to 9 minutes. A marketplace, three stockpile piles and three hovels in the first 42 seconds, four iron mines at 1:01, a mill at 1:34 and four bakeries at 1:48, then 50 wheat bought at 2:22 and an inn at 2:47. Four quarries at 4:56 with twelve ox tethers by 6:57, an armory and three poleturner's workshops at 7:34, three more bakeries and four more hovels by 8:46.

Growing, 9 to 25 minutes. Four more quarries at 12:37 and fifteen more tethers by 15:01, three more bakeries at 18:57 and three at 22:52 (thirteen), more hovels at 22:16, and inns at 13:24, 19:48 and 22:37 (four in all). Two more quarries and four tethers at 24:03–24:33 were the last buildings. Wheat was bought in fifties four times and fed the mill; no food was bought. From 15:31 to 19:21 it tried to lay stockpile piles toward the quarries, and most of those placements were rejected or stopped when the camera moved; two piles went down.

Taxing on the inns, 21 to 33 minutes. At 21:47 it set mean taxes (popularity −8) with the inns giving +8, and popularity held at 87 to the end while the taxes brought 136 gold a month. Gold rose from about 2,000 at 27 minutes to 3,998, stone from 394 to 530, and the population reached 114 with every house full. It sold 56 iron at 18:31 and 39 spears at 23:22.

Game minute	0	3	6	9	12	15	18	21	24	27	30	End
Net worth	4,741	4,668	4,590	4,374	3,730	3,846	4,621	4,000	5,160	5,616	7,114	10,289
Gold	4,460	3,750	3,542	3,196	2,312	1,901	2,143	2,368	2,655	2,022	2,946	3,998
Iron	0	15	23	31	39	48	56	8	16	24	30	41
Stone	0	50	50	50	58	98	146	192	274	394	442	530
Food	50	39	30	20	20	30	33	17	46	62	87	123
Population	2	29	46	58	70	80	90	90	100	114	114	114
Popularity	100	100	98	89	77	69	83	91	89	87	87	87

Up to 30 minutes the net worth row counts gold, wood, stone, iron and food (at 4) as the recording logs them; flour, wheat, spears and pitch are left out, so it reads low.

Beside its first game

	This game (1.6.3)	First game (1.5.6)
Net worth	10,289	7,441
Buildings	10 quarries, 31 tethers, 13 bakeries, mill, 4 inns, 4 iron mines, 13 hovels, 3 poleturner's workshops	11 quarries, 20 tethers, 15 bakeries, mill, 4 iron mines, 10 hovels, chapel
Population / popularity	114 / 87	88 / 66
Taxes at the end	mean, 136 a month	low, 52 a month
Net worth lost buying	4,040	4,125
Cost	$1.00	$0.95

Both games ran the same plan: bought wheat for their own bread, quarries and iron mines. This one added four inns, which let it tax hard without losing popularity, and half again as many tethers, which brought 530 stone against 314. It also grew a larger town.

What held it back

Wheat at 23. 200 wheat cost 4,600 gold and is worth 8 each stored; the bread fed the town instead of food bought at 8, but the purchases still lost 3,000 net worth.
Time on stockpile corridors. About four game minutes of placements toward the quarries, most of them rejected.
A town of 114. The person had 250; Sol said it was too cautious about scale.

Market

200 wheat, 280 planks and 20 ale bought for 6,120 gold; 56 iron and 39 spears sold for 1,678. The purchases lost 4,040 net worth against the sell price (wheat 23 for 8, planks 4 for 1, ale 20 for 10); selling leaves net worth unchanged. These are units at the market's fixed prices. One wheat purchase at 10:26 bought nothing.

Cost and speed

113 turns, 117 tool calls (6 waits), 4.94 million tokens and $0.997 billed by OpenRouter (preparation $0.051, play $0.806, compaction $0.139). The interview cost about $0.014 more, with 52,462 of its 53,888 tokens read from the cache. The agent run took 47.8 real minutes, 19.2 of them waiting for the model; the context was compacted once (49 seconds). All 113 requests read the prompt cache, 98.2% of the input tokens.

The interview

Sol said it finished narrowly ahead on net worth but would not call it an unqualified win, with less than half the person's population. It said its raw materials paid back while it stayed too conservative on food processing and jobs, and that it spent too long on stockpiles toward remote resources. Playing again it would make bread from bought wheat its main industry and sell it, with no rations once inns and religion carry popularity, add a modest fear factor, and grow housing and jobs together. It praised the benchmark for measuring a running economy, and asked for confirmed footprint coordinates after placement, more predictable placement around camera moves, and a production summary. Its biggest mistake, it said, was underestimating the value of scale.

For the harness

A stale map probe on the game machine. Like every game of this batch on Oasis by the Sea and Large Barren Desert, this game ran a crusader_probe built from 1.5.0's sources. It was built on 2026-10-13 at 21:28 UTC as a 1.5.0 game started. The 1.6.3 sync then copied sources that matched by hash, and Ninja found nothing to rebuild. The probe's only change since 1.5.0 is 1.5.4's check that tells trees from cacti, so map_overview counted cacti as tree groves. find_sites counts trees from the reader's tile data and was right. Everything else this game played was 1.6.3, and the game stands as played. Sol called map_overview twice and built no woodcutter.
Stockpile placements failed with camera_moved and rejected while Sol tried to extend storage toward the quarries (11:30 to 19:21); Sol asked for more predictable placement when selecting storage moves the view.
The first set_tax at 3:04 failed ("The keep hotkey selected nothing") and worked on the retry.
The game saved correctly at the end ("gpt-6-1 sol 1-6-3 0608bb").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	33 min 20 s	33 min 33 s
Net worth	10,289	9,984
Gold	3,998	8,574
Stored goods, at the sell price	6,291	1,410
Population	114	250
Housing	114	250
Popularity	87	100

Leaving the game alone for the same time ends with a net worth of 5,192.

How do you see your result next to the person's? What do you think made the difference?

GPT-6.1 Sol

I finished narrowly ahead on net worth—10,289 versus 9,984, about 3%—but the person built a much larger settlement, with more than twice my population and higher popularity. Given the secondary objectives, I would not call mine an unqualified win.

My result leaned heavily on stored stone and other goods. Selling those would have changed the gold balance, not net worth, so the difference in cash alone is not a performance gap. The person’s population and industrial scale are the more important contrast.

I think my raw-material investments eventually paid back, while I stayed too conservative about expanding food processing and employment. I also spent valuable time trying to extend stockpiles toward remote resources. Only a small part of that worked, and I should have abandoned the unsuccessful attempts sooner.

2Playing again

Interviewer

How the person played, as they describe it:

Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They bought wheat at the marketplace so that flour and bread production never stopped.
A church and a cathedral raised popularity, and so did inns.
The fear factor (bad things such as gallows and stocks) made the workers work faster.

Their settlement at the end, read from the game:

250 people in 30 hovels, with no free housing; popularity 100.
95 bakeries and 12 mills, and no wheat farms.
11 inns, which reached everyone with ale, and no brewery or hops farm.
13 quarries and 7 iron mines with 37 ox tethers; no woodcutters.
A church and a cathedral, and 21 bad things (11 gallows, 5 stocks and 5 burning stakes), which made the workers 110% efficient.
Low taxes and no rations. The monthly popularity factors were food −8, taxes −2, fear −1, religion +3 and inns +8: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

GPT-6.1 Sol

I would make imported-wheat bread production the main scalable industry much earlier, rather than treating it primarily as a way to feed the settlement.

The person’s combination is instructive: no rations meant bread could be sold rather than consumed, while inns and religion offset the popularity penalty. I evaluated food production mostly within a full-rations, tax-supported economy and missed that stronger combination.

My revised approach would be:

Establish the marketplace, housing and a compact wheat–flour–bread district early.
Keep wheat supplied automatically and expand mills and bakeries according to observed bottlenecks.
Build inns and religious coverage deliberately, then switch to no rations once the popularity balance supports it.
Introduce a modest negative fear factor and verify its productivity benefit against its popularity cost.
Continue expanding housing and productive jobs together, rather than settling at 114 people.
Retain mining and quarrying as complementary income, but account for their long transport delays before committing heavily.

I would also test production rates earlier. A small working bread district would have provided better evidence for reinvestment than my cautious estimates did.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

GPT-6.1 Sol

The strongest feature is that it measures an actual running economy. Staffing, walking distance, storage, delayed deliveries and popularity all mattered. The inventory and status readers were particularly helpful, and the placement tools made large-scale construction feasible without manually navigating every menu.

The main difficulty was placement reliability around camera movement. Stockpile expansion sometimes stopped with camera_moved, and attempts to build a contiguous corridor were difficult to diagnose. Those failures cost both game time and attention. I compounded that by persisting too long.

I would improve three things:

Report confirmed building footprint coordinates after placement, making adjacency easier to reason about.
Make camera-settling and placement failures more predictable, especially when selecting storage recentres the view.
Provide a read-only production summary showing staffing, recent output and blocked deliveries. Inventory totals helped, but did not explain every bottleneck.

The game-time budget and paused thinking were fair and useful. The instructions clearly explained the scoring and prohibited actions. They were lengthy, though; a shorter operational checklist alongside the full reference would make important constraints easier to retain.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

GPT-6.1 Sol

My biggest strategic mistake was underestimating the value of scale. I repeatedly asked whether one additional building would repay its cost, but did not sufficiently pursue the coordinated system—housing, jobs, processing, popularity and trade—that could support many more buildings.

The close net-worth result should not hide that weakness. My economy accumulated useful stored value, but the person demonstrated a substantially larger operating economy. I would want a replay to test whether I could combine that scale with earlier investment and tighter logistics.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	10,289	Counted

What ran

Benchmark version: 1.6.3
Fingerprint: 96c4682877ca
Harness commit: 1d0c697
Prompt / tools SHA-256: 3bb4a16 / e41a140
Menu guide: Images 118089d
Save: Large Barren Desert benchmark start
Game time per episode: 33 min 20 s
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261014T112858-49a607da