One episode · Large Barren Desert

Claude Opus 5.5

Benchmark 1.6.3AnthropicReasoning mediumPlayed by the maintainersFull-budget score

All of Claude Opus 5.5’s results and comparisons

Not part of the evaluation: the game is published, but not ranked.

Score

43.3

Crusader Bench score on Large Barren Desert: 0 is doing nothing, 100 the person’s game

Net worth

7,659

+2,467 over doing nothing, 51% of the person’s gain of 4,792

Population

21% of the person’s gain of 246 · popularity 0

Cost

~$1.44

2.4M tokens; ~ at list prices, the API reports no bill

Time

33 game min

33 min of wall-clock time, 44 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

Claude Opus 5.5
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

7,659net worth

Gold: 1,711
Stored goods, at sell price: 5,948
Over the baseline (do nothing): +2,467
Population / housing: 55 / 98
Popularity: 0
Food: 0
Structures (map-wide): 95
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock5,948 gold in goods

Good	Units	Price	Value
Gold	1,711	1	1,711
Stone	490	7	3,430
Iron	46	23	1,058
Swords	17	30	510
Metal armour	15	30	450
Pitch	20	10	200
Bows	8	15	120
Ale	9	10	90
Spears	9	10	90
Net worth			7,659

Beside a person’s gameLarge Barren Desert, the same game time

	This episode	The person	Left alone
Net worth	7,659	9,984	5,192
Gold	1,711	8,574
Stored goods, at the sell price	5,948	1,410
Population	55	250
Housing	98	250
Popularity	0	100

The person played Large Barren Desert for 33 min 33 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: July 1178
Idle peasants: 0
Popularity next month: −16
Food: −8
Taxes: −8
Taxes: Mean taxes, 66 gold a month
Rations: half; no food in the granary
Worker efficiency: 100%
Storage: 16 stockpile piles, 1 empty; 1 granary holding 0 food

Buildings66

Ox Tether21
Hovel11
Quarry10
Poleturner's Workshop6
Iron Mine4
Stockpile4
Blacksmith's Workshop2
Armorer's Workshop1
Armory1
Chapel1
Fletchers' Workshop1
Granary1
Inn1
Keep1
Marketplace1

Tool calls113

market_trade35
wait_and_observe17
build_structure14
find_sites14
observe10
place_near9
set_tax5
expand_storage3
building_info2
map_overview1
notebook_write1
set_rations1
status1

Play

Turns: 44
Buildings placed / attempted: 48 / 49
Anchor-tool placements: 15
Tool errors: 1
Raw inputs sent to the game: 663
Context compactions: 1
Game time: 33 min

Compute

Tokens: 2.4M
Served from cache: 95%
Output tokens: 21K
Cost, at list prices: ~$1.44
Wall-clock time: 33 min
of which inference: 7 min
Peak game memory: 3,498 MiB

The plan, before play

My understanding of the task

This is a 33-minute race on Large Barren Desert. The final score counts net worth most, then population, then popularity. Net worth is gold plus stored goods at their market sell price. Buying goods and paying for construction lower net worth, so a building only pays off if it produces more than it cost. There is no oasis grass, so I can't build any farms. Food has to be bought or come from the starting 55 meat. Soldiers and defences are not allowed.

Opening plan

First minute: build a marketplace, 3–4 hovels near the keep and woodcutters by any trees near the stockpile. Add stockpile room early, because the starting package is still being paid in and is waiting for space.
Iron: iron ore lies about 60 tiles east. I'll put up several iron mines there (iron sells for 23) and a second stockpile area if the walk proves long. Later, blacksmiths and armorers turn 1 iron into a sword or metal armour worth 30, stored in an armory.
Wood into weapons: poleturners (spears 10, pikes 18) and fletchers (bows 15) fed by woodcutters. If wood runs short, buying it at 4 a unit to make pikes or bows still adds value. I'll build an armory early and expand it.
Stone: quarries plus ox tethers on the western deposits, if the walk to the stockpile is workable.
Population and popularity: keep rations at full. I'll buy cheap food (apples or cheese at 8) only as needed, since mixing food types raises popularity. A chapel or church adds religion, and a later inn adds ale. Once popularity is comfortably above 50, I'll raise taxes to low or average for steady gold. I'll keep idle peasants under 24 by adding jobs.
Upkeep: watch storage, sell surplus raw goods only when piles fill up, and keep building production up to the end. In the last minutes I'll stop construction that won't pay back in time.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: Claude Opus 5.5 (claude-opus-5-5) on the Anthropic API, reasoning medium, output limit 32,768 tokens
Benchmark: Large Barren Desert construction, version 1.6.3, from the save the human reference game started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
Run 57a2cd90, series 20261014T083631-0973b922, 2026-10-14: Opus's second game on this map, a new series with no memory of the first, played on the latest version (the first game, series 20261012T113644-1024a7b0, played 1.5.6). The map's score for Opus is the mean of the two.

Result

	This game	First game	The person	Left alone
Net worth	7,659	8,757	9,984	5,192
Gold	1,711	2,513	8,574
Stored goods, at the sell price	5,948	6,244	1,410
Population / housing	55 / 98	58 / 58	250 / 250
Popularity	0 (−16 next month)	59	100

A full-budget score: the run used all 2,000 game seconds. Net worth grew by 2,467 over leaving the game alone, 51% of the person's 4,792, against 3,565 in the first game. The stored goods are 490 stone (3,430), 46 iron (1,058), 17 swords (510), 15 metal armour (450), 20 pitch (200), 8 bows (120), 9 ale (90) and 9 spears (90). The pitch and some of the ale came with the start.

How the game went

Mines and a weapons wing, 0 to 6 minutes. Four iron mines in its first turn, three hovels, two stockpile piles, a marketplace at 1:08, then an armory at 1:18 with poleturner's, blacksmith's, armorer's and fletcher's workshops around it by 5:26: ten weapon workshops in all, six of them poleturners, and an inn. Planks were bought from 3:08 to feed them.

Quarries, 3 to 22 minutes. Ten quarries between 3:41 and 20:48, 21 ox tethers by 21:57, eleven hovels and a chapel. The stone lies about 90 tiles from the stockpile and arrived slowly at first, then at about 22 a minute from 18 minutes. The population grew to 95 by 24 minutes, with taxes off from 12:17 and popularity held at 52 on half rations.

A full armory. The armory held 38 of its 50 places by 9 minutes, and a second one found no room (6:23). From 9:44 Opus sold spears and swords every few minutes to keep the workshops delivering: 130 spears and 15 swords for 1,750 gold. It bought 530 planks and 395 apples and cheese in all.

The last four minutes. At 29:36, with 18 food on half rations, it set mean taxes (popularity −8 a month) to collect from the frequent paydays. Ninety seconds later popularity was 14, falling 10 more a month, and 10 people had gone; it judged a bribe not worth the gold and kept the taxes. The game ended with popularity 0, no food and 55 people.

Game minute	0	3	6	9	12	15	18	21	24	27	30	End
Net worth	4,741	5,159	3,799	3,397	3,677	3,413	3,633	3,616	3,721	4,111	4,867	7,659
Gold	4,460	4,109	2,801	2,244	2,211	1,958	1,668	1,128	688	638	922	1,711
Iron	0	14	18	23	26	29	31	34	36	39	43	46
Stone	0	50	50	50	74	82	146	202	274	330	410	490
Food	50	80	40	42	83	46	55	60	63	62	21	0
Population	2	27	47	60	64	76	79	85	94	95	95	55
Popularity	100	92	78	79	57	52	52	52	52	52	47	0

Up to 30 minutes the net worth row counts gold, wood, stone, iron and food (at 4) as the recording logs them; weapons, ale and pitch are left out, so it reads low.

Beside its first game

	This game (1.6.3)	First game (1.5.6)
Net worth	7,659	8,757
Buildings	10 quarries, 21 tethers, 4 iron mines, 11 hovels, armory and 10 weapon workshops, inn, chapel	10 quarries, 16 tethers, 4 iron mines, 6 hovels
Iron / stone kept	46 / 490	99 / 506
Population / popularity	55 (95 at most) / 0	58 / 59
Bought	395 food, 530 planks	food and wood, 4,960 gold in all
Cost	$1.44	$2.01

The quarries did as well as in the first game. The difference is the weapons wing: iron went into swords and armour instead of the stockpile, and planks bought at 4 to feed the poleturners are worth 1 each. The town grew larger than the first game's (95 people against 58) until the last four minutes.

What held it back

The weapons chain lost value. 530 planks for 2,120 gold (worth 530), and the armory was nearly full from 9 minutes, so the workshops depended on selling weapons at 10 a spear to keep going.
Food bought at a loss: 395 apples and cheese for 3,160 gold, worth 1,580.
No room to grow. At 29 minutes 24 of its 95 people were idle and only 3 places were free.

The tax raise at the end cost popularity (52 to 0) and 40 people, but by the formula it probably raised the score. Mean taxes brought about 780 gold in the last 3½ minutes, about 13 points of the score. Popularity and population count 10% each, and even holding the 29-minute values (52 and 95 people) they would have added about 7 points; with food running out both were falling anyway. Without the raise the score would have been about 37 at best, against 43.3. Opus itself judged that the raise "clearly lowered my score".

Market

395 food (205 apples, 190 cheese) and 530 planks bought for 5,280 gold; 130 spears and 15 swords sold for 1,750. The purchases lost 3,170 net worth against the sell price (food 8 for 4, planks 4 for 1); selling leaves net worth unchanged. These are units at the market's fixed prices.

Cost and speed

44 turns, 113 tool calls (17 waits), 2.44 million tokens and $1.441 at Anthropic's list prices (preparation $0.167, play $1.216, compaction $0.058). The interview cost about $0.352 more, none of its 66,767 tokens read from the cache (see For the harness). The agent run took 33.3 real minutes, 6.8 of them waiting for the model; the context was compacted once (18 seconds). All 44 requests read the prompt cache, 96.7% of the input tokens.

The interview

Opus said its net worth came mostly from stone, and that the person won with a self-funding wheat, flour and bread chain and with popularity from inns, a church and a cathedral, while it bought food at a loss and sat at 52 popularity with no taxes. It called its late tax raise a mistake, a gamble on a wrong estimate of how fast popularity would fall. Playing again it would start quarries and tethers in the first minutes, process bought wheat into bread, treat inns and churches as investments, look at the fear factor, avoid an end-game tax spike, and stop buying wood for spears. It found the placement tools and the stats with popularity factors and storage room useful; it lost reasoning across compaction, could not see production rates, and asked for a short popularity forecast. Its armory got boxed in early with no warning while it placed buildings around it.

For the harness

A stale map probe on the game machine. Like every game of this batch on Oasis by the Sea and Large Barren Desert, this game ran a crusader_probe built from 1.5.0's sources. It was built on 2026-10-13 at 21:28 UTC as a 1.5.0 game started. The 1.6.3 sync then copied sources that matched by hash, and Ninja found nothing to rebuild. The probe's only change since 1.5.0 is 1.5.4's check that tells trees from cacti, so map_overview counted cacti as tree groves. find_sites counts trees from the reader's tile data and was right. Everything else this game played was 1.6.3, and the game stands as played. Opus called map_overview once and built no woodcutter.
No warning while popularity fell. The 90-second wait after the tax raise returned with popularity at 14 and game_events empty; nothing interrupted the wait. A warning when popularity, food or gold drops sharply has been proposed.
The interview missed the prompt cache. Its last reply came 5.9 minutes before the interview (a 120-second wait ran into the deadline, then the game saved), past the 5-minute cache life, so all 65,521 tokens of context were written to the cache again, most of its $0.35 cost.
The armory found no room for a second one at 6:23 and was boxed in by the workshops placed around it; the "Armory full" warning came at 27:28.
The game saved correctly at the end ("claude opus 5-5 1-6-3 57a2cd").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	33 min 20 s	33 min 33 s
Net worth	7,659	9,984
Gold	1,711	8,574
Stored goods, at the sell price	5,948	1,410
Population	55	250
Housing	98	250
Popularity	0	100

Leaving the game alone for the same time ends with a net worth of 5,192.

How do you see your result next to the person's? What do you think made the difference?

Claude Opus 5.5

I finished at 7,659 net worth against the person's 9,984, and about 2,500 above leaving the game alone. On population (55 against 250) and popularity (0 against 100) I lost badly. My net worth came mostly from stone. Once I had about 11 quarries and roughly 25 ox tethers working, stone rose by about 35 a minute, and I ended with 450 stone (≈3,150). Iron and weapons added a little.

What made the difference:

A self-funding food and income loop. The person bought wheat and turned it into bread at a very large scale (95 bakeries). That made gold and fed and pleased the population at the same time. I bought food for every meal at 8 gold a unit, which steadily lost net worth.
Popularity as an engine. Eleven inns, a church and a cathedral let them hold 100 popularity while still taxing. I sat at 52 with no taxes for most of the game, so I had no tax income. Population was also stuck at the idle-peasant cap of 24.
My late tax hike was a mistake. I raised taxes to "mean" in the last four minutes, and popularity fell from 52 to 0 much faster than I had worked out. Population dropped from 95 to 55. It gained a few hundred gold and cost a lot of population and popularity score.
Early waste. I spent heavily on wood and food in the first half. I set up quarries late, and their first stone took a long time to arrive because they are about 90 tiles from the stockpile.

2Playing again

Interviewer

How the person played, as they describe it:

Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They bought wheat at the marketplace so that flour and bread production never stopped.
A church and a cathedral raised popularity, and so did inns.
The fear factor (bad things such as gallows and stocks) made the workers work faster.

Their settlement at the end, read from the game:

250 people in 30 hovels, with no free housing; popularity 100.
95 bakeries and 12 mills, and no wheat farms.
11 inns, which reached everyone with ale, and no brewery or hops farm.
13 quarries and 7 iron mines with 37 ox tethers; no woodcutters.
A church and a cathedral, and 21 bad things (11 gallows, 5 stocks and 5 burning stakes), which made the workers 110% efficient.
Low taxes and no rations. The monthly popularity factors were food −8, taxes −2, fear −1, religion +3 and inns +8: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

Claude Opus 5.5

Start quarries and tethers in the first minutes. They were my only strong producer. I would also put a stockpile closer to the quarries if the game allowed it, or simply add tethers early.
Build a value-added processing chain. I'd buy wheat and process it into flour and bread, after checking the margins myself. Bread both feeds the people and sells.
Treat inns, churches and the cathedral as investments. High popularity allows steady taxes and draws people into more housing. I'd also add enough jobs that idle peasants never block growth.
Look at fear-factor buildings. I never considered them for worker efficiency.
No end-game tax spike. If I raised taxes at all, I'd do it gently and early, with popularity built up to absorb it.
Stop sinking gold into wood for spears. Low workshop output and a boxed-in armory made that chain weak. Each wood bought at 4 sells for 1, and I had to keep selling weapons to make room.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

Claude Opus 5.5

What helped:

Tile-based placement, find_sites and place_near. Building far away without fiddly pixel work was easy.
The stats JSON with popularity factors and per-good storage room, and the market tool's net-worth change. They made the economics clear.

What got in my way:

My own context loss across compaction. My notes didn't hold enough of my reasoning.
No easy view of production rates. I estimated them from inventory changes.
I couldn't see how fast popularity falls month to month at a given tax level. A short popularity forecast would have stopped my final mistake.

What I'd change: the time limit is fine. Building blocks of storage was awkward: my armory got boxed in early and nothing warned me while I was placing buildings around it.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

Claude Opus 5.5

My main lesson is that I played too defensively and only on the raw-materials side. I treated popularity as a limit to stay above instead of something to raise and spend. The person's result shows that on a map with no farms, buying a cheap input and processing it at scale, backed by strong popularity, beats extracting raw materials. I also want to own the final tax decision plainly. It was a gamble based on a wrong estimate of how fast popularity would fall, and it clearly lowered my score.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	7,659	Counted

What ran

Benchmark version: 1.6.3
Fingerprint: 96c4682877ca
Harness commit: 1d0c697
Prompt / tools SHA-256: 3bb4a16 / e41a140
Menu guide: Images 118089d
Save: Large Barren Desert benchmark start
Game time per episode: 33 min 20 s
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261014T083631-0973b922