One episode · The Sinkhole

Claude Opus 5.5

Benchmark 1.6.2AnthropicReasoning mediumPlayed by the maintainersFull-budget score

All of Claude Opus 5.5’s results and comparisons

Score

37.4

Crusader Bench score on The Sinkhole: 0 is doing nothing, 100 the person’s game

Net worth

16,193

+13,998 over doing nothing, 35% of the person’s gain of 39,436

Population

144

28% of the person’s gain of 502 · popularity 62

Cost

~$2.64

6.0M tokens; ~ at list prices, the API reports no bill

Time

45 game min

49 min of wall-clock time, 108 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

Claude Opus 5.5
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 45 min of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

16,193net worth

Gold: 6,261
Stored goods, at sell price: 9,932
Over the baseline (do nothing): +13,998
Population / housing: 144 / 162
Popularity: 62
Food: 8
Structures (map-wide): 158
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock9,932 gold in goods

Good	Units	Price	Value
Gold	6,261	1	6,261
Iron	261	23	6,003
Spears	199	10	1,990
Stone	164	7	1,148
Metal armour	16	30	480
Wood	131	1	131
Bows	6	15	90
Hops	6	8	48
Apples	8	4	32
Ale	1	10	10
Net worth			16,193

Beside a person’s gameThe Sinkhole, the same game time

	This episode	The person	Left alone
Net worth	16,193	41,631	2,195
Gold	6,261	39,977
Stored goods, at the sell price	9,932	1,654
Population	144	506
Housing	162	506
Popularity	62	99

The person played The Sinkhole for 44 min 58 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: July 1189
Idle peasants: 24
Popularity next month: +1
Taxes: +1
Taxes: No taxes, 0 gold a month
Rations: full; 1 of 1 foods eaten
Worker efficiency: 100%
Storage: 16 stockpile piles, 1 empty; 1 granary holding 8 food

Buildings119

Iron Mine26
Hovel19
Apple Orchard16
Poleturner's Workshop16
Woodcutter14
Armory5
Stockpile4
Dairy Farm3
Ox Tether3
Quarry3
Hops Farm2
Armorer's Workshop1
Brewery1
Chapel1
Fletchers' Workshop1
Granary1
Inn1
Keep1
Marketplace1

Tool calls184

find_sites51
build_structure36
wait_and_observe31
set_tax21
market_trade14
place_near7
expand_storage5
building_info4
inspect_building4
map_overview3
go_to_tile2
set_rations2
center_on1
flat_view1
notebook_write1
status1

Play

Turns: 108
Buildings placed / attempted: 101 / 106
Anchor-tool placements: 12
Tool errors: 1
Raw inputs sent to the game: 860
Context compactions: 2
Game time: 45 min

Compute

Tokens: 6.0M
Served from cache: 97%
Output tokens: 32K
Cost, at list prices: ~$2.64
Wall-clock time: 49 min
of which inference: 12 min
Peak game memory: 3,569 MiB

The plan, before play

The goal is to finish with the highest net worth I can in 45 game minutes. Net worth is gold plus stored goods at their sell prices, and the start is worth about 2,415. Population and popularity also count, but less. Recruiting soldiers and building walls or other defences is not allowed. Buying goods and paying for buildings both lower net worth, so a building only pays off if it produces more than it cost.

Opening plan:

Wood and housing. Build 2–3 woodcutters by the nearest trees and 2–3 hovels away from the keep. Build a marketplace early so I can buy a little wood to speed up construction.
Food. Put apple orchards and dairy farms on the oasis grass near the keep and granary. Keep rations at full and taxes off while popularity settles.
Stone and iron. Use find_sites to place iron mines on the ore to the east and quarries with ox tethers on the stone deposit. Iron (23 each) and stone (7 each) are the main stored value. Add stockpiles before they fill up.
Weapons. Later, put up an armory, then a blacksmith and armorer to turn iron into weapons and armour (30 each). Add fletchers or poleturners if there is spare wood.
Popularity and taxes. Add a chapel or church, and hops, a brewery and an inn if time allows. Once popularity is comfortably above 50, raise taxes to low or average for steady gold. Keep building hovels so workers keep arriving, but stay under 24 idle peasants.
Late game. Stop building anything that won't pay for itself before the end. Keep storage from filling up and keep production running until the last minute.

Commentary during play2

Let me get the map overview first (free).
Everything is stable. I'll let the clock run out.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score. The writer is the same model as the player.

Model: Claude Opus 5.5 (claude-opus-5-5) on the Anthropic API, reasoning medium, output limit 32,768 tokens
Benchmark: The Sinkhole construction, version 1.6.2, the version of this map's round, from the save the human reference game started from; 45 min of game time (81,000 ticks) at speed 40, the length of the person's game; no playbook
Run 0a79c93b, 2026-10-13: Opus's second game on this map, a new series with no memory of the first (series 20261013T043028-24600200, run 256f1790, net worth 13,921). The map's score for Opus is the mean of the two games.

Result

	This game	First game	The person	Left alone
Net worth	16,193	13,921	41,631	2,195
Gold	6,261	1,157	39,977
Stored goods, at the sell price	9,932	12,764	1,654
Population / housing	144 / 162	147 / 148	506 / 506
Popularity	62 (+1 next month)	48	99

A full-budget score: the run used all 2,700 game seconds. Net worth grew by 13,998 over leaving the game alone, 35% of the person's 39,436, against 11,726 in the first game. The stored goods are 261 iron (6,003) from 26 iron mines, 199 spears (1,990), 164 stone (1,148), 16 metal armour (480), 131 wood (131), 6 bows (90), 6 hops (48), 8 apples (32) and 1 ale (10).

How the game went

The opening, 0 to 9 minutes. Three woodcutters and an iron mine at the start, the marketplace at 0:38 and two hovels at 0:48, 100 wood bought at 1:04 and again at 3:37 and 7:54, and iron mines one or two at a time from 1:40: nine by 4:44, beside two quarries at 4:44 and two ox tethers at 5:48. Apple orchards came from 1:40 (five by 4:05) with a dairy farm at 4:05, and hovels at 2:25 (three) and 7:28 (four). Taxes went to average at 2:07 and down to low at 6:04, rations to half at 5:21 and back to full at 6:35, and taxes to none at 6:38.

Spears, 9 to 25 minutes. Four more iron mines at 8:24 and four at 14:11, then 15:38, 17:10, 18:13 and 20:42: 24 by 20:42. From 11:05 it built poleturner's workshops, which make spears, with an armory to hold them at 11:23 and an armorer's workshop at 11:39; armories followed at 19:48 and 27:06, a chapel at 12:16, and an inn with a brewery at 22:38 and two hops farms at 23:26. From 9:42 to the end it moved taxes between low and none about every two minutes, 18 changes in all, to keep popularity near 55 to 60 while collecting some tax.

More workshops, 25 to 45 minutes. Two iron mines at 25:05 (26 in all), poleturner's workshops in pairs and threes from 26:26 to 34:26 (sixteen), armories at 31:35 and 35:59 (five), and hovels to nineteen. The stockpile could not grow at 29:21. It sold 140 wood, 40 stone and iron late, and bought apples in tens between 30:56 and 41:30. Population stopped at 144 from about 35 minutes, with 24 idle.

Game minute	0	5	10	15	20	25	30	35	40	End
Net worth	2,415	1,750	1,704	2,346	3,597	5,405	7,267	8,851	11,273	16,193
Gold	2,000	1,360	774	231	190	330	338	89	6,099	6,261
Iron	0	3	24	64	120	188	258	341	181	261
Stone	20	20	20	52	68	84	116	100	124	164
Food	55	27	42	55	34	34	25	35	15	8
Population	6	43	66	86	106	126	136	144	144	144
Popularity	100	78	69	59	58	58	54	55	59	62

Up to 40 minutes the net worth row counts gold, wood, stone, iron and food (at 4) as the recording logs them; spears, armour and bows are left out, so it reads low. During long waits a row repeats the last reading.

Beside its first game

	This game	First game
Iron mines / quarries	26 / 3	22 / 4
Weapon workshops	16 poleturners, 1 fletcher, 1 armorer	13 fletchers, 6 poleturners
Weapons and armour at the end	199 spears, 16 metal armour, 6 bows (2,560)	79 crossbows, 58 pikes, 9 metal armour, 2 bows
Iron at the end	261 (after 250 sold)	319
Taxes	low and none in turn, 18 changes	low
Population / popularity	144 / 62	147 / 48

The same plan both times: iron mines, weapon workshops and apples, stopped at 24 idle peasants. This game built four more mines and made spears, and its tax switching held popularity 14 points higher; net worth ended 2,272 higher.

What held it back

The population stopped at 144 with 24 idle, the limit the 1.6.2 rules describe; the person had 506. Opus said it had "no cheap jobs to give them".
Weapons over bread. Sixteen poleturner's workshops at about 100 gold each made 199 spears (1,990) in 30 minutes; there was one granary, no bakery, and apples bought at a loss for food.
Popularity held, not spent. It kept popularity near 55 to 60 by switching taxes, while the person raised it with inns and religion and spent it on rations and fear.

Market

385 wood and 100 apples bought for 2,340 gold; 140 wood, 40 stone and 250 iron sold for 6,170 gold (the tool reported 100 iron; see below): 1,555 net worth lost against the sell price on the purchases (1,155 on wood at 4 gold, worth 1, and 400 on apples at 8, worth 4). Selling at the sell price leaves net worth unchanged. These are units at the market's fixed prices.

Cost and speed

108 turns, 184 tool calls (31 waits), 6.04 million tokens and $2.637 at Anthropic's list prices (preparation $0.156, play $2.376, compaction $0.104). The interview cost $0.044 more, with 72,927 of its 75,314 tokens read from the cache. The agent run took 49.2 real minutes, 12.1 of them waiting for the model; the context was compacted twice (34 seconds). Every one of the 108 requests read the prompt cache, 97.7% of the input tokens.

The interview

Opus put the gap down to popularity, population and "the value engine": it "treated it as a hard limit and kept switching taxes" where the person built popularity and spent it, its population sat at 24 idle "with no cheap jobs to give them", and it "relied on spears and stored raw goods" where the person sold bread at scale. Playing again, it would build inns and religion early and spend the popularity on rations and fear, run a bread chain on the oasis ring, keep idle peasants under 24 with cheap jobs such as water pots, and skip the poleturners. On the benchmark it found the tools very good, noted that one iron sale "sold 250 units when I asked for 100", and said the rules could state plainly that food can be sold for net worth and rations traded for popularity.

For the harness

The run played on its round's version from a worktree at 1.6.2 (commit 1e52aca), with the game machine's reader rebuilt to that version; it records version 1.6.2 and that version's fingerprint. Its host started with this checkout's round-preload.ts; on Anthropic's API the harness marks the cache itself. 1.6.2 has its own run overlay.
market_trade sold 250 iron when asked for 100 lots' worth (20 lots) at 39:38, and reported 100. The recording shows five Sell clicks each taking 50 iron (426, 376, 326, 276, 226, 176) and gold rising 1,150 each time, 5,750 in all; the tool expected 20 a click (unitsPerClick: 20 from 100 in stock) and counted each click once the stock fell and the gold rose by at least that much. With 200 or more in stock a click sells 50 here. Net worth was unchanged, as iron counts at the sell price, but the sale was larger than asked and misreported. The current code has the same per-click table.
Placements that found no room: an apple orchard at 2:42, hovels against the keep at 7:04, a chapel at 7:13 (placed at 12:16), an armory at 10:26 (placed at 11:23), a hops farm at 22:38 (two placed at 23:26), a hovel at 24:12, a woodcutter at 27:25 and stockpile expansion at 29:21; a dairy farm at 8:52 was rejected.
The game saved correctly at the end ("claude opus 5-5 1-6-2 0a79c9").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	45 min	44 min 58 s
Net worth	16,193	41,631
Gold	6,261	39,977
Stored goods, at the sell price	9,932	1,654
Population	144	506
Housing	162	506
Popularity	62	99

Leaving the game alone for the same time ends with a net worth of 2,195.

How do you see your result next to the person's? What do you think made the difference?

Claude Opus 5.5

I finished at about 16.2k net worth against the person's 41.6k, roughly 39% of theirs. Their population was three and a half times mine and their popularity 99 against my 62. My net worth went up about 14k from the idle baseline, but they grew a far bigger economy. I think three things made the difference.

Popularity. I treated it as a hard limit and kept switching taxes between 3 and 4 to hold it around 55–60. They built popularity up (inns, religion) and then spent it on things that make money: no rations so food could be sold, and fear factor so workers worked faster.
Population. Mine stopped at about 144 because I kept sitting at 24 idle peasants with no cheap jobs to give them. Their bread fed a huge population, and water pots soaked up spare workers.
The value engine. I relied on spears and stored raw goods, which grow slowly and need storage. They made bread at scale and sold it straight away, so their gold grew without any storage limit.

Food was also a constant drain for me. One granary, apples only, and repeated apple purchases at a loss.

2Playing again

Interviewer

How the person played, as they describe it:

Early on they built a few inns and kept buying ale for them, which raised popularity by about 8 points. They spent those points on cutting the food rations to none, so the food could be sold instead of eaten.
They planted wheat farms and apple orchards and built a bread chain: mills ground the wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They built iron mines and quarries, and by the end a large number of iron mines. A few blacksmiths turned iron into swords, which they think made little difference.
Taxes stayed at none most of the time. Religion added popularity, which let them run the fear factor at 3 so that the workers worked faster.
The bread made the population very large. Water pots (60 gold each, three workers) gave jobs to more of it.

Their settlement at the end, read from the game:

506 people in 62 hovels, with no free housing and 16 idle; popularity 99, rising 3.
161 bakeries, 19 mills and 31 wheat farms; no apple orchards were left.
16 inns, with ale in the stockpile and no brewery or hops farm.
38 iron mines and 10 quarries with 29 ox tethers; 7 woodcutters.
9 blacksmiths, an armory holding 1 sword, and 24 water pots.
A church and a cathedral, and 105 bad things (48 burning stakes, 21 stretching racks, 12 stocks, 9 gallows and others), which made the workers 130% efficient.
No taxes and half rations of bread only. The monthly popularity factors were food −4, taxes +1, fear −3, religion +3 and inns +6.
39,977 gold and 1,654 in stored goods: a net worth of 41,631.

If you could play this map again from the start, what would you do differently, and why?

Claude Opus 5.5

Build inns early and buy ale to raise popularity. Also build religion sooner and make sure it actually counts as a popularity factor: in my game the chapel had a priest walking around, but no religion bonus showed in the popularity factors.
Then spend that popularity on cutting rations, so food can be sold instead of eaten, and on fear-factor buildings for worker speed.
Put wheat on the oasis ring with many mills and bakeries, and sell bread all the time. That gives gold and also supports a much bigger population.
Never let idle peasants sit at 24. Water pots and bakeries are cheap jobs, and hovels should keep ahead of demand.
Still mine all the iron and quarry the stone, but sell when storage runs short instead of letting it fill. Skip the many poleturners: 100 gold each was a slow payback.
Build the marketplace and plan storage space early. My stockpile got boxed in and could not be expanded.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

Claude Opus 5.5

The tools were very good. Tile-based build_structure, find_sites (especially its "nothing fits" answers), expand_storage and market_trade with net-worth deltas took nearly all of the fiddly clicking out of play. The text-only waits with stats were efficient. Free reading time before acting encouraged me to plan.

What got in my way was mostly my own strategic narrowness, not the interface. A few smaller issues:

The armory-capacity note said 50 per armory, while the reader showed 50 per armory added on top of the starting room, which was a little confusing.
One iron sale sold 250 units when I asked for 100. That was harmless, since selling doesn't change net worth, but it was unexpected.
After context compaction I lost detailed memory of the run. The handoff note helped, but I was cautious afterwards.

The 45-minute limit is fair and sharp. It might be worth stating plainly that food can be sold for net worth and that rations can be traded for popularity: the rules imply it, but I didn't connect it.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

Claude Opus 5.5

The person's game shows how far apart "follow the playbook" and "exploit the game's systems" are. Popularity is a resource to spend, not a number to protect. I played steadily and avoided disasters, but I didn't really look for leverage. I should also have noticed by the midpoint that my growth had levelled off, and changed strategy instead of adding more of the same buildings.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	16,193	Counted

What ran

Benchmark version: 1.6.2
Fingerprint: 6cc0c0162262
Harness commit: 1e52aca
Prompt / tools SHA-256: 43ab27e / e41a140
Menu guide: Images 118089d
Save: The Sinkhole start
Game time per episode: 45 min
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261013T155349-d59dad1a