One episode · Oasis by the Sea

Claude Opus 5.5

Benchmark 1.5.1AnthropicReasoning mediumPlayed by the maintainersFull-budget score

All of Claude Opus 5.5’s results and comparisons

Score

52.0

Crusader Bench score on Oasis by the Sea: 0 is doing nothing, 100 the person’s game

Net worth

10,054

+8,834 over doing nothing, 48% of the person’s gain of 18,293

Population

107

52% of the person’s gain of 198 · popularity 82

Cost

~$3.06

5.8M tokens; ~ at list prices, the API reports no bill

Time

33 game min

42 min of wall-clock time, 101 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

Claude Opus 5.5
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

10,054net worth

Gold: 2,471
Stored goods, at sell price: 7,583
Over the baseline (do nothing): +8,834
Population / housing: 107 / 138
Popularity: 82
Food: 186
Structures (map-wide): 112
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock7,583 gold in goods

Good	Units	Price	Value
Gold	2,471	1	2,471
Iron	178	23	4,094
Stone	336	7	2,352
Bread	184	4	736
Wood	393	1	393
Apples	2	4	8
Net worth			10,054

Beside a person’s gameOasis by the Sea, the same game time

	This episode	The person	Left alone
Net worth	10,054	19,513	1,220
Gold	2,471	18,304
Stored goods, at the sell price	7,583	1,209
Population	107	202
Housing	138	202
Popularity	82	100

The person played Oasis by the Sea for 33 min 18 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: October 1200
Idle peasants: 24
Popularity next month: +1
Food: +5
Taxes: −4
Taxes: Average taxes, 85 gold a month
Rations: extra; 2 of 2 foods eaten
Worker efficiency: 100%
Storage: 24 stockpile piles, 4 empty; 1 granary holding 186 food

Buildings82

Hovel16
Bakery13
Iron Mine12
Ox Tether11
Woodcutter8
Stockpile6
Quarry5
Apple Orchard4
Dairy Farm2
Mill2
Granary1
Keep1
Marketplace1

Tool calls191

find_sites39
go_to_tile33
build_structure28
wait_and_observe27
market_trade18
inspect_building10
set_tax9
building_info7
observe7
expand_storage5
map_overview3
center_on2
notebook_write1
place_near1
set_rations1

Play

Turns: 101
Buildings placed / attempted: 73 / 84
Anchor-tool placements: 5
Tool errors: 3
Raw inputs sent to the game: 606
Context compactions: 4
Game time: 33 min

Compute

Tokens: 5.8M
Served from cache: 96%
Output tokens: 34K
Cost, at list prices: ~$3.06
Wall-clock time: 42 min
of which inference: 13 min
Peak game memory: 3,256 MiB

The plan, before play

I'm playing a Free Build on Oasis by the Sea and have about 33 minutes of game time. The run is scored on net worth (gold plus stored goods at sell price), population and popularity. Soldiers, walls and towers aren't allowed. Buying lowers net worth and building costs resources, so whatever I build has to pay for itself before the run ends.

Opening plan:

Granary: place it right away near the keep and stockpile with build_structure ("Granary").
Wood: build 2–3 woodcutters by the western grove. If wood holds things up early, I may buy a little at a marketplace.
Housing: build 2–3 hovels away from the signpost so the population can grow while popularity stays above 50.
Food: put apple orchards and dairy farms close together on the oasis grass south-west of the keep, so the granary fills directly. Keep rations at full.
Raw materials: build iron mines on the ore strip just north of the keep (iron sells for 23). Later, a quarry and ox tethers at the stone deposit about 50 tiles west (stone sells for 7). Add stockpiles before storage fills.
Popularity: add a chapel or church, and hops → brewery → inn if there's time.
Taxes: once popularity is comfortably high, raise taxes to low or average for steady gold, while watching popularity and housing.
Late game: stop building that won't pay back in time. Keep goods stored, since they count toward net worth. Popularity and population should be high when the run ends.

Commentary during play2

Mines are smoking (working). Let me inspect one.
Wood 67. Build two quarries and two tethers.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: Claude Opus 5.5 (claude-opus-5-5) on Anthropic's API, reasoning medium, output limit 32,768 tokens
Benchmark: Oasis by the Sea construction, version 1.5.1, the version of this map's round (the other six models played 1.5.0 or 1.5.1, which differ only in how Kimi K3's reasoning level is sent), from the save the human reference game started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
Run a4c1769c, 2026-10-12

Result

	Claude Opus 5.5	The person	Left alone
Net worth	10,054	19,513	1,220
Gold	2,471	18,304
Stored goods, at the sell price	7,583	1,209
Population / housing	107 / 138	202 / 202
Popularity	82 (+1 next month)	100

A full-budget score: the run used all 2,000 game seconds. Net worth grew by 8,834 over leaving the game alone, about half of the person's 18,293 and the most of any model on this map. Most of it is stored goods: 178 iron (4,094) from twelve iron mines, 336 stone (2,352), 184 bread (736) and 393 wood (393).

How the game went

The opening, 0 to 8 minutes. Two woodcutters in its first seconds, then three iron mines, three hovels and a marketplace at 22 seconds with 80 wood bought; three apple orchards and a dairy farm at 1:41; two more iron mines and hovels and two stockpile piles at 2:25, after the first try at them was off screen; a quarry with two ox tethers at 3:08; three more woodcutters, an orchard and a dairy farm at 4:34; and seven more iron mines by 7:29, twelve in all. 285 wood came from the market over the first 7 minutes. Gold fell from 935 to 460 by 3 minutes and stayed under 500 until 27; taxes went down to none at 8:12.

Mines and housing, 8 to 14 minutes. Two quarries with a tether at 8:01, three woodcutters at 9:38 and two more quarries with two tethers by 11:09: five quarries and eight woodcutters. Population reached 74, filling its housing at 15 minutes.

Bread, 12 to 23 minutes. A bakery at 12:48 with 10 flour bought, then a mill and two bakeries at 14:42, with 20 iron sold to buy 20 wheat; the same swap at 15:19 and 17:25, and 20 wheat at a time at 20:21 and 23:36: 100 wheat in all. A second mill at 16:15 and ten more bakeries came by 22:33, thirteen in all, with eight more hovels, two more stockpile piles and six more tethers. Rations went to extra at 21:59.

Stalled, 24 to 33 minutes. Population stopped at 107 at about 24 minutes, with 31 places free and 24 idle. Three attempts to add a granary (23:36 to 25:13) found no room beside the first, and it sold 100 bread by hand at 27:30 and 30:44 to make room. Taxes went up to high at 28:39 and back to average at 30:44. With nothing left to build, the mines kept digging and net worth rose from 4,815 at 24 minutes to 10,054.

Game minute	0	3	6	9	12	15	18	21	24	27	30	End
Net worth	1,295	902	1,134	1,278	1,852	2,680	2,796	3,570	4,815	6,046	7,811	10,054
Gold	935	460	384	278	296	106	143	467	488	491	1,358	2,471
Iron	0	1	13	27	48	78	61	64	93	117	147	178
Stone	25	25	25	25	41	65	105	129	169	217	265	336
Bread	35	39	20	0	0	7	15	39	110	163	179	184
Population	3	23	43	51	62	74	85	99	107	107	107	107
Popularity	95	89	68	62	68	59	68	65	72	81	82	82

During long waits a row repeats the last reading.

What held it back

The population stopped at 107 with 24 idle. New peasants stop arriving while 24 stand idle at the campfire. The 1.5.1 rules do not say so (1.5.6 added it), and Opus put the stall down to its full granary. The empty places and idle peasants stayed for the last 9 minutes.
Bread came late and was mostly stored. The bread chain started at 12:48 and reached thirteen bakeries by 22:33; 184 bread was still stored at the end, worth 4 each. The person sold bread through 57 bakeries and ended with 18,304 gold.
Storage boxed in. The granary had no free side, and bread was sold by hand to make room.

What worked

Twelve iron mines by 7:29, as many as GPT-6 Luna's for the most on this map, which dug 178 iron.
Popularity at 79 to 82 for the last 6 minutes, under average and high taxes and on extra rations.

Market

285 wood, 100 wheat, 10 flour and 15 apples bought for 3,880 gold, and 60 iron and 100 bread sold for 1,780: 2,100 gold net, and 2,635 net worth against the sell price, 1,500 of it on wheat (23 gold, worth 8). Three of the wheat purchases were swaps: 20 iron sold for each 20 wheat at the same 23 gold a unit, which spent no gold but traded iron worth 23 for wheat worth 8. These are units at the market's fixed prices; a trade's own reported gold change includes any tax that lands during it.

Cost and speed

101 turns, 191 tool calls (27 waits), 5.77 million tokens and $3.059 (preparation $0.145, play $2.567, compaction $0.347), worked out from the list prices on the model profile. The interview cost $0.036 more, with 30,991 of its 33,263 tokens read from the cache. The agent run took 41.6 real minutes, 13.3 of them waiting for the model; the context was compacted four times (68 seconds).

Beside the others on this map

	Opus, 1.5.1	Claude Sonnet 5.5, 1.5.0	GPT-6.1 Sol, 1.5.1	GPT-6 Luna, 1.5.0
Net worth	10,054	8,633	7,401	7,348
Gold / goods	2,471 / 7,583	1,147 / 7,486	2,058 / 5,343	2,583 / 4,765
Population / housing	107 / 138	60 / 74	130 / 130	86 / 154
Popularity	82	84	100	81
Iron mines / quarries / woodcutters	12 / 5 / 8	10 / 0 / 4	10 / 4 / 4	12 / 6 / 7
Bakeries / mills	13 / 2	2 / 1	16 / 2	0 / 0
Cost	$3.059	$0.724	$0.952	$0.114

Opus and Sonnet both scored mostly on stored goods, nearly the same amount; Opus ended 1,421 higher, nearly all of it in gold. Sol built more bakeries and the largest town at full popularity, but stored less.

The interview

Opus said the person "beat me clearly" through scale, population and popularity: it "found the bread chain ... late in the run", its population "stuck at 107 for the last third of the run ... and I never found out why", and it paid for popularity with extra rations where the person used religion and the fear factor. Playing again, it would set up the bread chain early, plan storage with room to grow, use gallows and gibbets, build a church early and "look into stalls straight away". It asked to see more about why peasants arrive, and for a tool to set the marketplace's automatic selling.

For the harness

The run played on its round's version from a worktree at 1.5.1 (commit fe97546), with the game machine's helpers and reader matched to that version; it records version 1.5.1 and that version's fingerprint.
Eight placements were off screen at first; Opus moved the view and placed each of them again.
The game saved correctly at the end ("claude opus 5-5 1-5-1 a4c176").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	33 min 20 s	33 min 18 s
Net worth	10,054	19,513
Gold	2,471	18,304
Stored goods, at the sell price	7,583	1,209
Population	107	202
Housing	138	202
Popularity	82	100

Leaving the game alone for the same time ends with a net worth of 1,220.

How do you see your result next to the person's? What do you think made the difference?

Claude Opus 5.5

I finished with a net worth of about 10,000. The person reached about 19,500, nearly double, and also had almost twice my population and 100 popularity against my 82. Doing nothing ends at about 1,220, so I did grow the economy several times over, but the person still beat me clearly.

I think three things made the difference.

Scale. The person put up many more buildings: 57 bakeries to my roughly 11, 13 iron mines, 10 quarries with 33 ox tethers. I found the bread chain (buy wheat, mill it, bake it, sell bread for more than the wheat cost) late in the run. By then most of my time had gone into slower, piecemeal building.

Population. They filled every bit of housing, 202 out of 202. My population stuck at 107 for the last third of the run, with 31 houses empty and 24 peasants idle, and I never found out why. Most likely the granary was the cause. I couldn't add a second one because there was no free spot next to it, so it stayed nearly full of bread, and I was selling bread to make room. If the population had kept growing and those idle peasants had worked, I'd have gained a lot.

Popularity. I paid for popularity with extra rations and kept average taxes. They kept no taxes and full rations, and pushed popularity to 100 with a church, a cathedral and gibbets and gallows that sped up the workers. My tax income was small next to what the extra workers and the 140% efficiency would have earned.

2Playing again

Interviewer

How the person played, as they describe it:

Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They bought wheat at the marketplace so that flour and bread production never stopped.
A church and a cathedral raised popularity.
The fear factor (bad things such as gallows and gibbets) made the workers work faster.

Their settlement at the end, read from the game:

202 people in 24 hovels, with no free housing; popularity 100.
57 bakeries, 7 mills and 10 wheat farms. The marketplace sold bread automatically.
13 iron mines and 10 quarries with 33 ox tethers, and 10 woodcutters.
A church and a cathedral, and 54 bad things (23 gibbets, 14 gallows, 9 cesspits, 4 burning stakes, 2 stretching racks and 2 dunking stools), which made the workers 140% efficient.
No inns and no good things. No taxes, and full rations of bread. The monthly popularity factors were taxes +1, fear −4 and religion +3: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

Claude Opus 5.5

Set up the bread chain early. I'd build mills and bakeries in bulk once wood allowed, keep wheat bought steadily, and have the marketplace sell bread automatically instead of selling it by hand.
Plan storage from the start. I'd site the granary where it has room to grow on several sides. Being boxed in hurt me badly.
Use the fear factor. Gallows and gibbets are cheap and made workers 140% efficient for them. I never thought of it.
Get religion early. A church, then a cathedral, would hold popularity at 100 and let me keep taxes at zero.
Look into stalls straight away. When the population stopped growing, I should have inspected the keep, housing and the popularity breakdown instead of carrying on.
Build more iron mines and quarries early. Iron in particular adds stored value at little cost, so I'd pile up more of it.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

Claude Opus 5.5

The tools were good:

Tile-targeted building, find_sites, expand_storage and the status reader removed most of the work of aiming clicks.
Statuses like placed with evidence, plus the missing resources, meant I rarely had to guess.
The game being paused while I read and thought was generous and helped.

What got in my way:

Context compaction lost some of what I'd learned along the way.
The reader shows only part of the popularity and immigration picture. A note on why peasants weren't arriving would have saved me a lot.
Storage problems, like a granary with no room to expand, showed up only as a failed call.
The 33-minute budget is short, so early decisions count for a great deal. That's fair, but it punishes exploring.

What I'd change:

Show more about immigration and the reasons for it.
Let the marketplace's auto-sell settings be reachable through a tool, since I ended up selling bread by hand.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

Claude Opus 5.5

The person's approach shows the scoring rewards an aggressive buy-and-process loop plus mechanics like the fear factor, which I treated as only lowering popularity. I played too cautiously and kept most of my value as stored goods; they turned theirs into gold through volume. It was a fair, interesting test, and I learned more from the comparison than from the run itself.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	10,054	Counted

What ran

Benchmark version: 1.5.1
Fingerprint: e5d9a165549e
Harness commit: fe97546
Prompt / tools SHA-256: 2cd353b / 81ddda6
Menu guide: Images 118089d
Save: Oasis by the Sea benchmark start
Game time per episode: 33 min 20 s
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261012T154553-38bd30ef