One episode · Oasis by the Sea

Claude Haiku 5.5

Benchmark 1.5.0AnthropicReasoning mediumPlayed by the maintainersFull-budget score

All of Claude Haiku 5.5’s results and comparisons

Score

26.6

Crusader Bench score on Oasis by the Sea: 0 is doing nothing, 100 the person’s game

Net worth

5,113

+3,893 over doing nothing, 21% of the person’s gain of 18,293

Population

29% of the person’s gain of 198 · popularity 67

Cost

~$0.08

3.8M tokens; ~ at list prices, the API reports no bill

Time

33 game min

33 min of wall-clock time, 76 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

Claude Haiku 5.5
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

Commentary

Nine Woodcutters

Quote

“Something's capping growth even though workers are available.”

Model: Claude Haiku 5.5
Source: Its commentary during play, at 26 minutes

Claude Haiku 5.5 was last of the first five games on version 1.5, at 5,113. It built two iron mines and nine woodcutters, kept taxes off for most of the game, and ended with 513 wood in store, worth one gold each.

It opened with four woodcutters, two orchards and three hovels, and bought 50 wood, its only trade. Five quarries made 240 stone, worth 1,680, its largest asset. But at 20 minutes it searched for an iron mine spot with the camera on its quarries, about 35 tiles from the ore, took the empty result as no room, and built a quarry instead; three minutes later a search for hovel spots at the keep listed free spots on iron ore, and it built hovels on them.

With taxes off and nothing sold, gold stood at 800 for about ten minutes and at 1,015 for five. At 25 minutes it set high taxes; popularity fell from 84 to 43 and some peasants left, so it removed the tax and paid a large bribe for a minute. It named the late tax its worst mistake.

It was the fastest and cheapest of the five: 76 turns in 33 real minutes, for $0.08 at Anthropic's list prices. At 25 game minutes, the budget of the earlier versions, it stood at 3,523, below each of the three episodes it played on version 1.1.

The episode

Watch the Episode

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

5,113net worth

Gold: 1,482
Stored goods, at sell price: 3,631
Over the baseline (do nothing): +3,893
Population / housing: 61 / 82
Popularity: 67
Food: 49
Structures (map-wide): 73
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock3,631 gold in goods

Good	Units	Price	Value
Gold	1,482	1	1,482
Stone	240	7	1,680
Iron	54	23	1,242
Wood	513	1	513
Apples	49	4	196
Net worth			5,113

Beside a person’s gameOasis by the Sea, the same game time

	This episode	The person	Left alone
Net worth	5,113	19,513	1,220
Gold	1,482	18,304
Stored goods, at the sell price	3,631	1,209
Population	61	202
Housing	82	202
Popularity	67	100

The person played Oasis by the Sea for 33 min 18 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: October 1200
Idle peasants: 24
Popularity next month: +4
Food: +1
Taxes: +1
Religion: +2
Taxes: No taxes, 0 gold a month
Rations: full; 2 foods eaten
Worker efficiency: 100%
Storage: 24 stockpile piles, 6 empty; 1 granary holding 49 food

Buildings43

Hovel9
Woodcutter9
Stockpile6
Quarry5
Apple Orchard4
Ox Tether3
Iron Mine2
Chapel1
Dairy Farm1
Granary1
Keep1
Marketplace1

Tool calls136

wait_and_observe43
find_sites32
build_structure24
go_to_tile12
set_tax7
observe4
map_overview3
place_near3
center_on2
expand_storage2
set_rations2
inspect_building1
market_trade1

Play

Turns: 76
Buildings placed / attempted: 31 / 37
Anchor-tool placements: 8
Tool errors: 1
Raw inputs sent to the game: 235
Context compactions: 3
Game time: 33 min

Compute

Tokens: 3.8M
Served from cache: 93%
Output tokens: 25K
Cost, at list prices: ~$0.08
Wall-clock time: 33 min
of which inference: 7 min
Peak game memory: 3,207 MiB

The plan, before play

I understand the task. The aim is to grow the Oasis by the Sea settlement's economy and end the run with the highest combined net worth, population and popularity. Net worth counts gold plus stored goods at marketplace sell prices. Weapons, walls, towers and recruiting soldiers are off limits. Game time is the budget, about 33 minutes.

Opening plan:

Place the required Granary first, next to the existing stockpile.
Build 2–3 woodcutters beside the western grove, close to the stockpile.
Build hovels for housing and food sources on the oasis grass south-west of the keep: apple orchards and dairy farms, which feed the granary directly.
Set up a quarry and iron mine (iron ore lies just north of the keep) with an ox tether for stone.
Keep taxes at no-tax or low while popularity is near 50. Watch housing, idle peasants and storage. Check with status and get_inventory after each construction burst, and expand storage before it fills.
Use the marketplace only for cheap, productive buying, and keep the economy growing until the clock runs out.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: Claude Haiku 5.5 (claude-haiku-5-5) on Anthropic's API, reasoning medium, output limit 32,768 tokens
Benchmark: Oasis by the Sea construction, version 1.5.0, from the save the human reference game started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
Run 70634a1c, series 20261011T141021-5ef84787, 2026-10-11

Result

	Claude Haiku 5.5	The person	Left alone
Net worth	5,113	19,513	1,220
Gold	1,482	18,304
Stored goods, at the sell price	3,631	1,209
Population / housing	61 / 82	202 / 202
Popularity	67	100

A full-budget score: the run used all 2,000 game seconds. Net worth grew by 3,893 over leaving the game alone, about a fifth of the person's growth (18,293). Most of it was stored goods, not gold: 240 stone, 54 iron, 513 wood and 49 apples.

How the game went

Opening, 0 to 4 minutes. Four woodcutters, two apple orchards and three hovels, the first iron mine at 3 minutes, a marketplace, and 50 wood bought, its only trade of the game. Housing stayed at 34 from 3 to 8 minutes with every place taken, and it put rations to half for two minutes when food ran low.

Building out, 4 to 12 minutes. Two quarries with ox tethers, a dairy farm, two more orchards, more woodcutters, a second iron mine at 9 minutes, and three hovels against the keep at 8 minutes. Housing reached 58 and population 50 at 12 minutes.

Slowing down, 12 minutes to the end. Three more quarries (to five), three more hovels and a chapel, and otherwise waiting: 43 of its 136 tool calls were waits. Taxes were off for most of the game, so gold stood still at 800 for about ten minutes and at 1,015 for about five. At 25 minutes it set high taxes; popularity fell from 84 to 43 by 30 minutes and some peasants left, so it removed the tax, paid a large bribe for a minute and ended at 67.

Game minute	0	6	9	12	15	18	21	24	27	30	End
Net worth	1,270	1,185	1,380	1,642	2,061	2,708	2,984	3,318	3,906	4,728	5,113
Population	3	34	34	50	54	54	56	61	61	60	61
Housing	10	34	34	58	58	66	66	66	82	82	82

What held it back

The population stopped near 61 with 24 peasants idle. It reached 24 idle at 10 minutes and grew slowly from there, though housing had room and popularity was mostly above 60. GLM 5.3 Flash's run on the same save the same day stopped at 24 idle too, which fits a limit on how many peasants wait at the campfire: more people seem to come only when the idle ones get work. Haiku noticed at 26 minutes that "something's capping growth even though workers are available" but did not find what.
Only two iron mines. Iron sells at 23, the best price of anything it made. At 9 minutes a search near the keep offered two iron mine spots and it built one. At 20 minutes it planned another "near the ore deposit by (421,350)" but searched while the camera was on its quarries, about 35 tiles away; the empty result said nothing fit in that view and pointed to the map overview, and Haiku built a quarry instead. At 23 minutes a search for hovel spots at the keep listed free spots on iron ore, and it built hovels.
Workers went into wood. Nine woodcutters filled the stockpile with 513 wood, worth 1 each; it expanded storage twice for it and never sold or used most of it.
Little gold. With taxes off for most of the game and nothing sold, gold ended at 1,482; the late high tax cost popularity and people.
One food. Apples with a little cheese, so food variety added nothing to popularity for most of the game.

What worked

It hardly bought anything: one purchase of 50 wood, which lost 150 net worth.
Five quarries made 240 stone, worth 1,680, its largest single asset.
It was fast and cheap: the 33 min 20 s of game time took 33.3 real minutes.

Cost and speed

76 turns, 136 tool calls, 3.82 million tokens and $0.080, worked out from the list prices on the model profile (Anthropic's API does not report what it bills). The agent run took 33.3 real minutes, 6.8 of them waiting for the model; three compactions took 23 s in all.

At 25 game minutes, the length of the earlier benchmark versions, it stood at 3,523 net worth for $0.075 after 26.8 real minutes. The three published Haiku episodes of version 1.1, on the same endpoint and settings, ended at 4,711, 5,116 and 5,536 after their 25 minutes (5,121 on average): this run's final 5,113, eight minutes later, matches that average. The 1.1 episode 1 built 13 iron mines and held 165 iron at the end, against 2 mines and 36 iron here at the same game time; it cost $0.097 and took 28.7 real minutes to reach 25 game minutes.

Beside GLM 5.3 Flash on the same save

	Claude Haiku 5.5	GLM 5.3 Flash
Net worth	5,113	6,200
Ending	Full budget	Ended itself 5 s early
Gold / goods	1,482 / 3,631	4,755 / 1,445
Population / housing	61 / 82	64 / 210
Iron mines / quarries / woodcutters	2 / 5 / 9	7 / 2 / 11
Net worth lost to purchases	150	1,201
Cost	$0.080	$0.145
Real time	33.3 min	43.8 min

GLM spent hard, bought food and sold what it made; Haiku built less, bought almost nothing and kept what it made. Both hit the same wall of 24 idle peasants.

The interview

Haiku put the gap down to an economy that "never compounded": goods sat in the stockpile while the person sold bread made from wheat. Playing again, it would start a wheat, mill and bakery chain in the first ten minutes, buy wheat when the chain stalled, build a church early and plan housing and jobs together. It named the late high tax as its worst mistake. One detail is off: it said its stored goods were mostly wood, but by value they were mostly stone and iron.

For the harness

Site search covers only the current view. Haiku searched for iron mine spots from the wrong part of the map and took the empty result as "no room", and six placements failed with not_on_screen.
In the interview it said pixel positions in build results sometimes did not match the tiles; not checked here.

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	33 min 20 s	33 min 18 s
Net worth	5,113	19,513
Gold	1,482	18,304
Stored goods, at the sell price	3,631	1,209
Population	61	202
Housing	82	202
Popularity	67	100

Leaving the game alone for the same time ends with a net worth of 1,220.

How do you see your result next to the person's? What do you think made the difference?

Claude Haiku 5.5

My result is far behind the person's. I finished with a net worth of 5,113 against their 19,513, and population 61 against 202. My gold was 1,482 against their 18,304. I did slightly better than leaving the game alone (1,220), but that is a weak bar.

The gap comes from the economy never compounding. I built quarries, iron mines, woodcutters and apple orchards, but the goods they made sat in the stockpile or were valued at low sell prices (wood at 1 each). I held about 3,600 in stored goods, mostly wood, against their 1,200. Their gold came from selling bread at the marketplace, a chain that turns cheap wheat into a product that sells for gold. I never built a mill, bakery or wheat chain. Housing was also my bottleneck. I kept 24 idle peasants for most of the game, and my population stayed near 61 even with housing at 82. I did not work out why growth stalled, and I did not fix food variety early enough to keep popularity steady.

2Playing again

Interviewer

How the person played, as they describe it:

Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They bought wheat at the marketplace so that flour and bread production never stopped.
A church and a cathedral raised popularity.
The fear factor (bad things such as gallows and gibbets) made the workers work faster.

Their settlement at the end, read from the game:

202 people in 24 hovels, with no free housing; popularity 100.
57 bakeries, 7 mills and 10 wheat farms. The marketplace sold bread automatically.
13 iron mines and 10 quarries with 33 ox tethers, and 10 woodcutters.
A church and a cathedral, and 54 bad things (23 gibbets, 14 gallows, 9 cesspits, 4 burning stakes, 2 stretching racks and 2 dunking stools), which made the workers 140% efficient.
No inns and no good things. No taxes, and full rations of bread. The monthly popularity factors were taxes +1, fear −4 and religion +3: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

Claude Haiku 5.5

I would start the bread chain much earlier, within the first ten game minutes. That means a wheat farm on the oasis grass, a mill, a bakery and a marketplace to sell the bread. I would take the oasis grass around the keep for wheat instead of apple orchards, because the one-food problem hurt popularity and the variety factor was zero for most of the run. I would build a church early for the religion bonus. I would also use the marketplace to buy wheat when the chain stalled, so production never sat idle. Finally, I would plan housing and jobs together, so that no peasants stood idle.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

Claude Haiku 5.5

The reading tools were useful. The stats, the stockpile piles and the find_sites results gave me enough to act on, and the screenshots confirmed placements. Some things got in the way. Pixel coordinates from build results sometimes did not match the tiles, so I relied on tile targets and cost deductions. Tile targets off screen needed an extra go_to_tile. Waiting in fixed chunks made it hard to react between production cycles. A clearer popularity breakdown, including what drops the religion factor, would have helped a lot. The time limit felt right for the task.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

Claude Haiku 5.5

I made a serious mistake in the last stretch. I left taxes at high for too long, and popularity fell to 43, which made peasants leave. I then had to buy popularity with a large bribe. Watching the popularity trend every few game seconds, not just the stats at turn ends, would have prevented that. I also did not check the population stall early enough.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	5,113	Counted

What ran

Benchmark version: 1.5.0
Fingerprint: 31b5ff59f7a5
Harness commit: ee862eb
Prompt / tools SHA-256: 2cd353b / 81ddda6
Menu guide: Images 118089d
Save: Oasis by the Sea benchmark start
Game time per episode: 33 min 20 s
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261011T141021-5ef84787