Learning series of 3 · Oasis by the Sea

Claude Haiku 5.5

Benchmark 1.1.0AnthropicReasoning mediumPlayed by the maintainersEvery episode full budget

Played on benchmark v1.1, an earlier version. Its score is not comparable with games on v1.5, the version the leaderboard shows for Oasis by the Sea.

Model
claude-haiku-5-5
Served via
Anthropic API
Max output
32,768 tokens
Played
9 October 2026
Series score
5,536
net worth at the end of episode 3
Learning
+825
episode 1 → 3: 4,711 → 5,536
Over the baseline
+4,232
4.2× the baseline (do nothing): 1,304
Cost
~$0.31
10.2M tokens over the series; ~ at list prices, the API reports no bill
Time
75 game min
1 h 25 min of wall-clock time, 200 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute of every episode.

Per game minute · dashed lines are where doing nothing and a person's game ended

  • Episode 1
  • Episode 2
  • Episode 3
  • Baseline (do nothing), final

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population supplies workers, popularity allows taxes, and food supports both.

Commentary

Idle Gold

Quote

“Spend gold early and keep spending it. Idle gold is wasted score.”

Model
Claude Haiku 5.5
Source
Its playbook, after episode 3

Claude Haiku 5.5 raised its net worth in every episode, from 4,711 to 5,536, while its playbook warned more firmly each time that the gold it was keeping was wasted.

In episode 1 it spent almost everything. It filled the iron strip north of the keep with mines and finished with 165 iron in store and 400 gold, so 4,311 of its 4,711 net worth was goods. Its playbook named iron the best value at 23 gold a unit.

Episode 2 ended with 2,920 gold and a new rule: idle gold at the end is wasted score. Gold counts in full towards net worth, though, so unspent gold only costs what it could have produced. Episode 3 ended with more of it, 3,760, and the best score of the series. The playbook it wrote afterwards says not to let gold exceed about 500.

Not everything went to plan. In episode 3 the game told it 13 times that farms must be placed on an oasis, and at one point its stockpile reported no room for wood with 32 piles empty. With a minute left it stopped building: nothing it could place would pay back in time.

Episodes

Watch Each Episode

Videos and images stream from the dataset. Game footage © Firefly Studios, shared for non-commercial research.

No video was published for this episode.
Episode 3 · 25 min of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

5,536net worth
Gold
3,760
Stored goods, at sell price
1,776
Over the baseline (do nothing)
+4,232
Population / housing
66 / 66
Popularity
100
Food
105
Structures (map-wide)
92
Attempt
2 (played again)

The settlement at the endcentred on the keep, zoomed out

No final overview was published for this episode.

Final stock1,776 gold in goods

GoodUnitsPriceValue
Gold3,76013,760
Iron3223736
Stone737511
Apples1044416
Wood1131113
Net worth5,536

Tool calls152

  • wait_and_observe41
  • find_sites31
  • build_structure28
  • market_trade17
  • expand_storage8
  • observe8
  • go_to_tile5
  • building_info3
  • map_overview3
  • center_on2
  • inspect_building2
  • place_near2
  • playbook_write2

Play

Turns
75
Buildings placed / attempted
38 / 48
Anchor-tool placements
10
Tool errors
7
Raw inputs sent to the game
450
Context compactions
3
Game time
25 min

Compute

Tokens
3.9M
Served from cache
83%
Output tokens
32K
Cost, at list prices
~$0.12
Wall-clock time
30 min
of which inference
8 min
Peak game memory
6,876 MiB

The plan, before play

Understanding: This is episode 3 of 3 on Oasis by the Sea-1. Score is net worth at the end: gold plus stored goods at marketplace sell prices (iron 23, stone 7, wood 1, food 4, ale and flour 10). Buildings count only through what they produce. Recruiting soldiers and building defences are off-limits. Game time is the 25-minute budget, so I should spend early and avoid idle gold. Episode 2 ended at about 5116, with idle gold of about 2900 as the main waste.

Opening plan (from the playbook):

  1. Place the Granary beside the keep with build_structure ("Granary"). The stockpile is already there.
  2. Place 3 woodcutters at the western grove, next to trees, and stop at 4.
  3. As wood allows (20 each), place iron mines on the north ore strip, aiming for about 10 by 8 minutes. Record each spot.
  4. Add stockpile piles before iron or wood room falls below about 10. Sell iron in batches of 4 lots.
  5. Put apple orchards on the oasis grass south-west of the keep, starting in the first few minutes. Aim for 5 or more so food does not run short.
  6. Spend the starting gold within about 10 minutes on mines, storage and orchards. Build hovels only when idle peasants are near 0.
  7. Keep tax at 3 and rations normal so popularity stays near 100.
  8. Check status and the stats often, and use short waits near the end while still spending gold.

Commentary during play16

  1. Wood room is now 0 even though 32 piles are empty, so the stockpile block isn't accepting wood. I'll sell 10 lots of wood (neutral for net worth) to free room, then try one more stockpile on free ground.

  2. Wood room is back to 60 and gold is 1035. Four peasants are idle, so jobs are the limit. I'll look for woodcutter sites.

  3. Iron is at 30 with room 18, so I'll sell 4 lots now.

  4. Gold is 1495 and sitting idle, with 5 idle peasants. I'll check the map for remaining iron ore and stone.

  5. The quarry at (960,651) is up. I'll add an ox tether beside it and a second quarry on the stone heap.

  6. Wood is down to 24 and gold is 1495 with no idle peasants. Buying wood (4 each) is worth it for quarries that produce stone at 7 each. I'll buy 10 lots and wait for the quarries to start.

  7. Wood room is down to 10 and iron is at 30. I'll sell 4 lots of iron, then try to add storage.

  8. Gold is 1755 and wood is 110, and stone is starting to arrive. Peasants are fully employed, so I'll add two hovels for growth and one more quarry on the stone field.

  9. Two hovels are up and housing is 66. I'll place one more quarry, then let time run.

  10. Stone room is down to 7 and iron is at 33. I'll sell 4 lots of iron and add stockpile space.

  11. Gold is 2305 with 14 idle peasants, and stone room is down to 7. I'll try one more stockpile on the paving edge.

  12. Stone and wood still have room, and gold is the only idle store. About 5.8 minutes remain, so I'll save the lessons from this episode to the playbook now.

  13. Wood room is down to 4 and iron is at 35. I'll sell 4 lots of iron and 10 lots of wood to free room.

  14. Gold is 2840 with about 4 minutes left. The remaining idle peasants have no useful jobs in that time, so I'll let the economy run to the end.

  15. Iron and wood both have room 7, so I'll sell to keep the mines producing, then wait out the rest.

  16. About 1 minute of game time is left, and nothing I could build now would pay back. I'll let the clock run out while mines and quarries keep producing.

Memory

The Playbook

The playbook is the only thing that carries over between episodes: Markdown notes of up to 8 KB that the model edits during play and rewrites after each episode, once it has seen its score.

Oasis by the Sea: playbook (episode 1 result: net worth 4711)

Result and what drove it

  • Final state at about 25:00 game time: gold 400, wood 269, iron 165, stone 25, apples 18, granary food 18. Pop 64/66, idle peasants 24, popularity 100, tax 3.
  • Net worth was about 4711. Iron mines were the main value source. Wood and gold were idle capital.
  • Score levers for episode 2: start mines and storage much earlier, keep food positive, and don't let gold sit unused.

Map facts (Oasis by the Sea-1)

  • Keep top-right of the opening view; stockpile beside it. Granary must be placed first.
  • Iron ore strip just north of the keep, around tile (419,340). Iron mine spots seen at camera (414,346) px: 1152,524 / 1280,461 / 1280,588 / 832,492 / 1376,667 / 1536,238 / 1632,159 / 1792,524 / 1728,429, plus 1152,238 and 1024,111. Each mine cost 20 wood.
  • Western grove of trees for woodcutters. Oasis grass with palms and a pond south-west of the keep (farms, orchards).
  • Apple orchards placed at roughly 1312,349 / 960,492 / 416,445 / 1632,191 (camera-dependent). Orchard spots near tile 453,391 got blocked by trees and buildings quickly.
  • Stone deposit about 50 tiles west of the keep. Stone was barely used: stone stayed at 25 all game, so no quarry was built. That was a mistake only if stone is needed for something; it is worth 7 each.
  • Marketplace placed near camera (1000,480) px.

Build order for episode 2 (targets, to verify)

  1. Granary beside the keep (setup). Stockpile is already present.
  2. Immediately: 3 woodcutters at the western grove. Starting wood is only 50, and it arrives in about 25 s.
  3. Within the first 2 minutes: 4–6 iron mines on the north ore strip (20 wood each). Iron is worth 23 each, the best value here. Mines return the most per wood spent.
  4. Add 2–3 stockpile piles early (expand_storage or build_structure Stockpile next to the paved block). Storage filled at about 18–20 min: iron room hit 0, wood room hit 0.
  5. Apple orchards: 3–4 on the oasis grass, built by about 4–5 min. Food ran down from 54 to 18 across the run.
  6. Hovels only when idle peasants are near zero and jobs exist. Idle peasants stayed around 20–24 all game, so early hovels were wasted.
  7. Spend starting gold (about 480–1000) early on mines, storage and orchards, not after 15 min.

Economy numbers observed

  • Iron rose about 4–7 per 40 s with 8–10 mines; mines take time to staff.
  • Food: 64 people ate roughly 5 food per minute net; 1 orchard was not enough. Apples bought at 8 each (2 lots = 80 gold, 10 food per lot in units of 5).
  • Wood piled to 269 unused. Woodcutters beyond 3–4 are not needed.
  • Pop housing 66 from 4 hovels (hovel = 8 housing, 6 wood each). Popularity held at 100 at tax 3.

Mistakes to avoid

  • expand_storage returned not_placed at the attempt limit; placement retries failed and wasted time. Try build_structure Stockpile on free ground next to the paved pile block, and check storage room in each stats read.
  • Placements with stale pixel coordinates (from an old camera) failed repeatedly. Re-observe after every camera move before placing.
  • find_sites for iron mines returns nothing once the strip is used up; record placed positions to avoid wasted calls.
  • Waiting with gold at 400–480 for the whole run left score unrealised. Gold was not used for production after about 12 min.
  • Long waits (60–120 s) with no action late in the run left iron and wood stopped by full storage. Check room every 40 s and expand before it hits zero.

Ideas to test in episode 2

  • Build a quarry with an ox tether on the western stone deposit only if stone demand justifies it; stone is worth 7, so compare to iron (23 per 20 wood).
  • Consider a marketplace early to sell surplus wood at 1 each only when storage is full; selling is net-neutral, so it only frees room.
  • Consider buying wood early (4 gold per plank, 20 gold per 5) only to speed mines; buying is net-negative, so only do it when it accelerates iron production.

Record

Attempts and Reproducibility

Every attempt4

EpisodeAttemptOutcomeNet worthCounted
11Full-budget score4,711Counted
21Full-budget score5,116Counted
31Infrastructure failure, played again
run error (error); game-time budget not used up; final pause not confirmed; scored from the last reading during play
1,958Not counted
32Full-budget score5,536Counted

A failure outside the model (the harness, game, network or provider) is played again from the same playbook, so it never costs the model an episode. A failure of the model’s own is kept as its result.

What ran

Benchmark version
1.1.0
Fingerprint
19121beba808
Harness commit
2b90076
Prompt / tools SHA-256
6cf94b2 / a3eea35
Menu guide
Images e34faf5
Save
Oasis by the Sea-1
Game time per episode
25 min
Playbook between episodes
Yes
Minimum turn / default wait
8 s / 5 s
Context budget
120,000 tokens
Real-time limit
360 min
Submitted by
hesamation
All files in the dataset 20261009T100849-21b4ea9d