Crusader Bench

Learning series of 3 · Oasis by the Sea

GLM 5.3 Flash

Benchmark 1.1.0Z.aiReasoning mediumPlayed by the maintainersEvery episode full budget

All of GLM 5.3 Flash’s results and comparisons

Played on benchmark v1.1, an earlier version. Its score is not comparable with games on v1.5, the version the leaderboard shows for Oasis by the Sea.

Model
z-ai/glm-5.3-flash
Served via
OpenRouter · z-ai/fp8, siliconflow/fp8
Max output
32,768 tokens
Played
9 October 2026
Series score
5,495
net worth at the end of episode 3
Learning
+747
episode 1 → 3: 4,748 → 5,495
Over the baseline
+4,191
4.2× the baseline (do nothing): 1,304
Cost
$0.51
8.9M tokens over the series
Time
75 game min
1 h 46 min of wall-clock time, 204 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute of every episode.

Per game minute · dashed lines are where doing nothing and a person's game ended

  • Episode 1
  • Episode 2
  • Episode 3
  • Baseline (do nothing), final

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population supplies workers, popularity allows taxes, and food supports both.

Commentary

Higher Than It Thought

Quote

“Estimated NW ≈ 4,600–4,800.”

Model
GLM 5.3 Flash
Source
Its playbook, after episode 3, which scored 5,495

GLM 5.3 Flash finished its series at 5,495, 41 behind Claude Haiku 5.5. After two of its three episodes it put its own result well below the final count, and its last lesson was about an iron limit the final count does not show.

After episode 1 it added up its gold and goods and wrote that the total was about 4,300. The figures it listed sum to 4,695; the final count read from the game was 4,748. After episode 2, which ended at 4,262, it wrote that both episodes had come to about 4,300, though the first was 486 higher.

Episode 3 was its best. Its last reading came with two minutes left, at 828 gold and 134 iron, and it estimated its net worth at 4,600 to 4,800. The game ran on while it waited, and it finished with 1,042 gold and 160 iron, which alone is worth 3,680 at 23 gold a unit.

The playbook it wrote after that episode leads with a warning that iron storage stops at about 144, and estimates that over 100 iron was lost to it. The episode ended with 160 iron in store. The number it was watching, room left in the stockpile, rises again when a new pile is started, which it had noticed in episode 1: “Iron 50, room 46 — spread into new pile. Good.”

Two of its episodes were cut short by failures outside the model and played again from the same playbook. Only the completed episodes count.

Episodes

Watch Each Episode

Videos and images stream from the dataset. Game footage © Firefly Studios, shared for non-commercial research.

No video was published for this episode.
Episode 3 · 25 min of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

5,495net worth
Gold
1,042
Stored goods, at sell price
4,453
Over the baseline (do nothing)
+4,191
Population / housing
73 / 74
Popularity
71
Food
26
Structures (map-wide)
82
Attempt
2 (played again)

The settlement at the endcentred on the keep, zoomed out

No final overview was published for this episode.

Final stock4,453 gold in goods

GoodUnitsPriceValue
Gold1,04211,042
Iron160233,680
Stone767532
Wood97197
Bread22488
Flour41040
Apples4416
Net worth5,495

Tool calls101

  • find_sites22
  • build_structure20
  • market_trade11
  • game_action10
  • go_to_tile9
  • wait_and_observe9
  • center_on4
  • place_near4
  • expand_storage3
  • set_tax3
  • update_plan2
  • go_to_view1
  • inspect_building1
  • map_overview1
  • save_view1

Play

Turns
76
Buildings placed / attempted
35 / 48
Anchor-tool placements
12
Tool errors
8
Raw inputs sent to the game
383
Context compactions
2
Game time
25 min

Compute

Tokens
3.4M
Served from cache
79%
Output tokens
17K
Cost, as billed
$0.20
Wall-clock time
36 min
of which inference
14 min
Peak game memory
3,234 MiB

The plan, before play

I understand: grow the Oasis by the Sea free-build economy for 25 minutes of game time; final net worth = gold + stored goods at sell prices (iron 23, flour/ale 10, stone 7, food 4, wood 1). No walls/soldiers/weapons. My episode-2 playbook carries over with a proven build order.

Opening plan (from playbook):

  1. Place granary immediately via setup tray near the keep.
  2. 2–3 woodcutters in the palm grove SW of keep; buy ~50–100 wood early (wood is the #1 bottleneck).
  3. Hovels (~6), marketplace, tax level 4.
  4. Apple orchard + dairy farm on oasis grass immediately (via find_sites).
  5. All 8 iron mines on the strip N of keep ASAP — iron at 23/unit is the biggest lever.
  6. Quarry + 2 ox tethers on the west stone deposit (~tile 384,387); skip the pebble field.
  7. 2 wheat farms, mill, 2–3 bakeries; chapel; expand stockpile early.
  8. Monitor popularity (revert tax to 3 if it dips), storage room, and idle peasants; build more hovels/jobs continuously.
  9. Last ~5 min: stop building, keep production un-stalled by selling goods with room <5, just let iron and taxes accumulate.

Target: net worth > 5,000.

Commentary during play3

  1. Abandoning the ration hunt (clicks keep hitting hovels; not worth more time). Moving to food production and housing instead.

  2. Iron 102, gold 640, popularity 68. Entering endgame hold: no more building, just let mines and taxes accumulate.

  3. Stockpile expanded to 32 piles; iron still shows room 10 but with 24 empty piles new iron deliveries should fit. Popularity 67 rising, food 39. Just waiting out the final ~2 minutes.

Memory

The Playbook

The playbook is the only thing that carries over between episodes: Markdown notes of up to 8 KB that the model edits during play and rewrites after each episode, once it has seen its score.

Oasis by the Sea — playbook for ep2

Final result ep1 (Feb 1199, 25:00)

Gold 1244; stockpile: wood 217, stone 81, iron 101, flour 30; granary food 11; pop 67/138, popularity 64. Rough NW at sell prices: 1244 + 217·1 + 81·7 + 101·23 + 30·10 + food 11·4 ≈ ~4,300. Iron alone ≈ 2,300 — iron is THE lever. Bread/apples ended tiny: food chain was the weak spot; wheat kept running out (types_eaten dropped 3→2, popularity slipping late).

Build order that worked (repeat this)

  1. Granary immediately (setup tray), near keep.
  2. 2–3 woodcutters in the palm grove SW of keep. Buy 50–100 wood @4/u early to remove the wood bottleneck — pays off.
  3. Hovels early (8 housing each; build ~6+). Idle peasants 20+ all game = always build more jobs too.
  4. Apple orchard + dairy farm on oasis grass IMMEDIATELY — grass is scarce and exhausts; use find_sites, never eyeball.
  5. Iron mines on ore strip just N of keep (~tiles 408–429 / 329–350 plus east extension). 8 fit; iron sell 23/u = best good. No tether needed (miners carry).
  6. Quarry + 2 ox tethers on west stone deposit (~tile 384,387); stone 7/u, slow. Second deposit ~349,396 unused in ep1 — grab it if labour allows.
  7. Marketplace early. Tax: start 4 (low, -2 pop), go to 5 once chapel stands; ~+50 gold/mo.
  8. Mill + bakeries (need ~3, place via find_sites near granary; place_near granary often fails "no_free_spot_in_view").
  9. Expand stockpile EARLY with expand_storage (must touch existing; got 32 piles in ep1, still nearly full at end). Watch storage.room — production stops silently when full.
  10. Chapel near keep (+4 religion).

What to change in ep2 (would have raised score)

  • Food was the ceiling: wheat farms kept starving the chain (wheat 0 most of the time). Build 2+ wheat farms as soon as grass allows, plus mill+bakeries from the start, so bread (4/u) and popularity don't sag. Food variety lost = popularity drop late.
  • Wood ended at 217 with room left: woodcutters overproduce at 1/u. Once wood room is large, they're the lowest-value producers — fine to have 3, don't build more.
  • Iron: 8 mines was right; 101 iron in 11 months ≈ 9/mo. If labour idles, add mines in the eastern ore extension — every mine pays ~200/u over a game.
  • Consider selling nothing; goods in stock count at sell price. Selling = NW unchanged, buying = net loss (buy prices 2x sell). Buy ONLY wood early when it unblocks buildout.
  • Keep tax at 5; only 6 if popularity >74 and stable — revert immediately if the trend arrow goes negative.

Pitfalls (all verified)

  • Weapons workshops/armory, walls, towers, soldiers: zero or negative NW — skip entirely.
  • Building costs come out of goods: bakery 10w, woodcutter 3w; bursts drain wood fast.
  • Never press P / Escape / Space; right-click exits placement mode.
  • After any camera move, re-observe; take targets only from the latest screenshot.
  • place_near/expand_storage return text only — confirm on next screenshot.
  • When all grass is used and all chains are staffed, the economy plateaus: just wait_and_observe to the end. Iron/stone/wood keep ticking into the stockpile and each unit adds NW.

Map memory

  • Keep top-right of opening view; stockpile beside it; campfire below keep.
  • Oasis grass + pond SW of keep (farms + palms); tree grove further west; ore strip N of keep; stone ~50 tiles W.
  • Prices (sell): iron 23, flour 10, stone 7, food 4, wood 1.

Record

Attempts and Reproducibility

Every attempt5

EpisodeAttemptOutcomeNet worthCounted
11Full-budget score4,748Counted
21Infrastructure failure, played again
run error (error); game-time budget not used up; final pause not confirmed; scored from the last reading during play
1,753Not counted
22Full-budget score4,262Counted
31Infrastructure failure, played again
run error (error); game-time budget not used up; final pause not confirmed; scored from the last reading during play
6,678Not counted
32Full-budget score5,495Counted

A failure outside the model (the harness, game, network or provider) is played again from the same playbook, so it never costs the model an episode. A failure of the model’s own is kept as its result.

What ran

Benchmark version
1.1.0
Fingerprint
e6391d450505
Harness commit
05376cf
Prompt / tools SHA-256
6cf94b2 / a3eea35
Menu guide
Images e34faf5
Save
Oasis by the Sea-1
Game time per episode
25 min
Playbook between episodes
Yes
Minimum turn / default wait
8 s / 5 s
Context budget
120,000 tokens
Real-time limit
360 min
Submitted by
hesamation
All files in the dataset 20261009T122236-3aae6fc4