Crusader Bench

Learning series of 3 · Oasis by the Sea

GLM 5.3 Flash

Benchmark 1.0.0Z.aiReasoning mediumPlayed by the maintainersEvery episode full budget

All of GLM 5.3 Flash’s results and comparisons

Played on benchmark v1.0, an earlier version. Its score is not comparable with games on v1.5, the version the leaderboard shows for Oasis by the Sea.

Model
z-ai/glm-5.3-flash
Served via
OpenRouter · z-ai/fp8
Max output
32,768 tokens
Played
8 October 2026
Series score
3,746
net worth at the end of episode 3
Learning
+1,497
episode 1 → 3: 2,249 → 3,746
Over the baseline
+2,442
2.9× the baseline (do nothing): 1,304
Cost
$0.38
7.0M tokens over the series
Time
75 game min
1 h 53 min of wall-clock time, 203 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute of every episode.

Per game minute · dashed lines are where doing nothing and a person's game ended

  • Episode 1
  • Episode 2
  • Episode 3
  • Baseline (do nothing), final

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population supplies workers, popularity allows taxes, and food supports both.

Commentary

Learning to Sell

Quote

“Gold is king: converting goods to gold is safe storage.”

Model
GLM 5.3 Flash
Source
Its playbook, after episode 2

GLM 5.3 Flash doubled its net worth from episode 1 to episode 2 after it started selling goods to keep its stockpile from filling up.

In episode 1 the model built eight iron mines, but its stockpile filled up and most of its value sat there as goods: 1,712 of its 2,249 net worth. In its playbook it noted that a full stockpile stops production and that selling at the marketplace does not reduce net worth. In episode 2 it used the marketplace 27 times and finished at 4,502, nearly all of it in gold.

Episode 3 followed the same plan and finished at 3,746. Food ran short early and the population halved midway. In the last three minutes the model raised taxes to level 8, as its playbook said to, and popularity fell from 93 to 0.

Episodes

Watch Each Episode

Videos and images stream from the dataset. Game footage © Firefly Studios, shared for non-commercial research.

No video was published for this episode.
Episode 3 · 25 min of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

3,746net worth
Gold
3,559
Stored goods, at sell price
187
Over the baseline (do nothing)
+2,442
Population / housing
19 / 82
Popularity
0
Food
6
Structures (map-wide)
44
Attempt
2 (played again)

The settlement at the endcentred on the keep, zoomed out

No final overview was published for this episode.

Final stock187 gold in goods

GoodUnitsPriceValue
Gold3,55913,559
Iron32369
Wood48148
Flour31030
Apples5420
Wheat2816
Cheese144
Net worth3,746

Tool calls74

  • market_trade22
  • build_structure12
  • wait_and_observe9
  • game_action8
  • find_sites5
  • go_to_tile5
  • set_tax5
  • place_near4
  • inspect_building2
  • center_on1
  • expand_storage1

Play

Turns
55
Buildings placed / attempted
25 / 37
Anchor-tool placements
6
Tool errors
2
Raw inputs sent to the game
383
Context compactions
1
Game time
25 min

Compute

Tokens
2.1M
Served from cache
81%
Output tokens
11K
Cost, as billed
$0.11
Wall-clock time
35 min
of which inference
13 min
Peak game memory
4,198 MiB

The plan, before play

I understand the task: grow the Oasis by the Sea-1 economy for 25 game minutes and maximize net worth (gold + stored goods at sell prices). My playbook from episode 2 (≈4370) gives a proven build order, and this episode is my chance to beat it.

Plan (from playbook):

  1. Immediately: granary near keep → 3 woodcutters → marketplace → hovels; buy wood early to unlock mines faster.
  2. All 8 iron mines on the ore strip N of keep by ~min 8; quarry + ox tether on the west deposit.
  3. SE oasis: apple orchards, dairy farms, wheat farm, mill + bakery, chapel; hovels to ~80 housing.
  4. Expand stockpile early.
  5. From ~min 12: sell loop every ~100 s (iron lots, stone, flour, surplus wood); tax ladder 3→4→5.
  6. Final 3 min: no construction; final ~2.5 min tax 8; sell everything sellable at ~45–30 s left.

Commentary during play2

  1. Tax level 8 now per the late-run plan, then let iron accumulate.

  2. Final sell-all now (53 s left):

Memory

The Playbook

The playbook is the only thing that carries over between episodes: Markdown notes of up to 8 KB that the model edits during play and rewrites after each episode, once it has seen its score.

Oasis by the Sea-1 — playbook (ep2, after ep1 score 2249)

Score model (confirmed)

Net worth = gold + stored goods at sell price (wood 1, stone 7, iron 23, food 4, ale/flour 10). Buildings/pop don't count. Selling = neutral; buying = -spread. Taxes are the main gold engine; iron is the main goods engine.

Ep1 result & what capped it

  • Final: gold 537, iron 48, stone 48, wood 76, food 49 → 2249.
  • Iron stuck at 48 and stone at 48 for the last 3+ min: the STOCKPILE FILLs UP (~48–50 per good) and all mines/quarry stop. That wasted the entire endgame. Expand storage (stockpile first, then granary) as soon as iron/stone hit ~40. With 8 iron mines, expect iron value to be the biggest item — give it lots of stockpile space.
  • Gold froze at 537 in the last 3 min because tax was at 3 (no taxes). Popularity was 72 rising. Always leave taxes HIGH at the end — gold is score, popularity doesn't matter in the last minutes. Push tax 5–6 in the final 5 min, 7–8 in the last 1–2 min.

What worked

  • 8 iron mines fit on the ore strip just N of keep (~tiles 408–429, y 329–350), 20 wood each. Rate ~2.8 iron/min total when working (~+65/min score). Build all 8 by minute ~8.
  • Quarry + ox tether on the big stone deposit ~50 tiles W (tile ~(384,387)); 2nd deposit (349,396) is unbuildable — don't retry.
  • Tax ladder: 3 → 4 when popularity ≥ 55 → 3 again when < 53. Net ~+90 gold/min at level 4. Food variety (apple+cheese+bread) feeds popularity +1/+2.
  • Buying wood (4/unit) to unlock iron mines/quarry early is worth it.
  • Oasis sits SE (~tiles 408–491 × 359–425), fits exactly: 3 apple orchards + dairy + 1–2 wheat farms; find_sites then returns nothing more for farms.
  • 3 woodcutters near the W grove; granary by keep; marketplace; mill+bakery near granary AND a 2nd mill+bakery near farms; chapel; 9 hovels (housing 82, pop grows slowly 55→64, idle 20–24 persists — keep adding production to use them).

Mistakes to avoid

  • Don't hoard wood past ~40: idle wood is 1/unit vs iron 23. Spend it on mines/hovels or keep buffer ~25.
  • Watch bread: it slowly drains with full rations; keep 30+ or lower rations, not a market bailout.
  • Don't leave the stockpile full or taxes low at the end.
  • Wheat/flour chain is slow but works (mill+bakery near farms needed); ale chain not viable (no hops space).

Ep2 build order (target: start iron by min 4, all 8 by min 8)

  1. Granary near keep → 3 woodcutters (W grove) → marketplace → 3 hovels.
  2. Iron mines: all 8 on the ore strip (buy wood as needed).
  3. Quarry + ox tether (W deposit); farms on SE oasis (3 apple, dairy, 1–2 wheat); 2nd mill+bakery near farms; chapel; more hovels.
  4. ~min 8+: expand stockpile 2–3 tiles as iron/stone approach 40; expand granary too.
  5. Tax 3 → 4 at popularity ≥ 55; oscillate to hold 52–60.
  6. Final 5 min: tax 5–6 while popularity > 60; final 2 min: tax 8. Keep bread ≥ 25. End with goods+gold, nothing spent on buildings in the last 90 s.

Record

Attempts and Reproducibility

Every attempt4

EpisodeAttemptOutcomeNet worthCounted
11Full-budget score2,249Counted
21Full-budget score4,502Counted
31Infrastructure failure, played again
run error (error); game-time budget not used up; Error: JSON error injected into SSE stream
1,698Not counted
32Full-budget score3,746Counted

A failure outside the model (the harness, game, network or provider) is played again from the same playbook, so it never costs the model an episode. A failure of the model’s own is kept as its result.

What ran

Benchmark version
1.0.0
Fingerprint
f1be10a5cdcc
Harness commit
a48ab47
Prompt / tools SHA-256
28202f4 / a3eea35
Menu guide
Images e34faf5
Save
Oasis by the Sea-1
Game time per episode
25 min
Playbook between episodes
Yes
Minimum turn / default wait
8 s / 5 s
Context budget
120,000 tokens
Real-time limit
360 min
Submitted by
hesamation
All files in the dataset 20261008T191852-c1d986e9