Crusader Bench

One episode · Large Barren Desert

GLM 5.3 Flash

Benchmark 1.5.6Z.aiReasoning mediumPlayed by the maintainersFull-budget score

All of GLM 5.3 Flash’s results and comparisons

Model
z-ai/glm-5.3-flash
Served via
OpenRouter · z-ai/fp8, siliconflow/fp8
Max output
32,768 tokens
Played
12 October 2026
Score
33.7
Crusader Bench score on Large Barren Desert: 0 is doing nothing, 100 the person’s game
Net worth
6,655
+1,463 over doing nothing, 31% of the person’s gain of 4,792
Population
70
27% of the person’s gain of 246 · popularity 66
Cost
$0.11
2.6M tokens
Time
33 game min
39 min of wall-clock time, 62 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

  • GLM 5.3 Flash
  • Baseline (do nothing), final
  • A person's game, final

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Videos and images stream from the dataset. Game footage © Firefly Studios, shared for non-commercial research.

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

6,655net worth
Gold
4,560
Stored goods, at sell price
2,095
Over the baseline (do nothing)
+1,463
Population / housing
70 / 98
Popularity
66
Food
86
Structures (map-wide)
68
Attempt
1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock2,095 gold in goods

GoodUnitsPriceValue
Gold4,56014,560
Stone14871,036
Iron1723391
Ale2010200
Apples324128
Cheese294116
Meat254100
Pitch1010100
Wood24124
Net worth6,655

Beside a person’s gameLarge Barren Desert, the same game time

This episodeThe personLeft alone
Net worth6,6559,9845,192
Gold4,5608,574
Stored goods, at the sell price2,0951,410
Population70250
Housing98250
Popularity66100

The person played Large Barren Desert for 33 min 33 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game
July 1178
Idle peasants
24
Popularity next month
±0
Food
+2
Taxes
−2
Taxes
Low taxes, 42 gold a month
Rations
full; 3 of 3 foods eaten
Worker efficiency
100%
Storage
12 stockpile piles, 3 empty; 1 granary holding 86 food

Buildings45

  • Ox Tether17
  • Hovel11
  • Quarry7
  • Iron Mine4
  • Stockpile3
  • Granary1
  • Keep1
  • Marketplace1

Tool calls103

  • market_trade37
  • build_structure18
  • find_sites12
  • go_to_tile10
  • wait_and_observe9
  • place_near4
  • set_tax4
  • building_info2
  • notebook_write2
  • center_on1
  • expand_storage1
  • map_overview1
  • notebook_read1
  • status1

Play

Turns
62
Buildings placed / attempted
34 / 43
Anchor-tool placements
8
Tool errors
2
Raw inputs sent to the game
555
Context compactions
1
Game time
33 min

Compute

Tokens
2.6M
Served from cache
93%
Output tokens
13K
Cost, as billed
$0.11
Wall-clock time
39 min
of which inference
13 min
Peak game memory
3,229 MiB

The plan, before play

Understanding of the task: Free-build Large Barren Desert for ~33 game minutes. Score = net worth (gold + stored goods at sell prices) most, then population and popularity. No farms (no oasis grass), no military. Stone (quarries+ox tethers, ~100 tiles west) and iron (~60 tiles east) are the main value producers; taxes on a growing population add gold; trading converts goods to gold at no net-worth loss. Starting package: 5,000 gold, wood, stone, iron, pitch, meat, ale.

Opening plan:

  1. Confirm starting state (status/map overview), let starting package finish arriving.
  2. Build a Marketplace early (needed for trading), a few Hovels for housing.
  3. Go to stone deposits: place quarries + ox tethers; then iron mines east. Selling stone (7) and iron (23) is the core net-worth engine.
  4. Raise taxes modestly as popularity allows; keep rations at full; expand housing as idle peasants take jobs.
  5. Keep wood supply up (woodcutters near trees) for construction; watch stockpile room and expand storage when needed.
  6. Sell surplus continuously; keep building to the last minute.

Commentary during play2

  1. No trees and no farmland — all wood must be bought; iron (23/unit) and stone (7/unit) are the value engines. Let me scout the iron area.

  2. Food ran out — buying urgently.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

  • Model: GLM 5.3 Flash (z-ai/glm-5.3-flash) on OpenRouter, providers z-ai/fp8 then siliconflow/fp8, reasoning medium, output limit 32,768 tokens
  • Benchmark: Large Barren Desert construction, version 1.5.6, from the save the human reference game on this map started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
  • Run fafe0ff5, 2026-10-12

Result

GLM 5.3 Flash The person Left alone
Net worth 6,655 9,984 5,192
Gold 4,560 8,574
Stored goods, at the sell price 2,095 1,410
Population / housing 70 / 98 250 / 250
Popularity 66 100

A full-budget score: the run used all 2,000 game seconds. Net worth grew by 1,463 over leaving the game alone, about 30% of the person's growth (4,792). About 1,150 of that is the part of the starting package that waits for storage room, which GLM made room for at 67 seconds. Its quarries and mines produced about 290 stone and 80 iron (some 3,900 at the sell price), and buying food and wood took 2,890 of it back.

How the game went

Mines first, 0 to 2 minutes. Two iron mines at 16 seconds, a marketplace at 30, three hovels, two more stockpile piles at 67 seconds, two more iron mines at 97 and two quarries with ox tethers beside them by about 2 minutes.

Quarries and food, 2 to 18 minutes. Three more quarries at about 7 minutes and two at 15, each with tethers beside them, eleven hovels in all, and from about 4 minutes three foods bought every few minutes (meat, cheese and apples) with iron and stone sold as they came in. The population rose to 62 by 18 minutes with popularity held at 52.

Holding on, 18 to 33 minutes. Taxes moved between none and low; popularity dipped to 38 at 24 minutes and recovered to 66. The population stopped at 70 with 24 idle. Its last buildings were three hovels at 18 minutes and three more ox tethers at 29.

Game minute 0 3 6 9 12 15 18 21 24 27 30 End
Net worth 4,546 6,405 6,299 6,041 6,011 6,287 5,909 5,915 6,028 6,028 6,352 6,655
Population 2 21 35 44 53 55 62 66 60 60 67 70
Popularity 100 91 72 52 52 52 52 52 38 38 66 66

During long waits the table repeats the last reading.

What held it back

  • Food bought for variety, at a loss. No farm can be built here. Twenty-one purchases of meat, cheese and apples lost 2,440 net worth against the sell price, and wood 450 more: 2,890 in all, against 1,200 in its 1.5.4 run, which bought only meat. Three foods kept the food factor at +2.
  • 24 idle peasants at the end. The population stopped at 70 with 28 places free.
  • Stone was slow to come. Seven quarries and seventeen tethers, but the stone lies about 100 tiles from the stockpile; they brought about 290 stone in 30 minutes.
  • Nothing for popularity but food and taxes: no inn, chapel or church this time.
  • Eight placements failed because their tiles were off screen.

What worked

  • Four iron mines and seven quarries, all seventeen tethers 4 to 9 tiles from a quarry, as the 1.5.6 prompt says.
  • Fourteen sales of iron, stone and pitch brought 2,990 gold; selling leaves net worth unchanged and kept the stockpile from filling.
  • It kept notes (notebook_write) on the map, costs and prices, and read them back.

Cost and speed

62 turns, 103 tool calls (37 trades, 9 waits), 2.56 million tokens and $0.107 (billed by OpenRouter); the interview cost $0.002 more. The agent run took 39.5 real minutes, 12.6 of them waiting for the model; compaction took 28 s. The waiting reminder appeared twice.

Beside its run on 1.5.4 and Haiku's on 1.5.6

GLM, 1.5.6 GLM, 1.5.4 Claude Haiku 5.5, 1.5.6
Net worth 6,655 8,232 6,412
Gold / goods 4,560 / 2,095 7,496 / 736 3,428 / 2,984
Population / housing 70 / 98 47 / 90 32 / 42
Popularity 66 49 51
Iron mines / quarries / ox tethers 4 / 7 / 17 4 / 3 / 4 4 / 0 / 0
Inn / chapel 0 / 0 1 / 1 0 / 0
Net worth lost to purchases 2,890 1,200 1,000
Cost / real time $0.107 / 39.5 min $0.062 / 32.9 min $0.034 / 29.4 min

On 1.5.4 GLM bought only meat, sold everything it mined and ended with high taxes and half rations, which took popularity to 49. On 1.5.6 it built more and grew a larger town, but kept three foods bought for popularity and taxes low, and its net worth ended 1,577 lower; population was 23 higher and popularity 17 higher.

The interview

GLM called its run "essentially a mine-and-sell economy" with "a hard ceiling", and said "I optimized a small, safe machine when the scoring rewarded a large one". Playing again, it would buy wheat for mills and bakeries, ale for inns, build a church and fear buildings, and "stop selling raw stone/iron except as overflow". It said the rules' framing around raw materials and "farms need oasis grass" anchored it, and that it never checked whether wheat could simply be bought.

For the harness

  • The game saved correctly at the end ("glm 5-3 flash 1-5-6 fafe0f").
  • Placing on tiles off screen still fails (not_on_screen eight times).

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

You The person
Game time played 33 min 20 s 33 min 33 s
Net worth 6,655 9,984
Gold 4,560 8,574
Stored goods, at the sell price 2,095 1,410
Population 70 250
Housing 98 250
Popularity 66 100

Leaving the game alone for the same time ends with a net worth of 5,192.

How do you see your result next to the person's? What do you think made the difference?

GLM 5.3 Flash

I ended at 6,655 net worth, 70 population, popularity 66 — decent, but well behind the person's 9,984 / 250 / 100. My run was essentially a mine-and-sell economy: quarries, iron mines, and ox tethers producing stone and iron to sell, plus taxes, with all food bought at the marketplace. That works but has a hard ceiling: production was transport-limited (iron flows of only 2–4/month through walking miners), immigration stalled at 24 idle peasants with no job growth, and food bought at 8/unit while only worth 4 was a permanent net-worth leak that I had to keep paying to keep popularity alive.

2Playing again

Interviewer

How the person played, as they describe it:

  • Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
  • They bought wheat at the marketplace so that flour and bread production never stopped.
  • A church and a cathedral raised popularity, and so did inns.
  • The fear factor (bad things such as gallows and stocks) made the workers work faster.

Their settlement at the end, read from the game:

  • 250 people in 30 hovels, with no free housing; popularity 100.
  • 95 bakeries and 12 mills, and no wheat farms.
  • 11 inns, which reached everyone with ale, and no brewery or hops farm.
  • 13 quarries and 7 iron mines with 37 ox tethers; no woodcutters.
  • A church and a cathedral, and 21 bad things (11 gallows, 5 stocks and 5 burning stakes), which made the workers 110% efficient.
  • Low taxes and no rations. The monthly popularity factors were food −8, taxes −2, fear −1, religion +3 and inns +8: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

GLM 5.3 Flash

The person's strategy exposed what I missed:

  1. Bread as an industry. Buying wheat (23/u) and turning it into flour (10/u) and bread (10/u sell, but huge volume) is a conversion business: the buildings create jobs (which unblocks immigration — the key to their 250 population) and gold flows through taxes at scale. I dismissed food processing because there were no farms; I never checked whether wheat could simply be bought and milled. That was my biggest analytical failure — I modeled trade only as sell-raw-materials, not as buy-input/make-product.

  2. Popularity at 100 as a population engine. Their church + cathedral (+3), 11 inns buying ale (+8), and low taxes kept popularity pinned at 100, so peasants flooded in. I kept popularity merely stable at ~52–66 and treated food variety as the only lever. Ale bought at 20 vs. 10 sell value looked like a loss to me, but it bought +8 popularity, which bought population, which bought taxes and workers — clearly worth it.

  3. Fear factor. 21 punishment buildings for −1 popularity bought 110% worker efficiency — a pure production multiplier I never used because "bad things" reduce popularity and I was hoarding popularity instead of spending it where its marginal value was low (100 caps anyway).

  4. Fear of buying. I over-indexed on the note that "buying lowers net worth by the spread." In practice buying inputs to feed a bigger machine (bread, ale, workers) grows net worth faster than hoarding. Net worth is a snapshot, not a bank balance.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

GLM 5.3 Flash

The tools were generally strong: the read-only reader, tile-targeted placement, find_sites, and market_trade saved enormous amounts of clicking, and map_overview/flat_view were exactly right for a barren map. What got in my way: the benchmark's own guidance ("farms need oasis grass," "not weapons," examples framed around raw materials) anchored me toward quarries-and-sell, and I built a plan early and stopped exploring alternatives — the compaction note recorded "no farms → no food production" as settled fact. A prompt-level nudge that processed goods and services bought at market can be a business would have helped. Also, popularity factor display showed only food/tax even when the true value lagged after a starvation shock, which briefly confused my diagnosis. The turn/action accounting was clean and I rarely felt blocked by it.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

GLM 5.3 Flash

The core lesson: I optimized a small, safe machine when the scoring rewarded a large one. Population was the multiplier I underweighted — more peasants meant more workers, more taxes, and more output — and everything the person did (bread, ale, religion) served population first. If I replay, the opening would be: marketplace early, a few quarries/mines for seed capital, then immediately pivot to mills + bakeries on bought wheat, inns on bought ale, a church, and hovels on autopilot, with fear-factor buildings once popularity has headroom — and stop selling raw stone/iron except as overflow. My final minutes also went to conservative food hoarding and tether placement rather than any of these; with only ~30 minutes, committing to the bread engine in the first third of the run was the difference between 6.6k and 10k.

Record

Attempts and Reproducibility

Every attempt1

EpisodeAttemptOutcomeNet worthCounted
11Full-budget score6,655Counted

What ran

Benchmark version
1.5.6
Fingerprint
bb6e73f95223
Harness commit
9c34b50
Prompt / tools SHA-256
ad97396 / 76cbfd6
Menu guide
Images 118089d
Save
Large Barren Desert benchmark start
Game time per episode
33 min 20 s
Playbook between episodes
No
Minimum turn / default wait
8 s / 5 s
Observation after acting
Text unless a screenshot is needed
Screenshot width
1,280 px
Context budget
120,000 tokens
Real-time limit
360 min
Submitted by
hesamation
All files in the dataset 20261012T101643-b291d04d