One episode · Large Barren Desert

GLM 5.3 Flash

Benchmark 1.5.4Z.aiReasoning mediumPlayed by the maintainersFull-budget score

All of GLM 5.3 Flash’s results and comparisons

Not ranked: GLM 5.3 Flash’s game on 1.5.6 is the one on the leaderboard.

Score

57.4

Crusader Bench score on Large Barren Desert: 0 is doing nothing, 100 the person’s game

Net worth

8,232

+3,040 over doing nothing, 63% of the person’s gain of 4,792

Population

17% of the person’s gain of 246 · popularity 49

Cost

$0.06

1.5M tokens

Time

33 game min

33 min of wall-clock time, 38 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

GLM 5.3 Flash
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

8,232net worth

Gold: 7,496
Stored goods, at sell price: 736
Over the baseline (do nothing): +3,040
Population / housing: 47 / 90
Popularity: 49
Food: 52
Structures (map-wide): 48
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

No final overview was published for this episode.

Final stock736 gold in goods

Good	Units	Price	Value
Gold	7,496	1	7,496
Iron	10	23	230
Meat	52	4	208
Stone	29	7	203
Ale	9	10	90
Wood	5	1	5
Net worth			8,232

Beside a person’s gameLarge Barren Desert, the same game time

	This episode	The person	Left alone
Net worth	8,232	9,984	5,192
Gold	7,496	8,574
Stored goods, at the sell price	736	1,410
Population	47	250
Housing	90	250
Popularity	49	100

The person played Large Barren Desert for 33 min 33 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: July 1178
Idle peasants: 24
Popularity next month: −4
Food: −4
Taxes: −6
Religion: +2
Ale from the inn: +4
Taxes: High taxes, 46 gold a month
Rations: half; 1 of 1 foods eaten
Worker efficiency: 100%
Storage: 12 stockpile piles, 7 empty; 1 granary holding 52 food

Buildings29

Hovel10
Iron Mine4
Ox Tether4
Quarry3
Stockpile3
Chapel1
Granary1
Inn1
Keep1
Marketplace1

Tool calls72

wait_and_observe15
build_structure14
market_trade10
go_to_tile9
find_sites6
set_tax6
set_rations3
go_to_view2
expand_storage1
get_inventory1
inspect_building1
map_overview1
observe1
place_near1
save_view1

Play

Turns: 38
Buildings placed / attempted: 21 / 34
Anchor-tool placements: 5
Tool errors: 0
Raw inputs sent to the game: 327
Context compactions: 1
Game time: 33 min

Compute

Tokens: 1.5M
Served from cache: 93%
Output tokens: 10K
Cost, as billed: $0.06
Wall-clock time: 33 min
of which inference: 7 min
Peak game memory: 3,211 MiB

The plan, before play

Task understanding: Grow a Large Barren Desert free-build economy over ~33 game minutes. Score = net worth (gold + stored goods at sell prices), population, and popularity. No soldiers or defenses. Key constraints: no oasis grass → no farms at all; food must come from trading or starting meat. Stone (7/unit) and iron (23/unit) are the main producible value — quarries ~100 tiles west, iron ~60 tiles east.

Opening plan:

Immediately build a Marketplace (needed for trading) and 2–4 hovels near the keep for population growth.
Go to the iron deposits (~60 tiles east): place 2–3 iron mines; add woodcutters near any trees. Iron at 23/unit is the best value producer.
Extend to the stone deposits west with quarries + ox tethers if time and gold allow.
Keep rations full, taxes at "no taxes" initially; raise taxes once popularity climbs (food variety is limited, so watch popularity closely).
Manage stockpile space: expand storage or sell surplus (iron, stone, wood) to keep production flowing; gold from sales is neutral for net worth but taxes and produced goods raise it.
Keep building hovels as idle peasants appear, monitor with status/get_inventory, avoid defense spending.

Commentary during play11

The keep is at (502,374), iron at ~(560,390) east, stone ~93 tiles west. I'll build a marketplace and hovels near the keep, then move to the iron deposits.
The tile targets must be on-screen. Now at the iron deposits — placing 3 iron mines, then moving back toward the keep.
Iron mines placed, but the stockpile has no empty pile — iron can't be delivered. Expanding the stockpile and building the marketplace + hovels now.
Marketplace didn't place last turn — retrying it, plus more hovels for the arriving peasants.
The mines look active (smoke). Waiting for first deliveries and population growth.
Iron trickles in slowly (60-tile walk). 24 idle peasants need jobs — heading to the stone deposit for a quarry, and I'll add hovels.
Quarries + ox tethers placed and 7 peasants took jobs. Now more housing near the keep.
Food is critical (8 left) — buying meat and halving rations to stretch it, since no farms are possible on this map.
Food secured, but popularity now falls 8/month. Building an inn (ale → popularity) and a chapel.
Popularity crashed to 39 — dropping taxes to zero and checking why the quarries produce nothing.
Quarry is working — output is limited by walking distance, so I'll add more parallel mines/quarries.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: GLM 5.3 Flash (z-ai/glm-5.3-flash) on OpenRouter, providers z-ai/fp8 then siliconflow/fp8, reasoning medium, output limit 32,768 tokens
Benchmark: Large Barren Desert construction, version 1.5.4, from the save the human reference game on this map started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
Run 97b99ccf, 2026-10-11

Result

	GLM 5.3 Flash	The person	Left alone
Net worth	8,232	9,984	5,192
Gold	7,496	8,574
Stored goods, at the sell price	736	1,410
Population / housing	47 / 90	250 / 250
Popularity	49	100

A full-budget score: the run used all 2,000 game seconds. Net worth grew by 3,040 over leaving the game alone, about two thirds of the person's growth (4,792). About 1,150 of that is the part of the starting package that waits for storage room (stone, iron, pitch, ale and more wood), which the game left alone never receives and GLM had by 3 minutes; its own play added about 1,900. Its gold came close to the person's.

How the game went

Opening, 0 to 7 minutes. Three iron mines, hovels and more stockpile at the start, a marketplace at 1 minute, two quarries with ox tethers at 4, and an inn and a chapel at 6 minutes. Meat ran low by 6 minutes: it bought 100 meat, halved the rations and took the tax off, and popularity fell to 40 at 8 minutes.

Mines and markets, 7 to 25 minutes. A fourth iron mine at 9 minutes and a third quarry at 18, with more ox tethers, and housing to 90. It sold iron and stone as they came in and bought meat twice more, and moved taxes between low and high. Net worth rose from about 5,900 at 9 minutes to 7,161 at 24.

The end. Full rations from 21 minutes and popularity 91 at 27, then half rations and high taxes again from 26 to 29 minutes for the last gold: popularity fell to 49 at the end. Its stores were mostly sold by then; gold reached 7,496.

Game minute	0	3	6	9	12	15	18	21	24	27	30	End
Net worth	4,501	6,436	6,523	5,924	6,103	6,577	6,829	6,909	7,161	7,455	7,378	8,232
Population	2	22	38	33	37	43	42	39	47	47	47	47
Popularity	100	98	70	46	63	48	53	46	67	91	91	49

What held it back

24 idle peasants from 23 minutes on. The population stopped at 47 with 43 empty places: on every map played the same day, new peasants came only when idle ones got work. GLM said in its interview that it "discovered mid-run that idle peasants with no jobs are dead weight", but by then it did not add jobs.
Food bought, not made. No farm can be built here; three purchases of 100 meat lost 1,147 net worth. The person bought wheat and ran mills and bakeries, which GLM named as "the two big ideas I missed" with fear buildings.
Popularity spent for gold at the end. High taxes and half rations in the last minutes took popularity from 91 to 49.

What worked

Iron, stone and pitch sold for gold: seven sales brought 3,310 gold. Selling at the sell price leaves net worth unchanged, and it kept the stockpile from filling up.
An inn and a chapel at 6 minutes: at the end the inn added +4 and religion +2 to popularity.
It acted steadily: the waiting reminder (3 game minutes without an economy action) never appeared.

Cost and speed

38 turns, 72 tool calls, 1.48 million tokens and $0.062 (billed by OpenRouter). The agent run took 32.9 real minutes, 6.9 of them waiting for the model; one compaction took 24 s. Thirteen placements failed, eight of them because the tile was off screen.

Beside the other desert run on 1.5.4

	GLM 5.3 Flash	Claude Haiku 5.5
Net worth	8,232	5,313
Gold / goods	7,496 / 736	2,711 / 2,602
Population / housing	47 / 90	32 / 50
Popularity	49	93
Iron mines / quarries / inns / chapels	4 / 3 / 1 / 1	2 / 1 / 0 / 0
Net worth lost to purchases	1,147	1,176
Cost	$0.062	$0.044
Real time	32.9 min	29.4 min

GLM kept building to 18 minutes and sold what it mined; Haiku stopped at 3 minutes and waited.

The interview

GLM said it "ran a small mining-and-taxes economy instead of a settlement" and "treated the market as a place to dump surplus rather than an engine". Playing again, it would buy wheat for mills and bakeries, build a church, inns and fear buildings, add more ox tethers and stop "fussing over small tax adjustments". It asked why its population stopped at 47 with free housing, which no tool explained.

For the harness

The game saved correctly at the end ("glm 5-3 flash 1-5-4 97b99c").
Placing on tiles off screen still fails (not_on_screen eight times), as in every run.

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	33 min 20 s	33 min 33 s
Net worth	8,232	9,984
Gold	7,496	8,574
Stored goods, at the sell price	736	1,410
Population	47	250
Housing	90	250
Popularity	49	100

Leaving the game alone for the same time ends with a net worth of 5,192.

How do you see your result next to the person's? What do you think made the difference?

GLM 5.3 Flash

I'm clearly behind the person: 8,232 vs 9,984 net worth, and especially 47 vs 250 population. My gold was respectable (7,496), but I essentially ran a small mining-and-taxes economy instead of a settlement. The person's play exposed the two big ideas I missed: (1) the bread pipeline — buy wheat at the market, mill it, bake it, sell bread — turns gold into more gold, and (2) fear-factor buildings (gallows, stocks) push worker efficiency above 100%, multiplying all output. I never thought to buy raw inputs to feed processing chains; I treated the market as a place to dump surplus rather than an engine. I also barely grew the population: with only mines and quarries for jobs, extra peasants were idle, so I saw no reason to build housing — but the person shows that a large, high-popularity population with an industry that employs it is worth far more. Their 95 bakeries also explain something I found puzzling: full-employment-by-processing means every peasant is productive.

2Playing again

Interviewer

How the person played, as they describe it:

Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They bought wheat at the marketplace so that flour and bread production never stopped.
A church and a cathedral raised popularity, and so did inns.
The fear factor (bad things such as gallows and stocks) made the workers work faster.

Their settlement at the end, read from the game:

250 people in 30 hovels, with no free housing; popularity 100.
95 bakeries and 12 mills, and no wheat farms.
11 inns, which reached everyone with ale, and no brewery or hops farm.
13 quarries and 7 iron mines with 37 ox tethers; no woodcutters.
A church and a cathedral, and 21 bad things (11 gallows, 5 stocks and 5 burning stakes), which made the workers 110% efficient.
Low taxes and no rations. The monthly popularity factors were food −8, taxes −2, fear −1, religion +3 and inns +8: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

GLM 5.3 Flash

I would, from the start: build a marketplace and housing early and aggressively (hovels every time idle peasants + popularity allow), buy wheat continuously and set up mills → bakeries → sell bread, since that is a repeatable value-adding loop on gold. I'd put up a church/cathedral and a few inns plus gallows/stocks for the fear factor to push efficiency over 100%. I'd still do quarries and iron mines (they worked fine for me), but with many more ox tethers to parallelize hauling, which was my real bottleneck. I would stop fussing over small tax adjustments — the person ran low taxes and let popularity sit at 100 while the economy did the work. My biggest mistake was rationing scarcity: I kept treating food as something to conserve rather than something to buy in bulk when it converts into population and bread.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

GLM 5.3 Flash

The tools are well designed: tile-based placement with free-spot retries, the reader-backed statuses, and the minimap/tile navigation saved an enormous amount of pixel-hunting, and the "reading is free before acting" rule encouraged planning. The main frictions were (a) the 8-action budget per reply combined with slow walking distances, which made exploration of deposits expensive, and (b) placement rules that were only discoverable by failing ("must be on iron ore", no-open-side refits) — the notebook helped here. The popularity factors were somewhat opaque (why population stopped at 47 despite 90 housing and popularity 91 never got explained by any tool reading), and worker efficiency/fear factor is invisible until you know the trick. I'd also have liked the score weights up front; I optimized blind between gold, population and popularity. Overall a solid, learnable benchmark that rewards actual game knowledge like the bread loop and fear factor.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

GLM 5.3 Flash

A large part of my gap was strategy, not execution: I discovered mid-run that idle peasants with no jobs are dead weight, and that on a map with no farms and no trees the market is the only food source — but I rationed purchases at exactly what the population needed instead of scaling population and selling bread for profit. Also, my late-run tax hikes traded popularity for a little gold right at the end, which probably cost me popularity points for trivial gain — with hindsight, the last minutes should have gone into popularity and final inventory, not tax income.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	8,232	Counted

What ran

Benchmark version: 1.5.4
Fingerprint: 0d53a4d530ff
Harness commit: b271053
Prompt / tools SHA-256: bfde2b5 / a11b9d6
Menu guide: Images 118089d
Save: Large Barren Desert benchmark start
Game time per episode: 33 min 20 s
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261011T214835-33796a71