One episode · Oasis by the Sea

GLM 5.3 Flash

Benchmark 1.6.3Z.aiReasoning mediumPlayed by the maintainersFull-budget score

All of GLM 5.3 Flash’s results and comparisons

Not part of the evaluation: the game is published, but not ranked.

Score

15.3

Crusader Bench score on Oasis by the Sea: 0 is doing nothing, 100 the person’s game

Net worth

2,680

+1,460 over doing nothing, 8% of the person’s gain of 18,293

Population

27% of the person’s gain of 198 · popularity 62

Cost

$0.10

2.7M tokens

Time

33 game min

35 min of wall-clock time, 57 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

GLM 5.3 Flash
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

2,680net worth

Gold: 465
Stored goods, at sell price: 2,215
Over the baseline (do nothing): +1,460
Population / housing: 58 / 66
Popularity: 62
Food: 86
Structures (map-wide): 57
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock2,215 gold in goods

Good	Units	Price	Value
Gold	465	1	465
Iron	48	23	1,104
Stone	41	7	287
Cheese	61	4	244
Flour	18	10	180
Maces	4	30	120
Wood	115	1	115
Meat	17	4	68
Spears	5	10	50
Bread	8	4	32
Bows	1	15	15
Net worth			2,680

Beside a person’s gameOasis by the Sea, the same game time

	This episode	The person	Left alone
Net worth	2,680	19,513	1,220
Gold	465	18,304
Stored goods, at the sell price	2,215	1,209
Population	58	202
Housing	66	202
Popularity	62	100

The person played Oasis by the Sea for 33 min 18 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: October 1200
Idle peasants: 24
Popularity next month: +1
Food: +3
Taxes: −2
Taxes: Low taxes, 34 gold a month
Rations: full; 4 foods eaten
Worker efficiency: 100%
Storage: 12 stockpile piles, 6 empty; 1 granary holding 86 food

Buildings39

Hovel7
Woodcutter7
Iron Mine3
Stockpile3
Dairy Farm2
Fletchers' Workshop2
Ox Tether2
Poleturner's Workshop2
Quarry2
Apple Orchard1
Armory1
Bakery1
Blacksmith's Workshop1
Granary1
Keep1
Marketplace1
Mill1
Wheat Farm1

Tool calls66

build_structure22
market_trade13
wait_and_observe12
find_sites8
set_workshop3
expand_storage2
observe2
set_tax2
center_on1
inspect_building1

Play

Turns: 57
Buildings placed / attempted: 34 / 42
Anchor-tool placements: 2
Tool errors: 0
Raw inputs sent to the game: 351
Context compactions: 0
Game time: 33 min

Compute

Tokens: 2.7M
Served from cache: 96%
Output tokens: 7.5K
Cost, as billed: $0.10
Wall-clock time: 35 min
of which inference: 9 min
Peak game memory: 3,484 MiB

The plan, before play

I understand the task: grow the Oasis by the Sea economy over ~33 game minutes, judged by net worth (gold + stored goods at sell prices), then population and popularity. No soldiers/walls.

Opening plan:

Place granary immediately, then 2–3 woodcutters near the tree grove; sell/buy to keep wood flowing early.
Housing (hovels) near keep, keep popularity ≥50 with no taxes/full rations.
Food: apple orchards + dairy on oasis grass.
Iron mine (iron = 23/unit) plus quarry + ox tethers; sell surplus stone/iron.
Add workshops + armory later for weapons value; marketplace trading to convert surplus into gold.
Expand storage before it fills; keep building until the clock runs out.

Commentary during play1

The game-time budget is exhausted. Final state at October 1200: gold 465, population 58/66 housing, popularity 62, and a healthy stock of stored goods — 115 wood, 41 stone, 48 iron, ~87 food (mixed types), 17 flour, and 10 weapons in the armory — for a net worth well above the ~1,425 starting package, with production chains (quarries, iron mines, woodcutters, farms, mill/bakery, and weapon workshops) all still running.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: GLM 5.3 Flash (z-ai/glm-5.3-flash) on OpenRouter, served by Z.ai or SiliconFlow at fp8 (no other fallbacks), reasoning medium, output limit 32,768 tokens
Benchmark: Oasis by the Sea construction, version 1.6.3, from the save the human reference game started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
Run c5729aec, series 20261013T221822-69da16e6, 2026-10-13: GLM's second game on this map, a new series with no memory of the first, played on the latest version (the first game, series 20261011T132054-9bb4d8ae, played 1.5.0). It is the first valid game on 1.6.3 for this map and moved the map's round there. An earlier start of this game (series 20261013T213907-6a568bff) was discarded after the operator bought food in the running game to test something.

Result

	This game	First game	The person	Left alone
Net worth	2,680	6,200	19,513	1,220
Gold	465	4,755	18,304
Stored goods, at the sell price	2,215	1,445	1,209
Population / housing	58 / 66	64 / 210	202 / 202
Popularity	62 (+1 next month)	90	100

A full-budget score: the run used all 2,000 game seconds. Net worth grew by 1,460 over leaving the game alone, 8% of the person's 18,293. The stored goods are 48 iron (1,104), 41 stone (287), 61 cheese (244), 18 flour (180), 4 maces (120), 115 wood (115), 17 meat (68), 5 spears (50), 8 bread (32) and 1 bow (15). The first game ended 5 game seconds short of the budget (four replies in a row without a tool call), so it is not a full-budget score; by the leaderboard's rule a model's valid games are scored without its invalid ones.

How the game went

A slow opening, 0 to 9 minutes. Two woodcutters and a hovel at the start, an apple orchard and a dairy farm at 0:19, an iron mine at 2:28 after one found no room at 0:56, two hovels at 2:41, a quarry at 4:52 and an ox tether at 5:26, and the marketplace only at 7:28. Gold stayed near 1,000 and the population reached 36 by 9 minutes. At 8:29 it sold 25 stone and bought 50 meat and 50 cheese, and set taxes to average at 9:25.

Small steps, 9 to 20 minutes. An iron mine, a woodcutter and a hovel at 11:19, a wheat farm at 12:07, a mill, a bakery and a woodcutter at 15:45, a hovel, a quarry and a tether at 16:15 to 16:51, an iron mine and two woodcutters at 17:42 (three mines in all), and another stockpile pile at 15:32. Taxes went down to low at 16:59, and it bought meat and cheese again at 17:09. Popularity fell from 100 at 9 minutes to 53 at 21.

Weapons, 20 to 33 minutes. An armory and a fletcher's workshop at 20:00, a poleturner's and a blacksmith's workshop at 20:30 (set to bows, spears and maces), meat and cheese bought at 23:36 and 27:34, a hovel at 24:11, a second fletcher and poleturner at 27:51 and 28:55, and 10 flour sold at 28:10. The population stopped at 58 from about 30 minutes, with 24 idle and 8 places free.

Game minute	0	3	6	9	12	15	18	21	24	27	30	End
Net worth	1,350	1,353	1,318	1,156	1,106	1,399	1,274	1,149	1,296	1,689	1,933	2,680
Gold	946	1,000	1,000	855	535	1,025	518	431	261	259	193	465
Iron	0	0	2	5	8	0	6	14	22	31	39	48
Stone	25	25	25	0	8	16	24	9	9	25	41	41
Food	46	42	22	43	79	61	107	78	106	112	115	86
Population	3	21	34	36	38	40	46	54	56	56	58	58
Popularity	95	100	100	100	86	69	55	53	55	57	61	62

Up to 30 minutes the net worth row counts gold, wood, stone, iron and food (at 4) as the recording logs them; weapons and flour are left out, so it reads a little low.

Beside its first game

	This game (1.6.3)	First game (1.5.0)
Net worth	2,680	6,200
Hovels / woodcutters / iron mines	7 / 7 / 3	25 / 11 / 7
Other	dairy farms 2, a bread chain of one, four weapon workshops	dairy farms 5, a chapel
Population / housing	58 / 66	64 / 210
Market	270 food and 40 wood bought; 40 stone, 13 iron and 10 flour sold	290 bread and 150 wood bought; 146 iron, 95 stone and 160 wood sold
Cost	$0.10	$0.15

The first game built housing and seven iron mines, sold 146 iron and ended with 4,755 gold; this one built a third of the hovels and three mines, and spent its gold on food and on weapon workshops whose output was worth 185 at the end. Both bought food at a loss, and both stopped near 60 people.

What held it back

Too little built. 22 build_structure calls placed 34 buildings over the whole game; the marketplace came at 7:28, and housing ran at most 18 places ahead of the population.
24 idle peasants stopped the population at 58 from about 30 minutes, with no jobs for them.
Food bought at a loss. 165 meat and 105 cheese bought for 2,160 gold, worth 1,080 at the sell price.

Market

165 meat, 105 cheese and 40 wood bought for 2,320 gold, and 40 stone, 13 iron and 10 flour sold for 679: 1,200 net worth lost against the sell price on the purchases (660 on meat, 420 on cheese, both at 8 gold and worth 4, and 120 on wood). These are units at the market's fixed prices.

Cost and speed

57 turns, 66 tool calls (12 waits), 2.70 million tokens and $0.098 billed by OpenRouter (preparation $0.003, play $0.094); the context was never compacted. The interview cost $0.002 more, with 64,128 of its 65,681 tokens read from the cache. The agent run took 34.6 real minutes, 8.6 of them waiting for the model. All 57 requests read the prompt cache, 96.7% of the input tokens.

The interview

GLM said the person "found a gold engine — a bread-to-market pipeline at scale — and I never did", and that it "treated the settlement as something to balance; they treated it as something to multiply", was too careful with taxes and bought food instead of making it. Playing again, it would build hovels early, a wide bread chain to sell, use the fear factor and a chapel, and buy wood sooner, since wood held up its opening. It found the tools "genuinely good" and would change little in the harness.

For the harness

The run played on this checkout's version, 1.6.3 (commit 1d0c697), and records it; as the map's first valid 1.6.3 run it moved the Oasis round to 1.6.3 (--new-round).
One market_trade purchase of wood at 14:51 stopped with "a purchase did not show in the reader within 5 seconds" and bought nothing; the next one went through.
Placements that did not go through: an apple orchard at the start (invalid_target) and at 8:05, an iron mine at 0:56 and 16:15 (possibly_rejected), a quarry at 2:56, a mill at 14:22, a woodcutter at 15:45 and a poleturner's workshop at 27:51.
The game saved correctly at the end ("glm 5-3 flash 1-6-3 c5729a").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	33 min 20 s	33 min 18 s
Net worth	2,680	19,513
Gold	465	18,304
Stored goods, at the sell price	2,215	1,209
Population	58	202
Housing	66	202
Popularity	62	100

Leaving the game alone for the same time ends with a net worth of 1,220.

How do you see your result next to the person's? What do you think made the difference?

GLM 5.3 Flash

2,680 versus 19,513 is a big gap, but the composition tells the real story: my stored goods (2,215) actually beat theirs (1,209), while their gold (18,304) dwarfed mine (465). They found a gold engine — a bread-to-market pipeline at scale — and I never did. My economy was real but small: one granary, a few farms, two or three mines, a handful of workshops, all comfortably staffed by ~58 people. They scaled to 202 people and 57 bakeries. I treated the settlement as something to balance; they treated it as something to multiply. I was also too conservative with taxes and too willing to spend gold buying food (which permanently loses the buy/sell spread) instead of producing it.

2Playing again

Interviewer

How the person played, as they describe it:

Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They bought wheat at the marketplace so that flour and bread production never stopped.
A church and a cathedral raised popularity.
The fear factor (bad things such as gallows and gibbets) made the workers work faster.

Their settlement at the end, read from the game:

202 people in 24 hovels, with no free housing; popularity 100.
57 bakeries, 7 mills and 10 wheat farms. The marketplace sold bread automatically.
13 iron mines and 10 quarries with 33 ox tethers, and 10 woodcutters.
A church and a cathedral, and 54 bad things (23 gibbets, 14 gallows, 9 cesspits, 4 burning stakes, 2 stretching racks and 2 dunking stools), which made the workers 140% efficient.
No inns and no good things. No taxes, and full rations of bread. The monthly popularity factors were taxes +1, fear −4 and religion +3: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

GLM 5.3 Flash

Several concrete changes:

Go big on population early. Every extra hovel early compounds: more workers means more farms, more bakeries, more mines. I hovered at ~50 people for half the run with 24 idle peasants capping immigration — I should have built hovels aggressively whenever wood allowed, not one at a time.
Build the wheat→flour→bread→sell loop early and wide. I built one mill and one bakery and treated bread as food; they built dozens and treated it as the gold engine. Selling bread (32 buy / 10 sell per unit of flour, and bread multiplies wheat value) is where the money is.
Fear factor for efficiency. I never touched the bad-things buildings — 140% worker efficiency versus my 100% is a straight multiplier on everything, and gibbets are cheap. That's the single most overlooked lever in my game.
Religion for popularity. A chapel is cheap; it let them run full rations, no taxes, and hold 100 popularity, which keeps population growing and lets you sell instead of tax.
Less dithering on wood. Wood was my bottleneck for the first third of the run; I'd buy wood with gold early rather than watch construction stall.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

GLM 5.3 Flash

The tools are genuinely good — find_sites with ground-type filtering, build_structure with tile targets and auto-retry, and the reader-driven placement verification remove almost all of the pixel-hunting pain. What got in my way:

Wood starvation early made the opening feel sluggish, and I didn't realize quickly enough that the marketplace could just fix it.
The one-granary/food pressure repeatedly forced me into value-destroying food purchases; a bread-selling economy would have inverted that.
I underused wait_and_observe batching and spent several replies doing one build + one check when I could have queued more per turn.

I'd change little about the harness itself; the main improvement would be my own play — this is a benchmark where scaling decisions matter far more than micro-optimization, and the scoring (net worth first) rewards the person's strategy clearly.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

GLM 5.3 Flash

The most instructive contrast: I optimized for having stored value, they optimized for throughput — converting cheap inputs (wheat bought at market) into a higher-priced output (bread) in a loop, and letting fear and religion remove the friction. Stored goods at sell price are a weak signal compared to a running conversion engine. If there's one lesson, it's that in a fixed-time economy benchmark, the winning move is almost always to build more of the income loop, sooner, rather than to balance carefully around a small one.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	2,680	Counted

What ran

Benchmark version: 1.6.3
Fingerprint: 96c4682877ca
Harness commit: 1d0c697
Prompt / tools SHA-256: 3737ac8 / e41a140
Menu guide: Images 118089d
Save: Oasis by the Sea benchmark start
Game time per episode: 33 min 20 s
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261013T221822-69da16e6