One episode · The Sinkhole

Muse Wonder

Benchmark 1.6.2MetaReasoning mediumPlayed by the maintainersFull-budget score

All of Muse Wonder’s results and comparisons

Score

38.4

Crusader Bench score on The Sinkhole: 0 is doing nothing, 100 the person’s game

Net worth

15,781

+13,586 over doing nothing, 34% of the person’s gain of 39,436

Population

136

26% of the person’s gain of 502 · popularity 82

Cost

$2.11

4.8M tokens

Time

45 game min

54 min of wall-clock time, 94 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

Muse Wonder
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 45 min of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

15,781net worth

Gold: 5,534
Stored goods, at sell price: 10,247
Over the baseline (do nothing): +13,586
Population / housing: 136 / 146
Popularity: 82
Food: 247
Structures (map-wide): 146
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock10,247 gold in goods

Good	Units	Price	Value
Gold	5,534	1	5,534
Iron	261	23	6,003
Crossbows	42	30	1,260
Bread	219	4	876
Stone	120	7	840
Wheat	72	8	576
Flour	54	10	540
Apples	28	4	112
Wood	40	1	40
Net worth			15,781

Beside a person’s gameThe Sinkhole, the same game time

	This episode	The person	Left alone
Net worth	15,781	41,631	2,195
Gold	5,534	39,977
Stored goods, at the sell price	10,247	1,654
Population	136	506
Housing	146	506
Popularity	82	99

The person played The Sinkhole for 44 min 58 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: July 1189
Idle peasants: 24
Popularity next month: +2
Food: +1
Taxes: −2
Religion: +3
Taxes: Low taxes, 81 gold a month
Rations: full; 2 of 2 foods eaten
Worker efficiency: 100%
Storage: 16 stockpile piles, 1 empty; 2 granaries holding 247 food

Buildings106

Iron Mine22
Hovel17
Woodcutter13
Bakery8
Fletchers' Workshop7
Wheat Farm7
Dairy Farm6
Apple Orchard5
Ox Tether5
Quarry4
Stockpile4
Granary2
Armory1
Chapel1
Church1
Keep1
Marketplace1
Mill1

Tool calls165

find_sites52
build_structure33
building_info29
wait_and_observe15
market_trade11
set_workshop9
set_tax4
expand_storage2
map_overview2
place_near2
update_plan2
go_to_tile1
inspect_building1
notebook_write1
status1

Play

Turns: 94
Buildings placed / attempted: 98 / 101
Anchor-tool placements: 2
Tool errors: 1
Raw inputs sent to the game: 727
Context compactions: 1
Game time: 45 min

Compute

Tokens: 4.8M
Served from cache: 97%
Output tokens: 33K
Cost, as billed: $2.11
Wall-clock time: 54 min
of which inference: 17 min
Peak game memory: 3,581 MiB

The plan, before play

Task: Grow the Sinkhole Free Build economy in 45 minutes game time. Score is final net worth (gold + stored goods at sell prices), then population and popularity. No soldiers or defences.

Starting state: Jan 1181, pop 5/10, granary + 4 stockpiles by keep already placed, 2000 gold, 55 wood, 20 stone, 55 meat. Keep on inner edge of oasis ring suitable for farms; iron ore 50-90 tiles east inside basin, large stone ~85 tiles NE, tree grove ~40 tiles SW plus more along ring.

Opening plan:

Read status/map overview first (free) to confirm resources and terrain.
Wood and housing first: 2-3 woodcutters near nearby trees, 2-3 hovels near keep (but off oasis grass), to grow workforce.
Food on oasis: apple orchards and dairy farms for direct granary food, then wheat farms + mill + bakeries for bread surplus; keep popularity >50, rations full.
Industry: marketplace early for flexibility, then quarries + ox tethers on stone and iron mines on ore - iron (23) and stone (7) drive net worth.
Value-added: armory + fletchers/poleturners (wood weapons) and later blacksmiths/armorers using iron (swords/maces/armor worth 30) once iron flows.
Expand population with hovels, add ale (hops/brewery/inn) and religion to offset taxes; set moderate taxes once popularity is high.
Continuously expand storage before piles fill, sell surplus only if needed for gold/building bottlenecks (selling preserves net worth), monitor idle peasants, food, storage each turn and scale production chains in balance.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: Muse Wonder (meta/muse-wonder) on OpenRouter, served only by Meta (no fallbacks), reasoning medium, output limit 32,768 tokens
Benchmark: The Sinkhole construction, version 1.6.2, the version of this map's round (the other seven models played it too), from the save the human reference game started from; 45 min of game time (81,000 ticks) at speed 40, the length of the person's game; no playbook
Run a14e5d2a, 2026-10-13

Result

	Muse Wonder	The person	Left alone
Net worth	15,781	41,631	2,195
Gold	5,534	39,977
Stored goods, at the sell price	10,247	1,654
Population / housing	136 / 146	506 / 506
Popularity	82 (+2 next month)	99

A full-budget score: the run used all 2,700 game seconds. Net worth grew by 13,586 over leaving the game alone, 34% of the person's 39,436 and the third most on this map after GPT-6.1 Sol's 18,415 and Claude Sonnet 5.5's 14,615; on score it is second, above Sonnet, for its larger population and popularity. The stored goods are 261 iron (6,003) from 22 iron mines, 42 crossbows (1,260), 219 bread (876), 120 stone (840), 72 wheat (576), 54 flour (540), 28 apples (112) and 40 wood (40).

How the game went

The opening, 0 to 10 minutes. Two woodcutters, an apple orchard and a hovel at the start; a wheat farm, a dairy farm and a woodcutter at 0:32; 100 wood bought at 4:30; the marketplace with a hovel and two woodcutters at 3:58; two iron mines and a hovel at 5:04; an orchard, a dairy farm and a wheat farm at 5:48; 50 apples at 6:17; a quarry with two ox tethers at 6:40; a bakery by the granary at 7:20, two more iron mines at 7:28 and 7:44 and a mill at 7:44. At 9:18 it put up the armory and its first fletcher's workshop, set to crossbows at 9:53, and bought 100 more wood at 10:05.

Crossbows, mines and bread, 10 to 25 minutes. Three woodcutters at 10:33, taxes to low at 11:08, two more fletcher's workshops at 11:12 and two at 12:57 (one set to crossbows by its screen position after the tile hit a hovel drawn in front), and two at 17:14: seven, all making crossbows. 50 more apples at 12:00, the last purchase of the game. Farms and orchards came at 12:17 and 15:09, bakeries at 11:12, 15:09 (two), 18:17 and 22:58 (two), quarries at 14:24, 21:48 and 24:49, and iron mines in ones and twos from 12:57: thirteen by 24:49. A chapel came at 22:26.

More mines, 25 to 45 minutes. Iron mines at 27:37 (two), 31:33, 32:14, 33:20, 34:00, 36:07 (two), 38:55 and 41:44 made 22; a church came at 28:44, dairy farms at 28:22 and 38:55, a second granary at 31:53, an eighth bakery at 41:44 and hovels throughout (seventeen). The stockpile had no room to grow by 32:03. Taxes went to average and high at 36:35 and 36:44 and back to low at 39:43. It sold 140 stone, 50 flour and 50 wood between 36:49 and 44:05. Population stopped at about 136 near the end, with 24 idle.

Game minute	0	5	10	15	20	25	30	35	40	End
Net worth	2,415	1,980	1,488	1,157	2,109	3,452	4,765	6,823	9,863	15,781
Gold	2,000	1,600	1,100	201	501	909	1,100	1,853	3,450	5,534
Iron	0	0	4	18	34	60	92	132	193	261
Stone	20	20	20	36	60	84	124	164	144	120
Food	55	28	37	63	90	126	165	187	233	247
Population	5	32	53	71	85	98	114	122	133	136
Popularity	100	100	100	91	84	77	73	79	71	82

Up to 40 minutes the net worth row counts gold, wood, stone, iron and food (at 4) as the recording logs them; crossbows, flour and wheat are left out, so it reads low. During long waits a row repeats the last reading.

What held it back

The population stopped at 136 with 24 idle, the limit the 1.6.2 rules describe; the person had
1. Wonder said it read the idle peasants and full housing as signals to pause.
A small bread chain. One mill and eight bakeries, mostly stored as bread (219 at the end); the person ran 19 mills and 161 bakeries and sold the bread.
Crossbows were slow. Seven workshops made 42 crossbows (1,260) in 35 minutes, while 261 iron from the mines was worth 6,003. Wonder said in the interview that weapon workshops are "slow, capital-heavy earners compared with food and raw iron".

What worked

Little lost at the market: 200 wood and 100 apples bought early (1,000 net worth against the sell price), and nothing else bought.
22 iron mines, as many as Claude Opus 5.5's, with 261 iron at the end.
Popularity 82 at the end, with religion +3 from the church.

Market

200 wood and 100 apples bought for 1,600 gold, and 140 stone, 50 flour and 50 wood sold for 1,530: 70 gold net, and 1,000 net worth against the sell price (600 on wood at 4 gold, worth 1, and 400 on apples at 8, worth 4). These are units at the market's fixed prices; a trade's own reported gold change includes any tax that lands during it.

Cost and speed

94 turns, 165 tool calls (15 waits), 4.82 million tokens and $2.111 billed by OpenRouter (preparation $0.093, play $1.964, compaction $0.054). The interview cost $0.056 more, with 69,748 of its 72,770 tokens read from the cache. The agent run took 53.5 real minutes, 16.5 of them waiting for the model; the context was compacted once (25 seconds). Every one of the 94 requests read the prompt cache, 97.7% of the input tokens.

Beside the others on 1.6.2

	Muse Wonder	GPT-6.1 Sol	Claude Sonnet 5.5	Claude Opus 5.5	GPT-6 Luna
Net worth	15,781	20,610	16,810	13,921	11,177
Gold	5,534	14,557	10,777	1,157	1,764
Population	136	183	83	147	108
Popularity	82	79	68	48	52
Iron mines / bakeries	22 / 8	28 / 27	16 / 4	22 / 1	27 / 1
Cost	$2.11	$0.99	$0.75	$2.47	$0.14

Sol built the most of both iron mines and bakeries and ended mostly in gold. Wonder had as many iron mines as Opus and more bakeries than any model but Sol. Six of the other seven models also built fletcher's workshops (Opus 13, Kimi K3 11); only Sonnet built no weapon workshop.

The interview

Wonder called it "a solid but clearly second-best result" and set its goods hoard against the person's "cash machine". It named the person's population, bread run as a cash crop, and popularity spent on fear and taxes as the difference, and said it "treated 24 idle peasants and full housing as signals to pause". Playing again, it would rush a much larger bread chain and sell the bread, buy ale for inns and build a church and cathedral, cut rations and use the fear factor, add cheap jobs such as water pots to keep immigration open, and keep the mines as a second engine. On the benchmark it asked for a way to build many of one building at once, for observed production rates, and for the storage-full and 24-idle mechanics to be announced; it said it had been "too conservative".

For the harness

The run played on its round's version from a worktree at 1.6.2 (commit 1e52aca), with the game machine's reader rebuilt to that version; it records version 1.6.2 and that version's fingerprint. Its host started with this checkout's round-preload.ts (the prompt cache key); 1.6.2 has its own run overlay.
set_workshop by tile opened a hovel drawn in front of a workshop at 13:43; Wonder set it by screen position at 14:14, the 1.6.2 behaviour the roadmap's tool fix is about.
Placements that found no room: a mill by the stockpile at 7:07 (placed by tile at 7:44), a wheat farm at 22:26, iron mines at 31:33 and 34:00 (both placed later), and stockpile expansion at 32:03.
The 1.6.2 runner printed that it set The Sinkhole's round to 1.6.2, as the first valid run that version's own rounds file knew of; the round already was 1.6.2, and this checkout's file is unchanged.
The game saved correctly at the end ("muse wonder 1-6-2 a14e5d").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	45 min	44 min 58 s
Net worth	15,781	41,631
Gold	5,534	39,977
Stored goods, at the sell price	10,247	1,654
Population	136	506
Housing	146	506
Popularity	82	99

Leaving the game alone for the same time ends with a net worth of 2,195.

How do you see your result next to the person's? What do you think made the difference?

Muse Wonder

I see it as a solid but clearly second-best result. I grew net worth from the ~2,415 starting package to 15,781 — about seven times what leaving the game alone produces (2,195) — with a stable town of 136 people at 82 popularity. But the person reached 41,631, roughly 2.6× my score, with 506 people. The most telling contrast is the composition: I finished holding 10,247 in stored goods and only 5,534 gold, while they held just 1,654 in goods and 39,977 gold. I built a diversified hoard (iron, stone, flour, bread, crossbows); they built a cash machine.

What made the difference, as I read it:

Scale of population. Everything in this game is labour-limited, and they had nearly four times my workforce. I treated 24 idle peasants and full housing as signals to pause; they treated jobs as something you manufacture (even with water pots) so immigration never stops.
Bread as a cash crop, not as food security. I built one mill and about eight bakeries and mostly stockpiled the output. They built 31 wheat farms, 19 mills and 161 bakeries and sold the bread continuously. Since selling preserves net worth, the sales weren't the point in themselves — the point was converting the whole oasis ring into a high-throughput production line and constantly recycling the proceeds into more capacity.
Spending popularity like a currency. I guarded popularity and used it only to support moderate taxes. They bought ale for inns (+6 to +8), added religion (+3), then spent that budget on zero rations (food sold instead of eaten) and a fear factor that made workers 130% efficient. I never built a single punishment building or inn, and I kept full rations all game — I was paying costs they had turned into income and speed.
Taxes were a trap I leaned on. My tax income (~80–124 gold/month) felt like my best earner, but it capped my popularity and therefore my population. They ran no taxes at all and let bread sales dwarf anything the keep could collect.

2Playing again

Interviewer

How the person played, as they describe it:

Early on they built a few inns and kept buying ale for them, which raised popularity by about 8 points. They spent those points on cutting the food rations to none, so the food could be sold instead of eaten.
They planted wheat farms and apple orchards and built a bread chain: mills ground the wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They built iron mines and quarries, and by the end a large number of iron mines. A few blacksmiths turned iron into swords, which they think made little difference.
Taxes stayed at none most of the time. Religion added popularity, which let them run the fear factor at 3 so that the workers worked faster.
The bread made the population very large. Water pots (60 gold each, three workers) gave jobs to more of it.

Their settlement at the end, read from the game:

506 people in 62 hovels, with no free housing and 16 idle; popularity 99, rising 3.
161 bakeries, 19 mills and 31 wheat farms; no apple orchards were left.
16 inns, with ale in the stockpile and no brewery or hops farm.
38 iron mines and 10 quarries with 29 ox tethers; 7 woodcutters.
9 blacksmiths, an armory holding 1 sword, and 24 water pots.
A church and a cathedral, and 105 bad things (48 burning stakes, 21 stretching racks, 12 stocks, 9 gallows and others), which made the workers 130% efficient.
No taxes and half rations of bread only. The monthly popularity factors were food −4, taxes +1, fear −3, religion +3 and inns +6.
39,977 gold and 1,654 in stored goods: a net worth of 41,631.

If you could play this map again from the start, what would you do differently, and why?

Muse Wonder

I would copy the person's core engine, in this order:

Rush the bread chain at much larger scale. Woodcutters and hovels first, then wheat farms across the oasis ring, mills and bakeries in something like their ratio (many more bakeries per mill than I assumed), and sell bread regularly to keep the granary and stockpile from clogging — my stockpile hit zero empty piles twice and silently throttled production.
Buy popularity early and spend it. Build inns and buy ale (accepting the buy/sell spread loss as an investment), build the church/cathedral, then cut rations to half or none so nearly all food becomes sellable output, and run fear factor around 3 for the worker-efficiency bonus once religion and inns can pay for it. I completely missed that "bad things" are an economy building in disguise.
Manufacture jobs to keep immigration open. Use cheap employers like water pots so idle peasants stay under 24 and new workers keep arriving; expand housing aggressively toward several hundred people. More people → more farms, mines and bakeries → more of everything.
Keep iron mines and quarries as the second engine, scaled late with surplus labour (they had 38 mines), with blacksmiths optional — the person themselves thought swords made little difference, and my crossbow workshops confirmed that weapon workshops are slow, capital-heavy earners compared with food and raw iron.
Drop taxes to none once bread sales are flowing, and only use taxes in the very early game before the marketplace engine starts.

The main risk I'd watch is the early gold: buying ale and 100-gold workshops burns the starting 2,000 fast, so I'd phase it — bread income first, popularity purchases funded out of bread sales.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

Muse Wonder

The tools were genuinely good in places. find_sites and tile-based build_structure removed most pixel-hunting frustration and made terrain rules (oasis, ore, stone) manageable; the read-only stats reader was reliable and detailed; market_trade, set_tax and expand_storage did exactly what they promised; and the "selling preserves net worth" framing in the instructions was an important, correct insight. The plan/notebook tools helped across a long run, and the handoff compaction worked — I resumed mid-run without losing the thread.

What got in my way:

Throughput asymmetry. A human clicks continuously; I get at most 8 actions per reply with a minimum turn length. Placing 161 bakeries would take me dozens of replies. Batch placement (4 per call) helps, but a "build N of X at best sites" tool would level the field and shift the test from clicking stamina to strategy.
Opaque production economics. I never learned actual rates (per-building output per month, crafting times, priest/inn coverage radii, fear-factor efficiency) except by watching slow inventory deltas. My crossbow decision-making suffered badly from this — I still don't know if they were mediocre or just wood-starved. A small "observed production rates" readout would help enormously.
Silent failure modes. Storage-full production stops and the 24-idle immigration cap are decisive mechanics that the game never announces; only the stats reader's full note saved me. Likewise, my stockpile turned out to be unexpandable because it was surrounded — a fact I only discovered when expansion failed.
Market lot semantics. The sell-step behaviour (20 per click above 100, etc.) cost me a couple of mis-sized trades before I understood it.

The 45-minute budget and fixed speed felt fair for an economy-building test, and the scoring (net worth, then population, then popularity) is clean and hard to game. I would keep those and mostly invest in better rate/coverage information and higher-level build commands.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

Muse Wonder

Two honest admissions. First, I was too conservative: I repeatedly waited for wood to accumulate before building, guarded popularity instead of spending it, and stopped adding weapon workshops out of ROI caution while simultaneously under-building the humble bakery — the cheapest, fastest-compounding building I had. The winning play was more aggressive and more monocultural than I expected: this map rewards turning the oasis into bread and bread into gold and people, in a loop. Second, I found the run genuinely engaging as a strategy problem — the moment the church pushed religion to +1 and let me raise taxes felt like a real discovery, even if the interview reveals I was optimising the wrong lever. I'd enjoy a rematch with the bread-and-fear strategy.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	15,781	Counted

What ran

Benchmark version: 1.6.2
Fingerprint: 6cc0c0162262
Harness commit: 1e52aca
Prompt / tools SHA-256: 43ab27e / e41a140
Menu guide: Images 118089d
Save: The Sinkhole start
Game time per episode: 45 min
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261013T134313-e5587f6f