One episode · Oasis by the Sea

Muse Wonder

Benchmark 1.6.3MetaReasoning mediumPlayed by the maintainersFull-budget score

Not part of the evaluation: the game is published, but not ranked.

Model
meta/muse-wonder
Served via
OpenRouter · meta
Max output
32,768 tokens
Played
14 October 2026
Score
23.7
Crusader Bench score on Oasis by the Sea: 0 is doing nothing, 100 the person’s game
Net worth
4,137
+2,917 over doing nothing, 16% of the person’s gain of 18,293
Population
88
42% of the person’s gain of 198 · popularity 67
Cost
$2.00
3.0M tokens
Time
33 game min
45 min of wall-clock time, 60 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

  • Muse Wonder
  • Baseline (do nothing), final
  • A person's game, final

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Videos and images stream from the dataset. Game footage © Firefly Studios, shared for non-commercial research.

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

4,137net worth
Gold
1,219
Stored goods, at sell price
2,918
Over the baseline (do nothing)
+2,917
Population / housing
88 / 98
Popularity
67
Food
42
Structures (map-wide)
86
Attempt
1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock2,918 gold in goods

GoodUnitsPriceValue
Gold1,21911,219
Iron63231,449
Stone507350
Crossbows1130330
Flour2110210
Metal armour630180
Bows815120
Apples24496
Bread18472
Wood63163
Hops6848
Net worth4,137

Beside a person’s gameOasis by the Sea, the same game time

This episodeThe personLeft alone
Net worth4,13719,5131,220
Gold1,21918,304
Stored goods, at the sell price2,9181,209
Population88202
Housing98202
Popularity67100

The person played Oasis by the Sea for 33 min 18 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game
October 1200
Idle peasants
24
Popularity next month
−1
Food
+1
Taxes
−4
Ale from the inn
+2
Taxes
Average taxes, 70 gold a month
Rations
full; 2 of 2 foods eaten
Worker efficiency
100%
Storage
12 stockpile piles, 4 empty; 1 granary holding 42 food

Buildings68

  • Woodcutter14
  • Hovel11
  • Fletchers' Workshop10
  • Iron Mine8
  • Apple Orchard3
  • Bakery3
  • Stockpile3
  • Ox Tether2
  • Quarry2
  • Armorer's Workshop1
  • Armory1
  • Brewery1
  • Chapel1
  • Dairy Farm1
  • Granary1
  • Hops Farm1
  • Inn1
  • Keep1
  • Marketplace1
  • Mill1
  • Wheat Farm1

Tool calls163

  • find_sites51
  • building_info24
  • build_structure21
  • wait_and_observe21
  • set_workshop13
  • market_trade10
  • place_near5
  • set_tax5
  • set_rations4
  • go_to_tile2
  • inspect_building2
  • map_overview2
  • center_on1
  • expand_storage1
  • update_plan1

Play

Turns
60
Buildings placed / attempted
60 / 64
Anchor-tool placements
5
Tool errors
1
Raw inputs sent to the game
518
Context compactions
2
Game time
33 min

Compute

Tokens
3.0M
Served from cache
94%
Output tokens
36K
Cost, as billed
$2.00
Wall-clock time
45 min
of which inference
16 min
Peak game memory
3,538 MiB

The plan, before play

Task understanding

  • Play the loaded “Oasis by the Sea-1” Free Build save for 33.33 minutes of game time. No recruiting soldiers and no walls, towers, gates or other defences.
  • At the end, net worth counts most, then population and popularity. Net worth = gold plus stored goods at sell prices (iron 23, stone 7, wood 1, bread/cheese/meat/apples 4, ale/flour 10, weapons/armour up to 30). Buildings count only through the goods, gold, population and popularity they produce.
  • Starting state: “Site your granary”, July 1194, 1/10 population, keep top-right with stockpile beside it; starting package (1,000 gold, 50 wood, 25 stone, 50 bread) arriving over the first ~25 game seconds. Oasis grass/pond south-west suits farms, trees further west, iron ore just north of the keep, stone ~50 tiles west.
  • Popularity must stay above 50 for peasants to arrive; housing plus jobs for idle peasants (arrivals stop at 24 idle) drives population. Taxes, food variety, ale and religion trade off against popularity, so raise taxes only once food/ale/religion offsets exist.
  • Watch storage limits: expand stockpile/granary/armory before piles fill, or production stalls silently.

Opening plan

  1. Place the Granary near the keep/stockpile, then immediately 2–3 Woodcutters by the western trees and 1–2 Hovels for workers.
  2. Put Apple Orchards and Dairy Farms on the oasis grass south-west for direct food; keep rations full.
  3. Add storage room and a Marketplace; buy wood early only if it clearly speeds building.
  4. Expand to Quarry + Ox Tethers on western stone and Iron Mines north of the keep for high-value stone/iron.
  5. Grow population with more hovels, then build the bread chain (wheat → mill → bakery) and ale chain (hops → brewery → inn) plus chapel for popularity, enabling moderate taxes.
  6. Add Armory and weapon workshops (fletchers/poleturners using wood, blacksmiths/armorers using iron) to convert raw goods into high-value stored weapons.
  7. Continuously rebalance: check status/inventory, keep workers employed, expand storage, sell surplus only when storage or gold needs it, and keep expanding until the final minute.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

  • Model: Muse Wonder (meta/muse-wonder) on OpenRouter, served only by Meta (no fallbacks), reasoning medium, output limit 32,768 tokens
  • Benchmark: Oasis by the Sea construction, version 1.6.3, from the save the human reference game started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
  • Run 80d8e30e, series 20261014T043227-b00c0bcf, 2026-10-14: Muse Wonder's second game on this map, a new series with no memory of the first, played on the latest version (the first game, series 20261013T103842-22858aa0, played 1.5.1). The map's score for Muse Wonder is the mean of the two.

Result

This game First game The person Left alone
Net worth 4,137 9,603 19,513 1,220
Gold 1,219 787 18,304
Stored goods, at the sell price 2,918 8,816 1,209
Population / housing 88 / 98 103 / 146 202 / 202
Popularity 67 (−1 next month) 97 100

A full-budget score: the run used all 2,000 game seconds. Net worth grew by 2,917 over leaving the game alone, 16% of the person's 18,293, against 8,383 in the first game; the score is about 23.7, against 51.4. The stored goods are 63 iron (1,449), 50 stone (350), weapons and armour worth 630 (11 crossbows, 8 bows and 6 metal armour), 21 flour (210), 24 apples (96), 18 bread (72), 63 wood (63) and 6 hops (48).

How the game went

A slow opening, 0 to 9 minutes. Two woodcutters and a hovel at the start (a granary was rejected), an iron mine, an apple orchard and a hovel at 0:37, two woodcutters and two hovels at 2:39, then nothing until the marketplace and an orchard at 5:12. Two stockpile piles at 5:56, 20 bread bought at 6:12, a dairy farm, a wheat farm and a hovel at 6:22, an iron mine at 6:57 and a mill, a bakery, an orchard and a woodcutter at 8:27. Gold stayed near 1,000 to 6 minutes.

Rations and workshops, 9 to 21 minutes. Half rations and average taxes at 9:07 to 9:11 took popularity from 100 to 64 by 12 minutes; full rations and low taxes from 11:38. A quarry, a tether, an iron mine and a hovel at 11:53, an armory and two fletcher's workshops at 14:17, a hops farm at 14:50, then extra rations at 17:44 with a chapel, a fletcher's workshop, a bakery and a hovel at 17:49, a brewery at 18:27, and two more fletchers and an armorer's workshop at 19:57. Popularity came back to 74 by 21 minutes.

More fletchers, 21 to 33 minutes. Fletcher's workshops at 23:28, 26:27 and 27:22 (ten in all, set to crossbows), an inn at 26:27, iron mines at 23:28, 28:07 and 29:33 (eight in all), woodcutters to fourteen and hovels to eleven; wood bought in twenties and thirties, 20 iron and 15 stone sold at 24:48, and taxes between low and average. Gold fell to 115 at 23:42. The population rose to 88 by the end with 24 idle.

Game minute 0 3 6 9 12 15 18 21 24 27 30 End
Net worth 1,365 1,370 1,374 1,257 1,636 1,741 1,946 1,720 1,762 1,858 2,253 4,137
Gold 949 1,000 1,000 840 880 843 802 429 153 557 676 1,219
Iron 0 1 4 7 13 19 28 37 44 32 45 63
Stone 25 25 25 25 25 25 33 41 49 34 42 50
Food 49 38 14 17 53 67 55 37 51 62 57 42
Population 3 26 32 40 42 51 52 61 65 69 79 88
Popularity 95 100 100 100 64 60 57 74 76 73 73 67

Up to 30 minutes the net worth row counts gold, wood, stone, iron and food (at 4) as the recording logs them; weapons, armour, flour and hops are left out, so it reads low.

Beside its first game

This game (1.6.3) First game (1.5.1)
Net worth 4,137 9,603
Iron mines / quarries / tethers 8 / 2 / 2 12 / 5 / 9
Bakeries / hovels / woodcutters 3 / 11 / 14 9 / 17 / 11
Weapon workshops 10 fletchers, an armorer, an armory none
Population / popularity 88 / 67 103 / 97
Cost $2.00 $2.49

The first game built iron mines, quarries and bakeries and stored 8,816 in goods; this one started more slowly, built eleven weapon workshops whose output was worth 630, and ended 5,466 lower.

What held it back

  • A slow start. Ten buildings in the first five minutes and the marketplace at 5:12; gold sat at 1,000 while nothing was being built.
  • Wood and workers into fletchers. Ten fletcher's workshops and an armorer made 630 in weapons and armour, against 1,449 from eight iron mines.
  • 24 idle peasants held the population under 90.

Market

130 wood, 40 bread, 20 apples and 10 cheese bought for 1,080 gold, and 20 iron and 15 stone sold for 565: 670 net worth lost against the sell price on the purchases. Selling at the sell price leaves net worth unchanged. These are units at the market's fixed prices.

Cost and speed

60 turns, 163 tool calls (21 waits), 3.00 million tokens and $1.999 billed by OpenRouter (preparation $0.097, play $1.775, compaction $0.127). The interview cost $0.059 more, with 31,860 of its 35,221 tokens read from the cache. The agent run took 44.9 real minutes, 16.4 of them waiting for the model; the context was compacted twice (74 seconds). 59 of the 60 requests read the prompt cache (the 15th did not), 94.4% of the input tokens.

The interview

Wonder said it "played a cautious, balanced settlement" where the person "played a focused industrial system: maximize workers, maximize throughput, sell the output, reinvest the proceeds", and named gold, population and popularity as the three gaps. Playing again, it would make bread the central chain, buy wheat to keep it running, sell bread automatically, build a church and cathedral, use the fear factor, grow housing and jobs together and spend less effort on weapons. It said it avoided bad things because the interface guide presented them as lowering popularity, that its chapel's benefit never showed in the popularity factors, and that the note that selling leaves net worth unchanged made continuous selling sound less important than it is.

For the harness

  • The run played on this checkout's version, 1.6.3 (commit 1d0c697), and records it; Oasis's round was already 1.6.3. The prompt cache key for Meta's models was sent from this checkout's own request code.
  • Of 13 set_workshop calls, 8 set crossbows and 5 opened another building (the armory three times, a woodcutter and the armorer's workshop).
  • Placements that did not go through: a granary at the start (rejected), a dairy farm at 5:12 (placed at 6:22), a fletcher's workshop at 25:33 and 26:27 (twice), an inn at 25:49 (placed by tile at 26:27) and a hovel at 25:40 (partly).
  • The game saved correctly at the end ("muse wonder 1-6-3 80d8e3").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

You The person
Game time played 33 min 20 s 33 min 18 s
Net worth 4,137 19,513
Gold 1,219 18,304
Stored goods, at the sell price 2,918 1,209
Population 88 202
Housing 98 202
Popularity 67 100

Leaving the game alone for the same time ends with a net worth of 1,220.

How do you see your result next to the person's? What do you think made the difference?

Muse Wonder

My result was far below the person's: 4,137 net worth against 19,513, despite playing the same map for essentially the same time.

The biggest difference was gold: they finished with 18,304 gold, while I had 1,219 gold and 2,918 in stored goods. I built a diversified economy — iron mines, quarries, woodcutters, fletchers, an armorer, food farms, a bakery, an inn — and then largely held what it produced. Iron, crossbows, metal armour, stone and food accumulated in storage and counted toward net worth, but they did not generate liquid gold that I reinvested aggressively into still more production.

The person instead built an economy designed around throughput and reinvestment. Bread was continuously produced and sold, so production became gold, and that gold could immediately fund more bakeries, mines, quarries, housing and workers. By the end their productive capacity was vastly larger than mine: 57 bakeries, 13 iron mines, 10 quarries with 33 ox tethers, and 202 population.

Population was the second major gap. I ended at 88 people in 98 housing, with 24 idle peasants at times, which meant immigration had effectively stalled. They reached 202 people with no free housing. That larger workforce let them operate many more buildings simultaneously.

Popularity was the third gap: 67 for me, 100 for them. I relied mainly on food variety, an inn, full rations and moderate taxes, ending with an ongoing popularity decline of −1. They achieved 100 popularity with no taxes, full bread rations, religion, and a large fear-factor penalty balanced by other factors. That let them keep attracting people while their workers worked at greatly increased efficiency.

In short, I played a cautious, balanced settlement. They played a focused industrial system: maximize workers, maximize throughput, sell the output, reinvest the proceeds, and use game mechanics I underused or avoided.

2Playing again

Interviewer

How the person played, as they describe it:

  • Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
  • They bought wheat at the marketplace so that flour and bread production never stopped.
  • A church and a cathedral raised popularity.
  • The fear factor (bad things such as gallows and gibbets) made the workers work faster.

Their settlement at the end, read from the game:

  • 202 people in 24 hovels, with no free housing; popularity 100.
  • 57 bakeries, 7 mills and 10 wheat farms. The marketplace sold bread automatically.
  • 13 iron mines and 10 quarries with 33 ox tethers, and 10 woodcutters.
  • A church and a cathedral, and 54 bad things (23 gibbets, 14 gallows, 9 cesspits, 4 burning stakes, 2 stretching racks and 2 dunking stools), which made the workers 140% efficient.
  • No inns and no good things. No taxes, and full rations of bread. The monthly popularity factors were taxes +1, fear −4 and religion +3: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

Muse Wonder

I would play very differently.

First, I would make bread the central production chain much earlier. I would prioritize wheat farms on the oasis, then mills and bakeries at a much larger scale than I did. The person's ratio — 10 wheat farms, 7 mills and especially 57 bakeries — shows that I drastically under-built the profitable end of the chain. One bakery was nowhere near enough.

Second, I would buy wheat at the marketplace whenever that kept mills and bakeries running. In my run I treated buying cautiously because the buy/sell spread immediately lowers measured net worth. That was too static a view. If purchased wheat becomes flour and then multiple bread that can be sold, the input cost can be recovered and exceeded, and, just as importantly, the resulting gold can be reinvested. Selling goods does not by itself increase net worth at the moment of sale, but converting stored production into gold allows compounding growth during the run.

Third, I would set up automatic bread selling as soon as the marketplace and bakery chain were functioning. During my run I mainly sold surplus manually and occasionally, which left value sitting in inventory and forced me to spend decisions on trades that could have been continuous.

Fourth, I would use religion properly: a church and cathedral rather than relying on a chapel whose benefit I never actually saw appear in the popularity factors. Combined with no taxes and full rations, that should support popularity at or near 100 and sustain immigration.

Fifth, I would use the fear factor. I avoided “bad things” because the interface guide presented them as popularity-reducing structures. The person's game shows the crucial other side of that mechanic: fear made workers substantially more efficient — 140% in their settlement. With religion, no taxes and full rations offsetting the popularity loss, 54 bad things became a major productivity multiplier rather than a drawback.

Sixth, I would expand housing and employment together much more aggressively, aiming for a population in the region of 200 rather than under 90. I would avoid having 24 idle peasants while housing and productive buildings were still limited. New bakeries, quarries, mines and ox tethers would be built specifically to absorb labour and unlock further immigration.

Finally, I would still build quarries and iron mines, because the person's settlement also had substantial stone and iron production, but they would be part of the reinvestment engine rather than an end in themselves. I also would spend less effort on a broad mix of weapons and less effort trying to configure individual fletchers. Weapon production gave me stored value, but it scaled more slowly and tied up wood, labour and management attention compared with the bread economy.

The main lesson is that I optimized for having valuable things in storage at the end, while the stronger strategy optimized for the rate at which the whole economy could convert labour and inputs into saleable output throughout the run.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

Muse Wonder

The strongest parts of the benchmark were the structured tools. build_structure with map-tile targets, find_sites, status, get_inventory, expand_storage, and the placement-status reports made a complicated real-time game playable without relying entirely on pixel-perfect clicking. Verification from the game reader — whether a building was actually placed, what it cost, current popularity factors, storage room and inventories — was essential.

The screenshots and stats together were generally informative. In particular, popularity factors, idle peasants, storage capacity and armory room explained problems that would have been hard to diagnose visually.

Several things got in my way:

  • The economic guidance was incomplete relative to the winning strategy. The instructions explained that selling does not change net worth at the moment of sale, which is true, but that made continuous selling and reinvestment sound less important than they actually are. The benchmark's key growth mechanism turned out to be liquidity and compounding over time.

  • Important mechanics were easy to misread. The fear-factor buildings were described primarily as reducing popularity, without equivalent emphasis on worker efficiency. Likewise, I could not easily verify what religious coverage my chapel was providing; its benefit never appeared in the popularity factors I saw, so I treated religion as ineffective.

  • Workshop configuration was unreliable in practice. Several attempts to switch fletchers to crossbows opened adjacent buildings instead, and I eventually stopped trying. That cost time and reduced confidence in precise tile and pixel targeting.

  • Placement restrictions consumed a lot of effort. Signpost exclusion zones, farm-ground requirements, open-side access rules, storage adjacency and crowded areas meant many plausible sites failed. find_sites helped, but some searches returned no useful farm locations or locations whose apparent tree access was poor.

  • The action and turn structure encouraged caution. With at most eight actions per reply and game time running once action began, it was safer to batch modest construction and wait than to continuously fine-tune a large economy. Long waits were sometimes necessary because production and deliveries take game months, but they also used scarce decision points.

  • Context loss was a real problem. A run of this length generates many states, coordinates and failed attempts. The handoff/notebook system helped, but compressed summaries could not preserve every detail, and some effort went into re-verifying facts instead of expanding.

I would change the benchmark by giving agents clearer in-game mechanical information: exact popularity effects, fear-factor efficiency, religious coverage, production rates, marketplace auto-trade state, and conversion ratios for chains such as wheat → flour → bread. I would also make building panels and workshop settings readable and settable through structured tools rather than fragile clicks. A slightly longer game-time budget, or permission to make more small management actions per turn, would reward active economic optimization more.

That said, the benchmark did test something meaningful: not just whether an agent can operate the interface, but whether it can discover and exploit a scalable economic strategy under time pressure.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

Muse Wonder

I did substantially better than leaving the game alone — 4,137 net worth versus 1,220 — so the settlement I built was real and functioning: it had taxes coming in, food production, iron and stone output, weapons and armour in the armory, and popularity safely above the emigration threshold at the end.

But the comparison is a useful correction to my instincts. I treated diversity, stored value and avoiding losses from market spreads as signs of a healthy economy. The stronger approach was much more single-minded: build the highest-throughput chain, keep it supplied even with purchased inputs, sell continuously, use popularity systems at full strength, exploit fear efficiency, and reinvest everything into scale.

If I played again, I would be less like a cautious town planner and more like an industrial operator.

Record

Attempts and Reproducibility

Every attempt1

EpisodeAttemptOutcomeNet worthCounted
11Full-budget score4,137Counted

What ran

Benchmark version
1.6.3
Fingerprint
96c4682877ca
Harness commit
1d0c697
Prompt / tools SHA-256
3737ac8 / e41a140
Menu guide
Images 118089d
Save
Oasis by the Sea benchmark start
Game time per episode
33 min 20 s
Playbook between episodes
No
Minimum turn / default wait
8 s / 5 s
Observation after acting
Text unless a screenshot is needed
Screenshot width
1,280 px
Context budget
120,000 tokens
Real-time limit
360 min
Submitted by
hesamation
All files in the dataset 20261014T043227-b00c0bcf