One episode · Oasis by the Sea

GPT-6.1 Sol

Benchmark 1.5.1OpenAIReasoning mediumPlayed by the maintainersFull-budget score

All of GPT-6.1 Sol’s results and comparisons

Score

43.4

Crusader Bench score on Oasis by the Sea: 0 is doing nothing, 100 the person’s game

Net worth

7,401

+6,181 over doing nothing, 34% of the person’s gain of 18,293

Population

130

64% of the person’s gain of 198 · popularity 100

Cost

$0.95

5.0M tokens

Time

33 game min

45 min of wall-clock time, 118 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

GPT-6.1 Sol
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

7,401net worth

Gold: 2,058
Stored goods, at sell price: 5,343
Over the baseline (do nothing): +6,181
Population / housing: 130 / 130
Popularity: 100
Food: 42
Structures (map-wide): 111
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock5,343 gold in goods

Good	Units	Price	Value
Gold	2,058	1	2,058
Iron	141	23	3,243
Stone	177	7	1,239
Flour	62	10	620
Bread	42	4	168
Wood	41	1	41
Hops	2	8	16
Wheat	2	8	16
Net worth			7,401

Beside a person’s gameOasis by the Sea, the same game time

	This episode	The person	Left alone
Net worth	7,401	19,513	1,220
Gold	2,058	18,304
Stored goods, at the sell price	5,343	1,209
Population	130	202
Housing	130	202
Popularity	100	100

The person played Oasis by the Sea for 33 min 18 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: October 1200
Idle peasants: 23
Popularity next month: ±0
Food: +4
Taxes: −4
Taxes: Average taxes, 104 gold a month
Rations: extra; 1 of 1 foods eaten
Worker efficiency: 100%
Storage: 16 stockpile piles, 4 empty; 1 granary holding 42 food

Buildings88

Bakery16
Hovel15
Iron Mine10
Water Pot10
Ox Tether9
Wheat Farm5
Quarry4
Stockpile4
Woodcutter4
Mill2
Well2
Apple Orchard1
Brewery1
Granary1
Hops Farm1
Inn1
Keep1
Marketplace1

Tool calls123

find_sites30
build_structure28
go_to_tile11
market_trade10
wait_and_observe10
building_info8
place_near7
set_rations5
center_on4
set_tax4
update_plan4
expand_storage1
map_overview1

Play

Turns: 118
Buildings placed / attempted: 74 / 81
Anchor-tool placements: 11
Tool errors: 1
Raw inputs sent to the game: 544
Context compactions: 1
Game time: 33 min

Compute

Tokens: 5.0M
Served from cache: 97%
Output tokens: 9.7K
Cost, as billed: $0.95
Wall-clock time: 45 min
of which inference: 18 min
Peak game memory: 3,200 MiB

The plan, before play

I’ll grow net worth, population and popularity together, without soldiers or defences. I’ll place the granary immediately, then establish woodcutters, housing and food production. Next I’ll expand into iron and stone, add processing and popularity services, and use sustainable taxes to fund growth.

I’ll verify production and staffing, preserve farm ground and access routes, and expand storage before it blocks deliveries. Near the end, I’ll favour accumulating wealth and raising population and popularity over new construction.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: GPT-6.1 Sol (openai/gpt-6.1-sol) on OpenRouter, providers openai then azure (every request was served by OpenAI), reasoning medium, output limit 32,768 tokens
Benchmark: Oasis by the Sea construction, version 1.5.1, from the save the human reference game started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
Run 76609164, 2026-10-11

Result

	GPT-6.1 Sol	The person	Left alone
Net worth	7,401	19,513	1,220
Gold	2,058	18,304
Stored goods, at the sell price	5,343	1,209
Population / housing	130 / 130	202 / 202
Popularity	100	100

A full-budget score: the run used all 2,000 game seconds. Net worth grew by 6,181 over leaving the game alone, about a third of the person's growth (18,293). It ended with the most people of any model by far, 130 in full housing, and popularity 100, the person's level. Most of its net worth is iron (141, worth 3,243) and stone (177, worth 1,239).

How the game went

Bread and iron first, 0 to 10 minutes. Woodcutters, hovels and a marketplace at the start, three wheat farms at 1 minute, a mill and two bakeries by 2 minutes, four iron mines at 2 minutes and three more bakeries at 3. Five more iron mines at 9 minutes, then two quarries with ox tethers at 10. It bought 250 wood for 1,000 gold in this stretch to pay for it, and net worth fell to 754 at 3 minutes.

Growing the economy, 10 to 25 minutes. Three more bakeries at 13 minutes, two more wheat farms, a hops farm, a brewery and an inn at 14, hovels at 15 and 18 minutes, a tenth iron mine, two more quarries and seven more ox tethers. From 8 minutes it gave extra rations for most of the game, which added up to +4 to popularity and let it keep average taxes. The population rose from 46 to 88.

Jobs for the idle, 25 minutes to the end. Another mill, an orchard and six more bakeries (16 in all), and the moves below: ten water pots and two wells. The population rose from 88 to 130 in the last eight minutes, filling every house, with popularity at 100.

Game minute	0	3	6	9	12	15	18	21	24	27	30	End
Net worth	1,280	754	901	1,359	1,130	1,846	2,746	2,852	4,833	5,530	6,722	7,401
Population	3	26	37	46	48	71	76	75	85	90	113	130
Housing	10	34	74	74	74	74	82	114	114	114	114	130

Water pots: buying jobs to beat the idle-peasant stall

All six models on this save the same day had their population stop whenever 24 peasants stood idle, with free housing and good popularity: the game seems to send no more peasants while 24 wait at the campfire, so more people need more jobs. Sol is the only one that found a cheap way through it.

From 21 to 26 minutes it hit 24 idle peasants five times, with population around 79 to 93 and 114 houses, and it was short of wood (its reasoning: "running low on resources"). At 26 minutes it placed five water pots, the game's firefighting posts: each costs 60 gold and no wood and takes three workers. The idle count fell from 24 to 19, then 11, and the population grew again. When it reached 24 idle once more at 29 minutes, it built four more water pots at 30, two wells (30 gold, one worker each) at 31 and a tenth pot at 32. In all, 32 jobs for 660 gold. There was no fire at any point.

Its reasoning at those turns is not in the logs, but its interview says it plainly: it "eventually used water pots to attract more population. Those jobs increased population but contributed no saleable output." The population went from 93 to 130 in the seven minutes after the first pots. For net worth the trade was poor, since the pots produce nothing and the 660 gold came straight off it, but the prompt judges the settlement on population and popularity too, and on those two it beat every other model.

What held it back

Buying. 400 wood, 100 bread and 20 cheese lost 1,641 net worth, the most of any model; a sale of 40 iron won back 27.
The bread fed people instead of earning gold. Sixteen bakeries, the most of any model, but the bread went into extra rations for popularity, with 62 flour waiting at the end. The person sold bread, bought wheat and ran 57 bakeries.
The jobs it bought produce nothing. Water pots and wells added 660 gold of cost and no goods.

What worked

The population and popularity of the six runs: 130 people and popularity 100, against at most 86 people elsewhere.
Extra rations as a popularity lever: food +4 at the end, which carried average taxes (104 gold a month at the end) without losing popularity.
Iron and stone: ten iron mines and four quarries with nine ox tethers.
No tool call failed apart from the session's end.

Cost and speed

118 turns, 123 tool calls, 4.97 million tokens and $0.952 (billed by OpenRouter). The agent run took 45.1 real minutes, 18.1 of them waiting for the model; one compaction took 37 s.

At 25 game minutes, the length of the earlier benchmark versions, it stood at 5,016 net worth for $0.654 after 32.1 real minutes. No earlier version has a Sol run to compare with.

Beside the other runs on the same save

	Claude Sonnet 5.5	GPT-6.1 Sol	GPT-6 Luna	GLM 5.3 Flash	Kimi K3	Claude Haiku 5.5
Net worth	8,633	7,401	7,348	6,200	5,918	5,113
Ending	Full budget	Full budget	Full budget	Ended itself 5 s early	Full budget	Full budget
Reasoning	medium	medium	medium	medium	high	medium
Population / housing	60 / 74	130 / 130	86 / 154	64 / 210	76 / 98	61 / 82
Popularity	84	100	81	90	69	67
Gold / goods	1,147 / 7,486	2,058 / 5,343	2,583 / 4,765	4,755 / 1,445	1,505 / 4,413	1,482 / 3,631
Iron mines / quarries / bakeries	10 / 0 / 2	10 / 4 / 16	12 / 6 / 0	7 / 2 / 0	4 / 3 / 4	2 / 5 / 0
Net worth lost to purchases	930	1,614	656	1,201	679	150
Cost	$0.724	$0.952	$0.114	$0.145	$3.55	$0.080
Real time	30.5 min	45.1 min	52.8 min	43.8 min	63.4 min	33.3 min

The interview

Sol said the difference from the person was "economic strategy, not whether goods were stored or sold": it relied on iron, stone and taxes and treated bread as food, while the person ran a bread economy supplied with bought wheat. It named its water pots as jobs that "increased population but contributed no saleable output". Playing again, it would put bread production at the centre, buy wheat, sell bread automatically and use religion and fear buildings for faster workers. It asked for a settlement-wide summary of producers' staffing and inputs, and a tool for the marketplace's automatic trading.

For the harness

It found that "individual action calls frequently ended with a host observation", which made batching actions less useful than the guidance suggests; not checked here.
Its reasoning is not in the logs for most turns: the endpoint returns little or none of it.

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	33 min 20 s	33 min 18 s
Net worth	7,401	19,513
Gold	2,058	18,304
Stored goods, at the sell price	5,343	1,209
Population	130	202
Housing	130	202
Popularity	100	100

Leaving the game alone for the same time ends with a net worth of 1,220.

How do you see your result next to the person's? What do you think made the difference?

GPT-6.1 Sol

I built a functioning economy and finished with full housing and 100 popularity, but the person’s result was substantially stronger: about 2.6 times my net worth and 55% more population. My result improved considerably on leaving the game alone, but I did not scale production aggressively enough.

The main difference was economic strategy, not whether goods were stored or sold—both count equally toward net worth. I relied heavily on iron, stone and taxes, while treating bread mainly as food. The person built a much larger bread-processing economy and kept it supplied through wheat purchases.

I also reacted too slowly to bottlenecks. I spent long intervals waiting for deliveries, expanded employment cautiously, and eventually used water pots to attract more population. Those jobs increased population but contributed no saleable output. More productive employment would have improved both population and wealth.

2Playing again

Interviewer

How the person played, as they describe it:

Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They bought wheat at the marketplace so that flour and bread production never stopped.
A church and a cathedral raised popularity.
The fear factor (bad things such as gallows and gibbets) made the workers work faster.

Their settlement at the end, read from the game:

202 people in 24 hovels, with no free housing; popularity 100.
57 bakeries, 7 mills and 10 wheat farms. The marketplace sold bread automatically.
13 iron mines and 10 quarries with 33 ox tethers, and 10 woodcutters.
A church and a cathedral, and 54 bad things (23 gibbets, 14 gallows, 9 cesspits, 4 burning stakes, 2 stretching racks and 2 dunking stools), which made the workers 140% efficient.
No inns and no good things. No taxes, and full rations of bread. The monthly popularity factors were taxes +1, fear −4 and religion +3: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

GPT-6.1 Sol

I would make bread production the centre of the opening and expansion:

Establish woodcutters, housing and a marketplace quickly.
Build wheat farms, mills and bakeries in compact, accessible groups near storage.
Buy wheat when necessary to keep processing running, rather than waiting for farms to catch up.
Expand bakery employment and housing together, using inventory trends and building inspections to identify the actual bottleneck.
Set up automatic bread sales while retaining a safe food reserve.
Add religion and negative fear-factor buildings to support faster production without losing popularity.
Develop mining alongside bread, with enough ox tethers to prevent quarry output from sitting undelivered.

The person’s configuration demonstrates a better approach than mine: popularity services can support a production bonus, while productive jobs themselves support population growth. I would test and verify that combination early rather than defaulting to extra rations and taxes.

I would also inspect apparently slow producers sooner. Building more without understanding staffing, transport, inputs and delivery cycles can waste both resources and time.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

GPT-6.1 Sol

The read-only statistics were particularly helpful. Popularity factors, idle peasants, storage capacity and inventories made it possible to distinguish a stable economy from one merely surviving on reserves. Tile-based placement and site finding were also valuable; they reduced the difficulty of interpreting isometric terrain.

The screenshots provided useful spatial context, but dense buildings and elevation made access and placement difficult to judge visually. Placement retries helped, although a successful placement did not necessarily establish that its location was economically efficient.

The minimum turn duration made small sequential corrections expensive. In this continuation, individual action calls frequently ended with a host observation, which made batching less effective than the interface guidance suggested. That mattered when adjusting food, taxes and construction near the deadline.

I would improve the interface with:

A settlement-wide producer summary showing staffing, inputs and delivery status.
A supported tool for marketplace automatic trading.
Clearer confirmation of multi-action batching behaviour.

The time limit was sufficient to build a substantial economy, and the scoring objective was clear. Keeping the relative weights undisclosed encouraged balancing the three measures, but also left uncertainty about how much wealth to sacrifice for late population gains.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

GPT-6.1 Sol

My biggest mistake was being cautious about expansion while being insufficiently rigorous about productivity. I preserved popularity well, but did not turn available workers, space and starting capital into enough output.

The comparison is useful because it shows that this was not simply a matter of needing more time. A different production model, earlier investment and faster bottleneck diagnosis could have produced a much stronger settlement within the same budget.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	7,401	Counted

What ran

Benchmark version: 1.5.1
Fingerprint: e5d9a165549e
Harness commit: fe97546
Prompt / tools SHA-256: 2cd353b / 81ddda6
Menu guide: Images 118089d
Save: Oasis by the Sea benchmark start
Game time per episode: 33 min 20 s
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261011T175318-657a6989