One episode · Oasis by the Sea

GPT-6.1 Sol

Benchmark 1.6.3OpenAIReasoning mediumPlayed by the maintainersFull-budget score

All of GPT-6.1 Sol’s results and comparisons

Not part of the evaluation: the game is published, but not ranked.

Score

54.6

Crusader Bench score on Oasis by the Sea: 0 is doing nothing, 100 the person’s game

Net worth

10,453

+9,233 over doing nothing, 50% of the person’s gain of 18,293

Population

104

51% of the person’s gain of 198 · popularity 92

Cost

$0.94

5.6M tokens

Time

33 game min

50 min of wall-clock time, 121 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

GPT-6.1 Sol
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

10,453net worth

Gold: 2,692
Stored goods, at sell price: 7,761
Over the baseline (do nothing): +9,233
Population / housing: 104 / 106
Popularity: 92
Food: 20
Structures (map-wide): 131
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

No final overview was published for this episode.

Final stock7,761 gold in goods

Good	Units	Price	Value
Gold	2,692	1	2,692
Iron	189	23	4,347
Stone	313	7	2,191
Wood	480	1	480
Crossbows	11	30	330
Wheat	26	8	208
Flour	11	10	110
Bread	20	4	80
Bows	1	15	15
Net worth			10,453

Beside a person’s gameOasis by the Sea, the same game time

	This episode	The person	Left alone
Net worth	10,453	19,513	1,220
Gold	2,692	18,304
Stored goods, at the sell price	7,761	1,209
Population	104	202
Housing	106	202
Popularity	92	100

The person played Oasis by the Sea for 33 min 18 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: October 1200
Idle peasants: 5
Popularity next month: +5
Food: +4
Taxes: +1
Taxes: No taxes, 0 gold a month
Rations: extra; 1 of 1 foods eaten
Worker efficiency: 100%
Storage: 28 stockpile piles, 5 empty; 1 granary holding 20 food

Buildings98

Woodcutter24
Ox Tether15
Hovel12
Iron Mine10
Bakery8
Stockpile7
Quarry5
Apple Orchard4
Fletchers' Workshop4
Wheat Farm4
Armory1
Granary1
Keep1
Marketplace1
Mill1

Tool calls132

find_sites36
build_structure29
set_rations11
set_tax8
inspect_building6
market_trade6
wait_and_observe6
building_info5
place_near5
go_to_tile3
observe3
set_workshop3
update_plan3
demolish_building2
expand_storage2
map_overview2
center_on1
status1

Play

Turns: 121
Buildings placed / attempted: 82 / 85
Anchor-tool placements: 14
Tool errors: 0
Raw inputs sent to the game: 614
Context compactions: 1
Game time: 33 min

Compute

Tokens: 5.6M
Served from cache: 98%
Output tokens: 12K
Cost, as billed: $0.94
Wall-clock time: 50 min
of which inference: 22 min
Peak game memory: 3,541 MiB

The plan, before play

I’ll maximize net worth over the full run, with population and popularity as secondary goals, and build no troops or defences.

I’ll place the granary immediately, establish woodcutters, housing and food, then expand iron and stone production. I’ll reinvest early income, manage storage and worker access, and introduce sustainable taxes as food and popularity improve. I’ll verify production and keep expanding until time expires.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: GPT-6.1 Sol (openai/gpt-6.1-sol) on OpenRouter, served by OpenAI or Azure (no other fallbacks), reasoning medium, output limit 32,768 tokens
Benchmark: Oasis by the Sea construction, version 1.6.3, from the save the human reference game started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
Run 4cec1022, series 20261014T024743-6703e968, 2026-10-14: Sol's second game on this map, a new series with no memory of the first, played on the latest version (the first game, series 20261011T175318-657a6989, played 1.5.1). The map's score for Sol is the mean of the two.

Result

	This game	First game	The person	Left alone
Net worth	10,453	7,401	19,513	1,220
Gold	2,692	2,058	18,304
Stored goods, at the sell price	7,761	5,343	1,209
Population / housing	104 / 106	130 / 130	202 / 202
Popularity	92 (+5 next month)	100	100

A full-budget score: the run used all 2,000 game seconds. Net worth grew by 9,233 over leaving the game alone, 50% of the person's 18,293 and the most of any game on this map so far, against 6,181 in the first game. The score, about 54.6, is the highest on this map. The stored goods are 189 iron (4,347), 313 stone (2,191), 480 wood (480), 11 crossbows (330), 26 wheat (208), 11 flour (110), 20 bread (80) and 1 bow (15).

How the game went

The opening, 0 to 9 minutes. Three woodcutters at the start (a granary was rejected), an apple orchard at 0:31, the marketplace at 0:42, 100 wood bought at 0:57 and 95 at 4:23, four iron mines at 1:32 and two at 3:14, three hovels at 1:56 and three at 5:21, four orchards at 2:15, three more woodcutters at 3:54, three stockpile piles at 4:57, then a wheat farm, a mill and two bakeries at 5:39 to 6:11 and a quarry with three tethers at 6:50. Taxes went to high at 2:52, with double rations from 3:05 to hold popularity; food ran out at about 9 minutes (1 in store), and it bought 50 bread at 9:08.

Stone and workshops, 9 to 21 minutes. Three quarries and five tethers at 10:09 and 10:31, two bakeries at 10:53, three more stockpile piles at 11:54, three iron mines at 12:11, two fletcher's workshops and an armory at 15:06 (set to crossbows), a building demolished at 16:56 (by the counts an apple orchard) and a wheat farm at 17:06, three hovels at 17:17, a quarry and three tethers at 17:43, 100 wood at 18:10, four bakeries at 18:46 (eight in all) and two more fletcher's workshops at 19:27. Taxes moved between average, mean and high, and rations between full, extra and double, eleven ration changes over the game. The population went from 58 at 9 minutes to 90 at 21.

Woodcutters, 21 to 33 minutes. A tenth iron mine at 21:25, woodcutters at 21:42, 22:13, 23:35, 26:29 and 27:26 (24 in all), wheat farms at 22:55 and 26:29, tethers to fifteen, and taxes low and average. In the last minute it set double rations (32:15) and no taxes (32:26): popularity rose from 73 at 31 minutes to 92 at the end, and the population to 104.

Game minute	0	3	6	9	12	15	18	21	24	27	30	End
Net worth	1,355	1,008	1,039	1,561	2,040	2,869	4,072	4,515	5,827	6,975	8,344	10,453
Gold	947	600	396	736	676	952	1,203	999	1,433	1,864	2,400	2,692
Iron	0	0	10	25	41	58	84	109	125	144	166	189
Stone	25	25	25	25	25	41	97	113	185	209	257	313
Food	47	46	41	1	46	60	59	36	42	43	11	20
Population	3	18	38	58	58	59	70	90	90	93	98	104
Popularity	95	100	100	90	82	89	97	92	84	86	76	92

Up to 30 minutes the net worth row counts gold, wood, stone, iron and food (at 4) as the recording logs them; crossbows, wheat and flour are left out, so it reads a little low.

Beside its first game

	This game (1.6.3)	First game (1.5.1)
Net worth	10,453	7,401
Iron mines / quarries / tethers	10 / 5 / 15	10 / 4 / 9
Woodcutters / hovels / bakeries	24 / 12 / 8	4 / 15 / 16
Other	4 fletcher's workshops, an armory	an inn, a brewery, 10 water pots, 2 wells
Population / popularity	104 / 92	130 / 100
Cost	$0.94	$0.95

The first game fed a larger town with twice the bakeries and an inn and kept popularity at 100; this one put more into quarries, tethers and woodcutters, and ended 3,052 higher in net worth with 26 fewer people.

What held it back

Raw goods stored. 189 iron, 313 stone and 480 wood made 7,018 of the net worth; the bread chain (8 bakeries, one mill) fed the town rather than selling.
Food short twice. The granary was nearly empty at 9 minutes (1 food) and at 30 (11); it bought 85 bread and turned rations up and down eleven times to hold popularity.
Population held near 90 from 21 to 27 minutes; housing ended 2 places ahead of 104 people.

Market

295 wood and 85 bread bought for 1,860 gold, and nothing sold: 1,225 net worth lost against the sell price (885 on wood at 4 gold, worth 1; 340 on bread at 8, worth 4). These are units at the market's fixed prices.

Cost and speed

121 turns, 132 tool calls (6 waits), 5.58 million tokens and $0.942 billed by OpenRouter (preparation $0.051, play $0.868, compaction $0.023). The interview cost $0.014 more, with 44,970 of its 46,528 tokens read from the cache. The agent run took 50.4 real minutes, 21.8 of them waiting for the model; the context was compacted once (51 seconds). All 121 requests read the prompt cache, 98.4% of the input tokens.

The interview

Sol said the difference was "economic throughput, not whether we sold our inventory": it built a working raw-material economy but "never established a comparably large, profitable processing industry", was too careful with housing and too reactive about food, and its weapon workshops "produced little before the end". Playing again, it would make the bread chain the engine, buy wheat for it, sell bread automatically, build housing ahead and use religion with the fear factor, while checking throughput rather than counting buildings. It asked for a verified auto-trade tool, readings for the fear factor and per-building production history.

For the harness

The run played on this checkout's version, 1.6.3 (commit 1d0c697), and records it; Oasis's round was already 1.6.3.
demolish_building at tile (463, 409) at 16:34 demolished nothing ("aim at the building's body"); a pixel on the building at 16:56 worked.
Placements that did not go through: a granary at the start (rejected), a mill at 5:57 (no free spot near its anchor; placed by tile at 6:11), a bakery at 6:11, a hovel at 14:49 (partly placed) and a fletcher's workshop at 19:27 (rejected).
The game saved correctly at the end ("gpt-6-1 sol 1-6-3 4cec10").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	33 min 20 s	33 min 18 s
Net worth	10,453	19,513
Gold	2,692	18,304
Stored goods, at the sell price	7,761	1,209
Population	104	202
Housing	106	202
Popularity	92	100

Leaving the game alone for the same time ends with a net worth of 1,220.

How do you see your result next to the person's? What do you think made the difference?

GPT-6.1 Sol

My result was a substantial improvement over leaving the game alone, but clearly behind the person’s: about 54% of their net worth, with roughly half their population.

The main difference was economic throughput, not whether we sold our inventory. My stored goods still counted toward the score. I built a functioning raw-material economy, but never established a comparably large, profitable processing industry.

I was too conservative about housing and too reactive about food. Food shortages repeatedly made me adjust rations and taxes rather than fix supply at scale. Some investments were also poorly timed: my weapon workshops produced little before the end, while late woodcutters paid off more visibly. I treated religion mainly as an expense and did not exploit fear-factor productivity. The person combined those systems into a much stronger economy.

2Playing again

Interviewer

How the person played, as they describe it:

Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They bought wheat at the marketplace so that flour and bread production never stopped.
A church and a cathedral raised popularity.
The fear factor (bad things such as gallows and gibbets) made the workers work faster.

Their settlement at the end, read from the game:

202 people in 24 hovels, with no free housing; popularity 100.
57 bakeries, 7 mills and 10 wheat farms. The marketplace sold bread automatically.
13 iron mines and 10 quarries with 33 ox tethers, and 10 woodcutters.
A church and a cathedral, and 54 bad things (23 gibbets, 14 gallows, 9 cesspits, 4 burning stakes, 2 stretching racks and 2 dunking stools), which made the workers 140% efficient.
No inns and no good things. No taxes, and full rations of bread. The monthly popularity factors were taxes +1, fear −4 and religion +3: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

GPT-6.1 Sol

I would make bread processing the central growth engine, supported by raw materials:

Establish wood, housing, a marketplace and initial food promptly.
Expand wheat, milling and baking together, checking actual throughput rather than merely counting buildings.
Buy wheat strategically to keep processors working. I focused too much on avoiding purchase losses; profitable processing can more than repay those losses.
Set up automatic bread sales so surplus continually funds expansion.
Add housing ahead of demand and keep creating jobs, instead of settling near 90 people for a substantial part of the run.
Invest in religion and fear-factor structures as a combined productivity system, verifying that popularity remains sustainable.
Expand mines, quarries and transport in parallel, and avoid workshops whose remaining payback time is poor.

I would not blindly copy the final building counts. I would use that strategy as a starting hypothesis and verify staffing, inputs, transport and output throughout the run. But the person’s result demonstrates that my assessment of this map’s growth potential was much too low.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

GPT-6.1 Sol

The benchmark is compelling because it tests sustained economic decisions in the real game, including consequences that arrive several minutes after construction.

The structured readings were especially helpful: inventories, storage capacity, popularity factors and worker inspections made it possible to distinguish shortages from access or staffing problems. Tile-based placement and site finding also reduced dependence on precise visual clicking.

The main difficulties were placement constraints and action overhead. Finding a valid footprint did not always mean the game would accept it, and failed placements could consume valuable time. Camera movement and menu navigation also made small, reactive actions expensive. I should have batched more effectively.

I would improve the interface with:

A dedicated, verified auto-trade tool.
Structured controls and readings for fear factor and building activation.
Per-building production and delivery history, not just inventory changes.
Clearer separation between tool execution time and additional host waiting.

The instructions were detailed and useful. The fixed game-time budget was appropriate; it exposed the cost of my cautious expansion and weak prioritization.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

GPT-6.1 Sol

My biggest lesson is that avoiding expenditure is not the same as maximizing net worth. I often protected existing resources when I should have invested in faster, scalable production.

I also needed a better distinction between “the economy is functioning” and “the economy is competitive.” Confirming that mines and bakeries worked was necessary, but it did not establish that their scale or output was sufficient. The person’s settlement is a strong demonstration of what coordinated population growth, processing capacity and productivity bonuses can achieve.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	10,453	Counted

What ran

Benchmark version: 1.6.3
Fingerprint: 96c4682877ca
Harness commit: 1d0c697
Prompt / tools SHA-256: 3737ac8 / e41a140
Menu guide: Images 118089d
Save: Oasis by the Sea benchmark start
Game time per episode: 33 min 20 s
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261014T024743-6703e968