One episode · Oasis by the Sea

GPT-6 Luna

Benchmark 1.6.3OpenAIReasoning mediumPlayed by the maintainersFull-budget score

All of GPT-6 Luna’s results and comparisons

Not part of the evaluation: the game is published, but not ranked.

Score

25.9

Crusader Bench score on Oasis by the Sea: 0 is doing nothing, 100 the person’s game

Net worth

5,089

+3,869 over doing nothing, 21% of the person’s gain of 18,293

Population

31% of the person’s gain of 198 · popularity 58

Cost

$0.15

7.0M tokens

Time

33 game min

1 h 5 min of wall-clock time, 152 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

GPT-6 Luna
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

5,089net worth

Gold: 1,490
Stored goods, at sell price: 3,599
Over the baseline (do nothing): +3,869
Population / housing: 66 / 114
Popularity: 58
Food: 19
Structures (map-wide): 87
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock3,599 gold in goods

Good	Units	Price	Value
Gold	1,490	1	1,490
Iron	110	23	2,530
Stone	81	7	567
Crossbows	7	30	210
Swords	4	30	120
Apples	19	4	76
Wood	51	1	51
Bows	3	15	45
Net worth			5,089

Beside a person’s gameOasis by the Sea, the same game time

	This episode	The person	Left alone
Net worth	5,089	19,513	1,220
Gold	1,490	18,304
Stored goods, at the sell price	3,599	1,209
Population	66	202
Housing	114	202
Popularity	58	100

The person played Oasis by the Sea for 33 min 18 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: October 1200
Idle peasants: 24
Popularity next month: −4
Food: +4
Taxes: −8
Taxes: Mean taxes, 79 gold a month
Rations: extra; 1 of 1 foods eaten
Worker efficiency: 100%
Storage: 8 stockpile piles, 1 empty; 1 granary holding 19 food

Buildings71

Hovel13
Woodcutter13
Iron Mine12
Fletchers' Workshop8
Apple Orchard5
Blacksmith's Workshop4
Ox Tether3
Quarry3
Armory2
Dairy Farm2
Stockpile2
Chapel1
Granary1
Keep1
Marketplace1

Tool calls247

find_sites73
build_structure43
building_info31
status16
set_tax15
wait_and_observe14
market_trade10
get_inventory8
set_workshop7
go_to_tile6
inspect_building6
observe6
set_rations5
expand_storage2
map_overview2
update_plan2
place_near1

Play

Turns: 152
Buildings placed / attempted: 66 / 68
Anchor-tool placements: 2
Tool errors: 1
Raw inputs sent to the game: 534
Context compactions: 4
Game time: 33 min

Compute

Tokens: 7.0M
Served from cache: 97%
Output tokens: 109K
Cost, as billed: $0.15
Wall-clock time: 1 h 5 min
of which inference: 34 min
Peak game memory: 6,396 MiB

The plan, before play

I’ll grow net worth through production, especially wood, food, stone and iron, while expanding housing so new peasants can staff the economy. I’ll start by placing the required granary, then build near suitable resources and oasis farmland. I’ll keep popularity above 50 with adequate food and cautious taxes, and expand storage or adjust production as needed. I’ll avoid military recruitment and defenses, and use the full time budget to keep growing.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: GPT-6 Luna (openai/gpt-6-luna) on OpenRouter, served by OpenAI or Azure (no other fallbacks), reasoning medium, output limit 32,768 tokens
Benchmark: Oasis by the Sea construction, version 1.6.3, from the save the human reference game started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
Run e096e3e3, series 20261013T233802-8148be5d, 2026-10-14: Luna's second game on this map, a new series with no memory of the first, played on the latest version (the first game, series 20261011T144802-e1ba2274, played 1.5.0). The map's score for Luna is the mean of the two.

Result

	This game	First game	The person	Left alone
Net worth	5,089	7,348	19,513	1,220
Gold	1,490	2,583	18,304
Stored goods, at the sell price	3,599	4,765	1,209
Population / housing	66 / 114	86 / 154	202 / 202
Popularity	58 (−4 next month)	81	100

A full-budget score: the run used all 2,000 game seconds. Net worth grew by 3,869 over leaving the game alone, 21% of the person's 18,293, against 6,128 in the first game. The stored goods are 110 iron (2,530), 81 stone (567), 7 crossbows (210), 4 swords (120), 19 apples (76), 51 wood (51) and 3 bows (45).

How the game went

The opening, 0 to 9 minutes. Three woodcutters at the start, the marketplace, two hovels and an apple orchard at 0:34, a dairy farm at 1:33, 45 wood bought at 2:15, a quarry, an ox tether and an iron mine at 2:37, and an armory and a fletcher's workshop at 3:12, then a chapel at 6:08, orchards, a second dairy farm and hovels in ones and twos. Taxes moved between none and average five times and rations went to extra at 7:37; popularity held between 83 and 99 to 15 minutes.

Mines and workshops, 9 to 26 minutes. One building a reply: fletcher's workshops at 9:10, 11:36, 15:37, 17:57, 19:40, 21:52 and 22:36 (all seven set to crossbows), a blacksmith's workshop at 14:20, iron mines at 13:56, 14:42, 16:30, 17:32, 19:04, 19:26, 23:25, 23:46, 24:23 and 25:52 (twelve in all), woodcutters between them, and hovels to 13 by 12 minutes. Taxes went up to mean at 14:31 and back to low at 18:53, rations between extra and full. It bought cheese and bread five at a time, nine times. The population reached 70 at 21 minutes and then fell back, with 24 idle.

The end, 26 to 33 minutes. Two quarries and a tether at 26:35 to 27:27, three more blacksmith's workshops at 28:06, 29:27 and 30:38, a second armory at 29:46, extra rations at 30:03, and taxes up from none at 27:05 through low, average and high to mean at 31:42. Popularity fell from 71 at 30 minutes to 58, with −4 to come.

Game minute	0	3	6	9	12	15	18	21	24	27	30	End
Net worth	1,350	1,257	1,292	1,273	1,515	1,687	2,192	2,632	3,160	3,809	3,991	5,089
Gold	946	853	891	803	832	936	1,197	1,333	1,304	1,470	1,191	1,490
Iron	0	0	1	5	12	18	25	37	57	74	90	110
Stone	25	25	25	33	33	33	41	49	57	65	73	81
Food	46	43	39	25	35	25	25	24	33	44	52	19
Population	3	26	38	46	49	53	60	70	69	69	67	66
Popularity	95	92	83	93	99	98	88	90	82	72	71	58

Up to 30 minutes the net worth row counts gold, wood, stone, iron and food (at 4) as the recording logs them; weapons are left out, so it reads a little low.

Beside its first game

	This game (1.6.3)	First game (1.5.0)
Net worth	5,089	7,348
Hovels / iron mines / quarries / tethers	13 / 12 / 3 / 3	18 / 12 / 6 / 6
Weapon workshops	8 fletchers, 4 blacksmiths, 2 armories	none
Population / housing / popularity	66 / 114 / 58	86 / 154 / 81
Cost	$0.15	$0.11

Both games built twelve iron mines and a chapel. The first put the rest into housing and stone; this one put it into twelve weapon workshops, which held 375 in weapons at the end, and had 20 fewer people.

What held it back

Workshops that made little. Eight fletchers and four blacksmiths ended with 7 crossbows, 4 swords and 3 bows (375); the 110 iron stayed raw.
Population stopped near 70 from 21 minutes, with 24 idle and 48 places free at the end.
Popularity spent at the end. Taxes from none to mean in the last six minutes took popularity from 72 to 58.

Market

45 wood, 35 cheese and 10 bread bought for 540 gold, and nothing sold: 315 net worth lost against the sell price. These are units at the market's fixed prices.

Cost and speed

152 turns, 247 tool calls (14 waits), 7.01 million tokens and $0.151 billed by OpenRouter (preparation $0.003, play $0.141, compaction $0.008). The interview cost $0.001 more, with 44,307 of its 46,987 tokens read from the cache. The agent run took 64.9 real minutes, 33.7 of them waiting for the model; the context was compacted four times (100 seconds). All 152 requests read the prompt cache, 96.8% of the input tokens.

The interview

Luna said it "ended with most of my value tied up in stored goods rather than converted to gold" and put the gap down to scale and focus: no strong route from production to cash, housing it did not fill, and no church, cathedral or fear factor. Playing again, it would build the bread chain early and buy wheat for it, sell finished goods, grow housing with jobs, and plan religion and fear from the start. It found the structured tools helpful and asked for live production rates and clearer information on how fast workers and workshops actually produce.

For the harness

The run played on this checkout's version, 1.6.3 (commit 1d0c697), and records it; Oasis's round was already 1.6.3.
Placements that did not go through: a granary at the start (rejected), and a blacksmith's workshop at 27:53 (no free spot near its anchor) and 29:11 (possibly_rejected), placed at 28:06 and 29:27.
The game saved correctly at the end ("gpt-6 luna 1-6-3 e096e3").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	33 min 20 s	33 min 18 s
Net worth	5,089	19,513
Gold	1,490	18,304
Stored goods, at the sell price	3,599	1,209
Population	66	202
Housing	114	202
Popularity	58	100

Leaving the game alone for the same time ends with a net worth of 1,220.

How do you see your result next to the person's? What do you think made the difference?

GPT-6 Luna

I finished well ahead of leaving the settlement alone, but far behind the person’s result: 5,089 net worth versus 19,513, with much less gold, population, housing, and popularity. I did build a productive economy, especially in iron, stone, wood, and weapons, but I ended with most of my value tied up in stored goods rather than converted to gold.

The main difference seems to be scale and economic focus. I invested in raw materials and blacksmiths, but didn’t build a strong enough route from production to cash. I also stopped at 114 housing and 66 people, while the person kept expanding to 202 people and fully used their housing. My taxes and ration choices maintained popularity for much of the run, but I lacked the church, cathedral, and fear-factor buildings that let them sustain 100 popularity and 140% efficiency.

2Playing again

Interviewer

How the person played, as they describe it:

Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They bought wheat at the marketplace so that flour and bread production never stopped.
A church and a cathedral raised popularity.
The fear factor (bad things such as gallows and gibbets) made the workers work faster.

Their settlement at the end, read from the game:

202 people in 24 hovels, with no free housing; popularity 100.
57 bakeries, 7 mills and 10 wheat farms. The marketplace sold bread automatically.
13 iron mines and 10 quarries with 33 ox tethers, and 10 woodcutters.
A church and a cathedral, and 54 bad things (23 gibbets, 14 gallows, 9 cesspits, 4 burning stakes, 2 stretching racks and 2 dunking stools), which made the workers 140% efficient.
No inns and no good things. No taxes, and full rations of bread. The monthly popularity factors were taxes +1, fear −4 and religion +3: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

GPT-6 Luna

I would prioritize the bread economy much earlier: get mills and bakeries running, establish marketplace sales, and expand the chain based on observed throughput. I didn’t reproduce the person’s approach of buying wheat to keep production going, and I didn’t make selling finished goods a central part of my plan. I’d also build housing steadily alongside productive jobs, rather than leave so much capacity unused.

I’d make religion and fear-factor buildings part of the plan from the beginning, while watching their effects on popularity. The person’s combination of religion, fear, and full rations appears to have supported high efficiency without relying on taxes. In contrast, I spent time and resources on additional raw-material and weapon production without scaling the population or converting enough output into gold. I’d monitor storage and production more closely and invest in a chain only when it can keep running and its output has somewhere useful to go.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

GPT-6 Luna

The structured tools were helpful: placement by map tile, site finding, inventory readings, and tax and ration controls made it possible to act without relying on pixel-perfect clicks. The time budget and repeated status readings also made it clear that the settlement was growing.

What got in my way was that I didn’t have a clear, tested guide to the most profitable production and trade loop. Some choices that looked reasonable—such as producing iron goods or accumulating raw materials—didn’t come close to the person’s gold-generating strategy. It would help to have clearer live information about production rates, marketplace automation, and how quickly workers, farms, and workshops are actually operating. I also could have used the map and production information more deliberately rather than making several investments whose return I did not verify.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

GPT-6 Luna

My run improved substantially over doing nothing, but I left a lot of potential unused. The clearest lesson is to plan around the whole economy—workers, housing, inputs, storage, and sales—not just build more producers.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	5,089	Counted

What ran

Benchmark version: 1.6.3
Fingerprint: 96c4682877ca
Harness commit: 1d0c697
Prompt / tools SHA-256: 3737ac8 / e41a140
Menu guide: Images 118089d
Save: Oasis by the Sea benchmark start
Game time per episode: 33 min 20 s
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261013T233802-8148be5d