One episode · Large Barren Desert

GPT-6 Luna

Benchmark 1.6.3OpenAIReasoning mediumPlayed by the maintainersFull-budget score

All of GPT-6 Luna’s results and comparisons

Not part of the evaluation: the game is published, but not ranked.

Score

-3.4

Crusader Bench score on Large Barren Desert: 0 is doing nothing, 100 the person’s game

Net worth

4,409

−783 over doing nothing, -16% of the person’s gain of 4,792

Population

19% of the person’s gain of 246 · popularity 78

Cost

$0.07

2.5M tokens

Time

33 game min

45 min of wall-clock time, 60 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

GPT-6 Luna
Baseline (do nothing), final
A person's game, final

Net Worth

gold plus stored goods at the sell price

Gold or Goods

Gold
Goods

Population

peasants

Popularity

0–100

Food in the Granary

units

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Episode 1 · 33 min 20 s of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

4,409net worth

Gold: 537
Stored goods, at sell price: 3,872
Over the baseline (do nothing): −783
Population / housing: 50 / 98
Popularity: 78
Food: 25
Structures (map-wide): 68
Attempt: 1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock3,872 gold in goods

Good	Units	Price	Value
Gold	537	1	537
Iron	74	23	1,702
Crossbows	42	30	1,260
Swords	11	30	330
Ale	28	10	280
Bows	10	15	150
Meat	25	4	100
Wood	50	1	50
Net worth			4,409

Beside a person’s gameLarge Barren Desert, the same game time

	This episode	The person	Left alone
Net worth	4,409	9,984	5,192
Gold	537	8,574
Stored goods, at the sell price	3,872	1,410
Population	50	250
Housing	98	250
Popularity	78	100

The person played Large Barren Desert for 33 min 33 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game: July 1178
Idle peasants: 24
Popularity next month: ±0
Taxes: −6
Religion: +2
Ale from the inn: +4
Taxes: High taxes, 50 gold a month
Rations: full; 1 of 1 foods eaten
Worker efficiency: 100%
Storage: 16 stockpile piles, 9 empty; 2 granaries holding 25 food

Buildings49

Fletchers' Workshop18
Hovel11
Armory5
Iron Mine4
Stockpile4
Granary2
Blacksmith's Workshop1
Chapel1
Inn1
Keep1
Marketplace1

Tool calls137

market_trade22
building_info20
find_sites19
set_workshop17
build_structure12
inspect_building9
status8
set_tax7
get_inventory5
expand_storage4
observe4
place_near4
wait_and_observe4
map_overview1
update_plan1

Play

Turns: 60
Buildings placed / attempted: 33 / 33
Anchor-tool placements: 13
Tool errors: 1
Raw inputs sent to the game: 629
Context compactions: 2
Game time: 33 min

Compute

Tokens: 2.5M
Served from cache: 95%
Output tokens: 55K
Cost, as billed: $0.07
Wall-clock time: 45 min
of which inference: 18 min
Peak game memory: 3,518 MiB

The plan, before play

I’ll grow net worth through production and stored goods, while keeping population and popularity high enough to staff the economy. Since this map has no farmable oasis, I’ll prioritize housing and resource production on suitable terrain, then add workshops and storage as supplies and workers allow. I’ll manage taxes and rations carefully, check production and capacity as time passes, and avoid military spending and defenses.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

Model: GPT-6 Luna (openai/gpt-6-luna) on OpenRouter, providers openai then azure, reasoning medium, output limit 32,768 tokens
Benchmark: Large Barren Desert construction, version 1.6.3, from the save the human reference game started from; 33 min 20 s of game time (60,000 ticks) at speed 40, the length of the person's game; no playbook
Run 04b070e6, series 20261014T091943-e0b00398, 2026-10-14: Luna's second game on this map, a new series with no memory of the first, played on the latest version (the first game, series 20261012T123257-cf634001, played 1.5.6). The map's score for Luna is the mean of the two.

Result

	This game	First game	The person	Left alone
Net worth	4,409	7,354	9,984	5,192
Gold	537	1,246	8,574
Stored goods, at the sell price	3,872	6,108	1,410
Population / housing	50 / 98	106 / 130	250 / 250
Popularity	78	57	100

A full-budget score: the run used all 2,000 game seconds. Net worth ended 783 below leaving the game alone, against 2,162 above it in the first game. The stored goods are 74 iron (1,702), 42 crossbows (1,260), 11 swords (330), 28 ale (280), 10 bows (150), 25 meat (100) and 50 wood (50).

How the game went

A weapons town, 0 to 17 minutes. Three hovels, a chapel and a marketplace in the first 23 seconds, four iron mines by 3:31, three stockpile piles, then an armory, an inn and a fletcher's workshop at 1:56 and a blacksmith's workshop at 3:31. From 6:15 to 16:28 it added fletcher's workshops one to four at a time until there were eighteen, set them to crossbows (17 set_workshop calls), and bought planks to feed them. Eleven hovels by 12:59, a granary and a second armory, and taxes high, extortionate or mean for most of the game. It built no quarry and sold the 50 stone it started with.

Waiting on the workshops, 17 to 33 minutes. No more workshops after 16:28; three more armories at 22:52, when two held 31 of their 100 places. Gold fell to 232 at 15 minutes and stayed under 1,000, held up by taxes and sales of iron and pitch; meat and planks were bought twice more. The population stopped at 50 from 18 minutes with 24 idle and 48 places free, while popularity held between 76 and 98 on the inn (+4) and the chapel (+2). Of 137 tool calls, 42 read building information, inventory or status and 4 were waits. The workshops delivered 63 weapons in all.

Game minute	0	3	6	9	12	15	18	21	24	27	30	End
Net worth	4,711	5,569	4,255	3,393	2,825	1,813	1,956	2,258	2,031	2,315	2,089	4,409
Gold	4,440	4,602	2,499	1,172	907	232	470	820	380	740	137	537
Iron	0	15	18	26	33	31	38	46	53	60	67	74
Stone	0	50	50	50	0	0	0	0	0	0	0	0
Food	48	39	216	285	250	202	149	95	92	38	85	25
Population	2	28	37	40	44	46	50	50	50	50	50	50
Popularity	100	92	98	94	87	85	85	96	98	84	76	78

Up to 30 minutes the net worth row counts gold, wood, stone, iron and food (at 4) as the recording logs them; weapons and ale are left out, so it reads low.

Beside its first game

	This game (1.6.3)	First game (1.5.6)
Net worth	4,409	7,354
Buildings	18 fletcher's workshops, 5 armories, 4 iron mines, 11 hovels, blacksmith, inn, chapel	12 quarries, 39 tethers, 4 iron mines, 15 hovels, chapel, church, inn
Population / popularity	50 / 78	106 / 57
Net worth lost buying	3,790	3,675
Cost	$0.07	$0.13

The first game put its gold into quarries and tethers and ended with about 450 stone; this one put it into fletcher's workshops and planks. Both kept their iron.

What held it back

Workshops that could not pay for themselves. Eighteen fletcher's workshops cost 1,800 gold and 360 wood to build, and 550 planks were bought at 4 to feed them (worth 1 each). Their 52 crossbows and bows are worth 1,410.
No quarry. Stone was the first game's main product; this game had none.
Food bought at a loss: 450 meat and 35 other food for 3,880 gold, worth 1,940.
24 idle peasants. The population stopped at 50 with 48 places free.

Market

450 meat, 15 cheese, 15 apples, 5 bread, 550 planks and 20 ale bought for 6,480 gold; 50 stone, 20 pitch and 10 iron sold for 780. The purchases lost 3,790 net worth against the sell price (food 8 for 4, planks 4 for 1, ale 20 for 10); selling leaves net worth unchanged. These are units at the market's fixed prices.

Cost and speed

60 turns, 137 tool calls (4 waits), 2.55 million tokens and $0.067 billed by OpenRouter (preparation $0.003, play $0.061, compaction $0.004). The interview cost about $0.001 more, with 42,804 of its 44,873 tokens read from the cache. The agent run took 45.3 real minutes, 17.6 of them waiting for the model; the context was compacted twice (61 seconds). All 60 requests read the prompt cache, 95.7% of the input tokens.

The interview

Luna said it underperformed, ending below leaving the game alone: it produced valuable goods but not enough to repay what it spent building that production, and its population stopped at 50. Playing again it would grow housing, jobs and population much earlier, try a bread chain fed by bought wheat, and add inns, religion and the fear factor; it said its weapon workshops' output was too small to justify the investment at that scale. It found the inventory, population, popularity and storage readings useful, and named its own strategy as what got in the way. Its biggest mistake, it said, was treating "no farms" as a reason for a small weapons economy.

For the harness

A stale map probe on the game machine. Like every game of this batch on Oasis by the Sea and Large Barren Desert, this game ran a crusader_probe built from 1.5.0's sources. It was built on 2026-10-13 at 21:28 UTC as a 1.5.0 game started. The 1.6.3 sync then copied sources that matched by hash, and Ninja found nothing to rebuild. The probe's only change since 1.5.0 is 1.5.4's check that tells trees from cacti, so map_overview counted cacti as tree groves. find_sites counts trees from the reader's tile data and was right. Everything else this game played was 1.6.3, and the game stands as played. Luna called map_overview once and built no woodcutter.
The first armory found no free spot by the stockpile at 0:34; it went up by tile at 1:56.
The game saved correctly at the end ("gpt-6 luna 1-6-3 04b070").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

	You	The person
Game time played	33 min 20 s	33 min 33 s
Net worth	4,409	9,984
Gold	537	8,574
Stored goods, at the sell price	3,872	1,410
Population	50	250
Housing	98	250
Popularity	78	100

Leaving the game alone for the same time ends with a net worth of 5,192.

How do you see your result next to the person's? What do you think made the difference?

GPT-6 Luna

I underperformed: my net worth was 4,409, below both the person’s 9,984 and the 5,192 from leaving the game alone. I did produce and store valuable goods, but not enough to repay what I spent growing that production. I also ended with only 50 people and 98 housing, compared with 250 people and 250 housing in the person’s settlement.

2Playing again

Interviewer

How the person played, as they describe it:

Bread was one of their main sources of income: mills ground wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
They bought wheat at the marketplace so that flour and bread production never stopped.
A church and a cathedral raised popularity, and so did inns.
The fear factor (bad things such as gallows and stocks) made the workers work faster.

Their settlement at the end, read from the game:

250 people in 30 hovels, with no free housing; popularity 100.
95 bakeries and 12 mills, and no wheat farms.
11 inns, which reached everyone with ale, and no brewery or hops farm.
13 quarries and 7 iron mines with 37 ox tethers; no woodcutters.
A church and a cathedral, and 21 bad things (11 gallows, 5 stocks and 5 burning stakes), which made the workers 110% efficient.
Low taxes and no rations. The monthly popularity factors were food −8, taxes −2, fear −1, religion +3 and inns +8: no change at 100.

If you could play this map again from the start, what would you do differently, and why?

GPT-6 Luna

I would prioritize housing, jobs, and population much earlier. I got stuck at 24 idle peasants and did not create an effective way to turn that idle capacity into a much larger workforce. I spent heavily on weapon workshops, but their output and tax contribution were too small to justify the investment at that scale.

The person’s approach also showed that the lack of farmable land did not mean food production was out of reach: marketplace wheat could supply mills and bakeries. I should have explored that chain, along with more inns, religion, and fear-factor buildings to support popularity and worker efficiency. Instead, I relied on taxes and a small weapons industry, and spent substantial gold on food and wood.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

GPT-6 Luna

The inventory, population, popularity, and storage readings were useful for checking what was actually happening, and the tools made building and trading much more reliable than trying to infer everything from screenshots alone. The main thing that got in my way was my own strategy: I did not identify the marketplace-fed bread chain or the importance of scaling housing and jobs quickly enough. A concise example strategy for this barren map—especially how to balance bread production, selling, population growth, and popularity—would have helped.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

GPT-6 Luna

My biggest mistake was treating “no farms” as a reason to focus on a small weapons economy, rather than looking for ways to build a large population-supported economy using purchased inputs. The final goods total shows that production happened, but my choices did not convert it into enough net worth, gold, or population.

Record

Attempts and Reproducibility

Every attempt1

Episode	Attempt	Outcome	Net worth	Counted
1	1	Full-budget score	4,409	Counted

What ran

Benchmark version: 1.6.3
Fingerprint: 96c4682877ca
Harness commit: 1d0c697
Prompt / tools SHA-256: 3bb4a16 / e41a140
Menu guide: Images 118089d
Save: Large Barren Desert benchmark start
Game time per episode: 33 min 20 s
Playbook between episodes: No
Minimum turn / default wait: 8 s / 5 s
Observation after acting: Text unless a screenshot is needed
Screenshot width: 1,280 px
Context budget: 120,000 tokens
Real-time limit: 360 min
Submitted by: hesamation

All files in the dataset 20261014T091943-e0b00398