One episode · The Sinkhole (no arms)

Claude Opus 5.5

Benchmark 1.7.3No arms: weapons and armour count nothingAnthropicReasoning mediumPlayed by the maintainersFull-budget score
Model
claude-opus-5-5
Served via
Anthropic API
Max output
32,768 tokens
Played
19 October 2026
Score
39.3
Crusader Bench score on The Sinkhole (no arms): 0 is doing nothing, 100 the person’s game
Net worth
16,889
+14,694 over doing nothing, 37% of the person’s gain of 39,436
Population
146
28% of the person’s gain of 502 · popularity 67
Cost
~$1.82
3.4M tokens; ~ at list prices, the API reports no bill
Time
45 game min
40 min of wall-clock time, 61 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

  • Claude Opus 5.5
  • Baseline (do nothing), final
  • A person's game, final

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Videos and images stream from the dataset. Game footage © Firefly Studios, shared for non-commercial research.

Episode 1 · 45 min of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

16,889net worth
Gold
9,027
Stored goods, at sell price
7,862
Over the baseline (do nothing)
+14,694
Population / housing
146 / 202
Popularity
67
Food
168
Structures (map-wide)
164
Attempt
1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock7,862 gold in goods

GoodUnitsPriceValue
Gold9,02719,027
Iron237235,451
Wheat968768
Apples1694676
Stone967672
Wood1751175
Hops158120
Bows100
Net worth16,889

No arms: weapons and armour count 0 here, so the table adds up to the net worth.

Beside a person’s gameThe Sinkhole, the same game time

This episodeThe personLeft alone
Net worth16,88941,6312,195
Gold9,02739,977
Stored goods, at the sell price7,8621,654
Population146506
Housing202506
Popularity6799

The person played The Sinkhole for 44 min 58 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game
July 1189
Idle peasants
24
Popularity next month
+1
Taxes
+1
Religion
±0
Taxes
No taxes, 0 gold a month
Rations
full; 1 of 1 foods eaten
Worker efficiency
100%
Storage
16 stockpile piles, 0 empty; 2 granaries holding 169 food

Buildings124

  • Hovel24
  • Iron Mine23
  • Apple Orchard19
  • Woodcutter17
  • Wheat Farm8
  • Dairy Farm6
  • Fletchers' Workshop5
  • Hops Farm4
  • Ox Tether4
  • Quarry4
  • Stockpile4
  • Granary2
  • Armory1
  • Chapel1
  • Keep1
  • Marketplace1

Tool calls107

  • find_sites34
  • build_structure27
  • market_trade15
  • wait_and_observe10
  • set_tax7
  • inspect_building4
  • building_info2
  • expand_storage2
  • map_overview2
  • set_rations2
  • get_inventory1
  • go_to_tile1

Play

Turns
61
Buildings placed / attempted
117 / 122
Anchor-tool placements
1
Tool errors
0
Raw inputs sent to the game
782
Context compactions
2
Game time
45 min

Compute

Tokens
3.4M
Served from cache
96%
Output tokens
23K
Cost, at list prices
~$1.82
Wall-clock time
40 min
of which inference
6 min
Peak game memory
3,546 MiB

The plan, before play

The task: In 45 game minutes, grow the economy of The Sinkhole Free Build save as far as I can. At the end, net worth (gold plus stored goods at their sell price) counts most, then population, then popularity. Soldiers, walls and other defences are not allowed. The start has 2,000 gold, 55 wood, 20 stone and 55 meat, with a granary and four stockpiles by the keep.

Opening plan:

  1. Run map_overview and find_sites first, since they cost no game time.
  2. Put 2–3 woodcutters by the nearest trees, then 2–3 hovels.
  3. Build apple orchards and dairy farms on the oasis grass near the keep and granary. That gives food variety, a food surplus and more popularity.
  4. Build a marketplace early. If woodcutting is too slow, buy some wood to speed up construction.
  5. Expand toward the stone and iron:
    • quarries with ox tethers on the stone deposit about 85 tiles away
    • iron mines on the ore about 50–90 tiles east
    • Iron is worth 23 and stone 7, so stored stock adds directly to net worth.
  6. Keep adding hovels and jobs so the population keeps growing and idle peasants never reach 24.
  7. Once popularity is comfortably above 60:
    • add a chapel, then hops, a brewery and an inn for ale
    • raise taxes to low or average for steady gold
  8. Watch storage and add stockpiles and granaries before they fill. Later, a fletcher or poleturner can turn wood into weapons to sell for gold.
  9. In the last minutes, stop building and avoid buying. Then sell any weapons, because they count nothing while stored in the armory.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (drafted by Sonnet 5.5, checked by Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score. The checker is the same model as the player.

  • Model: Claude Opus 5.5 (claude-opus-5-5) on Anthropic's API, reasoning medium, output limit 32,768 tokens
  • Benchmark: The Sinkhole construction (no arms), version 1.7.3 (commit e0e0eb4), from the save the human reference game on this map started from; 45 min of game time at speed 40, the length of the person's game (44 min 58 s); no playbook; waits played fast. Weapons and armour in store count nothing toward net worth; sold at a marketplace, they bring their gold.
  • Run fc3d2140, series 20261019T082447-7e09b8c5, 2026-10-19, host on the game machine: Opus's no-arms game on this map, a fresh series with no memory of its others. Its weapons-counted pair is its 1.7.1 game (run 9c8871df, series 20261015T185216-6a2a0659, commit 674e7ba, host on a Mac); its Sinkhole games from earlier rounds are runs 256f1790 and 0a79c93b (1.6.2). This game counts on the website's leaderboard.

Result

This game (1.7.3, no arms) 1.7.1 game (weapons counted) 1.6.2, first game 1.6.2, second game The person Left alone
Net worth 16,889 15,424 13,921 16,193 41,631 2,195
Gold 9,027 2,186 1,157 6,261 39,977
Stored goods, at the sell price 7,862 13,238 12,764 9,932 1,654
of which weapons and armour 0 1,530 3,714 2,560
Population / housing 146 / 202 164 / 202 147 / 162 144 / 162 506 / 506 4
Popularity 67 (+1 next month) 47 (−2) 48 (−2) 62 (+1) 99 0
Score 39.3 34.7 31.4 37.4 100 0

A full-budget score: the run used all 2,700 game seconds. Net worth grew by 14,694 over leaving the game alone, 37.3% of the person's 39,436, against 33.5% in the 1.7.1 game. By the website's formula (80% net worth, 10% each population and popularity; shares from doing nothing to the person; net-worth share clipped to −1..2) the score is 39.3: 29.81 points from net worth, 2.83 from 146 people and 6.70 from popularity 67. It is the highest of Opus's four Sinkhole games. Gold is 53% of net worth and iron 32%: it mined 427 iron and sold 190 of it at 39:08 for 4,370 gold; the 237 kept are 5,451. The town ended with 24 idle peasants, 56 empty places in its hovels, no empty pile in the stockpile and taxes off.

How the game went

Times are the game clock when a call began, read from the recording; readings quoted from observations carry the run clock, which reads a few seconds later.

0 to 10 minutes: iron, then taxes and half rations. Three woodcutters (0:01), orchards and a dairy (0:45), four hovels (1:26), the marketplace (2:45), 100 planks (3:01); with 23 idle peasants it placed four iron mines at 3:53 (the call ran 78 s). Average taxes (6:49) and half rations (7:02), with the starting meat nearly gone (food 20 at 7:01), took popularity from 98 to 60 by 10:07; low taxes (8:39), a chapel for 250 gold (9:02), full rations (10:06) and no taxes (10:13) followed. First iron reached the stockpile at 6:36.

10 to 20 minutes: quarries, bows, iron in fives. A quarry and tether on the deposit 84 tiles away (11:13), a second quarry (12:24), and at 15:08 an armory and three fletcher's workshops. No stone came: it inspected a quarry (14:34) and its tether (14:54) and moved the camera (15:52); the first compaction was at 16:00, with stone still 20. Planks (16:03), five mines (16:49, 14 in all), the first stone (17:17), two more quarries (18:03), low taxes (19:17); population passed 100 at 20:24.

20 to 35 minutes: more jobs, then wheat. Dairies and orchards (20:54), hovels (21:55), the first bows sold (22:27), planks (23:08), two mines, four woodcutters and two workshops (23:58), five woodcutters (25:27), a second granary (27:18), no taxes (27:44). Five mines (28:01, 23 in all) and two wheat farms, then six more wheat farms and four hops farms with hovels, to 34:17; housing rose from 146 to 202. At 32:26 expand_storage found no free ground beside the stockpile, and at the 34:28 reading no pile was empty. The second compaction was at 34:27.

35 to 45 minutes: selling and waiting. The last build call was at 34:48 (two orchards). From 34:30 it made seven sales for room (wood, iron, stone), sold bows, set no taxes at 39:02 and otherwise waited 20 to 100 s at a time. Population stood at 146 from 35:31.

Game minute 0 5 10 15 20 25 30 35 40 End
Net worth 2,415 1,976 2,360 3,167 3,526 5,647 7,761 10,425 13,140 16,889
Gold 2,000 1,600 1,588 1,621 948 1,358 1,615 2,236 7,852 9,027
Iron 0 0 15 49 86 141 203 281 153 237
Stone 20 20 20 20 36 76 124 164 152 96
Food 55 38 56 49 80 88 125 129 145 168
Population / housing 6 / 10 38 / 42 56 / 58 75 / 82 97 / 114 112 / 146 134 / 146 144 / 202 146 / 202 146 / 202
Popularity 100 100 61 77 88 70 67 65 52 67
Iron mines placed 0 4 7 9 16 18 23 23 23 23

Each column is the last recording frame at or before that minute. Before End the net worth row counts gold, wood, stone, iron and food (at 4) and leaves out wheat and hops (under 100 at 35 minutes, about 450 at 40), so it reads low; End is the game's own figure. Mines are counted from the start of the call that placed them.

Weapons

It built them anyway, small, and sold what they made. Its opening reply planned for "a fletcher or poleturner" later to turn wood into weapons to sell, and to sell any in the last minutes because they count nothing in the armory. At 15:07, with 24 idle workers and wood to spare, its reasoning called fletchers "a great use"; at 15:08 it placed an armory (5 wood) and three fletcher's workshops (20 wood and 100 gold each). The 16:00 handoff priced a bow at 2 wood, worth 1 each, and a sale at 15, and said to sell bows often. Two more workshops followed at 23:58 (no reasoning was logged for that turn). It never called set_workshop, so all five made bows, 1.0 to 1.7 a minute together in the readout (0.2 to 0.3 each, 3 to 5 gold a minute), against 0.5 to 0.8 iron, 12 to 18 gold, for a mine. It called them "underperforming" (25:27), passed over fletchers as too slow at 32:54 and built no more.

They made 36 bows. It sold 35 for 525 in five calls of lots of five (22:27: 5, 30:17: 5, 40:54: 15, 42:55: 5, 44:44: 5), each reply matching the recording's gold; one bow was left, worth nothing. The wing cost 500 gold, 105 wood in buildings, 72 in bows (2 each, by prompt/game-mechanics.md) and five jobs, filled while idle peasants read 17 to 24 and open_jobs 0 in every reading, so no one was taken from other work. Gold ends +25; with the 177 wood at its sell value of 1 the wing returned 525 against 677, −152 of net worth (0.3 points); with the planks at the 4 gold it paid, about −683.

The 1.7.1 game built ten fletcher's workshops (armory 4:31, workshops from 5:01, the last four at 30:15) and two armories, set them to crossbows (eight set_workshop calls) and sold nothing. It ended with 43 crossbows (1,290) and 16 bows (240): 1,530, 9.9% of its 15,424. They cost 1,000 gold and 371 wood (workshops 200, armories 10, weapons 161), so under that rule the wing returned 1,530 against 1,371 (+159); under this game's rule the stock counts nothing and that game ends at 13,894 and 31.6 points. Across Opus's Sinkhole games the weapons shrank: 148 pieces worth 3,714 (27% of net worth) and 221 worth 2,560 (16%) in the 1.6.2 games, 59 worth 1,530 in 1.7.1, 36 bows here.

Iron went into no weapon in either game (bows and crossbows take wood only): it mined 427 here and 432 there. Wood went in at 177 against 371 and workers at 5 against 10; the 500 gold not spent on workshops stayed in the treasury (gold never fell below 921 after the opening). The logs show the rule changing the plan only in when and how much: the wing started at 15:08 here against 4:31 there, with five workshops against ten.

Beside its other games and the other models

This game 1.7.1 game Difference
Gold 9,027 2,186 +6,841
Iron 5,451 (237) 9,936 (432) −4,485
Stone 672 (96) 1,344 (192) −672
Weapons and armour 0 1,530 (59) −1,530
Wheat and hops 888 (111) 0 +888
Apples 676 (169) 284 (71) +392
Wood 175 144 +31
Net worth 16,889 15,424 +1,465
Gold, from 2,000 This game 1.7.1 game
Taxes (the recording's rises less sales) +2,782 +2,916
Sales +6,195 +460
Planks and apples bought −1,200 −1,940
Fletcher's workshops −500 −1,000
Chapel −250 −250
Gold at the end 9,027 2,186

The pair differs in more than weapons: the version (1.7.2 retries a blocked build_structure target only within its footprint's side plus a tile, where 1.7.1 searched 12, and keeps room beside stockpiles; 1.7.3 added this benchmark), the host (a Mac there, the game machine here) and the model's own choices. Here three placements moved 5, 2 and 11 tiles, inside their reach, and five were not placed; both stockpiles ran out of room (35:41 there, 32:26 here).

The score is 4.6 points higher: net worth +2.97, popularity +2.00, population −0.36. Scored by this game's rule the 1.7.1 game ends at 13,894, so this game stands 2,995 and 7.7 points ahead, not 1,465 and 4.6. Iron mined is level; the differences are in what else it made: 12 wheat and hops farms placed from 28:01 (+888 for 180 wood), 216 stone mined against 172 (first quarry at 11:13 against 20:12, first stone at 17:17 against 25:13), 740 less gold spent at the market (net worth lost 900 against 1,355), 500 less in workshops and taxes 134 lower. Popularity ended at 67 against 47: it set no taxes at 39:02, where the 1.7.1 game moved to average taxes at 41:27 and low at 43:02.

In the 1.7.1 round (weapons counted) Gemini 4 Argon scored 52.4 and GPT-6.1 Sol 50.4, ahead of Opus's 34.7, Claude Sonnet 5.5's 33.1 and Muse Wonder's 33.0; this game's 39.3 would stand third of eleven. Those games held 370 to 1,530 of weapons (Opus's the most), 0.8 to 3.1 points, so leaving them out moves no one past it. Among the no-arms Sinkhole games so far it leads: Claude Haiku 5.5 8,243 / 23.8, Claude Sonnet 5.5 15,246 / 35.3, this game 16,889 / 39.3. On the batch's other maps its no-arms game also beat its weapons-counted game (Oasis by the Sea 37.7 against 32.9, Large Barren Desert 67.1 against 19.1; those pairs share a version).

What held it back

  • Jobs, not housing. Idle peasants read 17 to 24 with no open job from 5 minutes, and 24 in 15 of the 19 readings after 22:30; population stood at 146 from 35:31 with 56 empty places. Jobs did bring arrivals: the five mines of 28:01 took idle from 24 to 17 at 29:11 and back to 23 by 30:34, population 122 to 137. Nothing was built after 34:48, with 2,236 to 3,137 gold in hand; its logged reasoning at 36:43 and 37:44 was to watch and wait.
  • A full stockpile. At 39:03 iron and stone had room 0 and the readout said their producers stop; iron read 16.7 a minute at 37:46 and 12.0 at 40:34. It sold wood for room, and at 39:08 iron. At 44:45 every pile held something; wheat and stone had room 0, iron room for 56 and planks for 29.
  • No chain beyond raw goods. The eight wheat farms (28:01 to 32:56) fed no mill and the four hops farms no brewery; one food was eaten in 30 of 48 readings.
  • Taxes against popularity. Three spells above no taxes (6:49 to 10:13, 19:17 to 27:44, 32:19 to 39:02) brought about 2,780 gold, 5.6 points at the formula's weight, and popularity read 60, 60 and 49 at their ends. The chapel's religion factor read 0 in every reading from 10:07.
  • Planks bought and sold. 300 bought at 4 (1,200 gold, 900 of net worth lost) and 320 sold at 1; wood stood at 103 to 180 between 24 and 35 minutes with 17 woodcutters.
  • The weapons wing: about −152 of net worth.

What worked

  • Iron. Four mines at 3:53, 23 by 28:01, 100 iron at 21:16; the 427 mined were worth 9,821 at 23 each, 58% of the final net worth.
  • Selling to keep the mines running. The 39:08 sale cleared 190 iron of room; iron went from 146 to 237 by the end (91 in 5:39, 16 a minute, 2,093 of net worth, with room 0 at 39:03).
  • A late crop. Twelve farms in the last 17 minutes were 888 of net worth for 180 wood.
  • Food. After the first 10 minutes it never fell under 44 and ended at 168.
  • Placements held: all 117 build_structure placements reported as placed are in the final count of buildings; five tries did not go through and no call was refused.

Market

Bought: 300 planks, three lots of 100 (3:01, 16:03, 23:08), 1,200 gold; no food. Sold: 35 bows for 525, 320 planks for 320 (four calls), 190 iron for 4,370 and 140 stone for 980 (two calls), 6,195 gold in all, where the replies add to 4,125. Only the 39:08 iron call differs: it asked for 100 and reported 100 units and 2,300 gold, and the recording shows iron 336 to 146 and gold 3,202 to 7,572 (the game sells 50 a click with 200 or more in stock; the reply's stock and gold were right). Planks (162, 122, 83 and 143 in stock) and stone (192 and 184) were under 200 and match. Selling left net worth unchanged except the bows (+525). The 15 calls took 289 game seconds. None of its 107 tool calls was a game_action, so it set no automatic trade by clicks, and every fall of gold in the recording is accounted for (1,200 planks, 500 workshops, 250 chapel).

Cost and speed

61 turns, 107 tool calls (34 find_sites, 27 build_structure, 15 market_trade, 10 waits, 7 set_tax, 14 others), 3.41 million tokens and $1.818 at Anthropic's list prices ($4 per million input, $20 output, $0.20 cache read, $5 cache write): preparation $0.164, play $1.539, compaction $0.115. The interview cost about $0.042 more (51,063 tokens, 48,422 read from the cache). The agent run took 40.0 real minutes, 6.0 of them waiting for the model. The context was compacted twice (16:00 and 34:27, 32 seconds in all); each compaction was one request and succeeded on the first. All 61 requests read the prompt cache (96.8% of input tokens) and none failed.

Of the 2,700 game seconds the 27 build calls took 1,452, the 10 waits 535 (played fast, 231 real seconds), the 15 market calls 289 and the host's 5-second top-up after 34 replies that acted without a screenshot 170. The build calls placed 117 buildings: 12.4 game seconds each, 4.3 a call. The 1.7.1 game's 29 calls took 1,512 s for 129 (11.7 s each, 4.4 a call). That game's host was a Mac and this one's the game machine, and the version and the mix of buildings differ, so the 0.7 s says nothing by itself.

The interview

Opus put the gap to the person (about 40% of their net worth) down to population capped by 24 idle peasants with no open job, popularity spent on low taxes, no product that adds value (wheat late, no mill), a stockpile that choked production, and early gold spent on planks. Playing again it would build the wheat, flour and bread chain early, plan storage around the stockpile, stop at six to eight woodcutters, get religion working and add inns, add jobs whenever idle reaches 24, and sell surplus continuously. It praised the tile-based tools and the readout's per-pile room and per-building production, and asked for a note on popularity factors and a warning when idle peasants are capped with housing empty. It did not mention bows, the fletchers or the armory.

Against the game these match: 16,889 against 41,631, 14,700 above leaving the game alone; 146 against 506 people and 67 against 99; 24 idle with no open job; the chapel's factor at 0; wheat farms late with no mill; low taxes at about 84 gold a month (the 32:19 reply); the stockpile full and wood, iron and stone sold in the last ten minutes; 17 woodcutters. These do not: it spent 1,200 gold on planks, not 800 (the second handoff also said 200 planks and 800 gold, with three lots already bought); "56 houses" are 56 places in 24 hovels; "twenty minutes" of stall is 24, from 20:40. Not checkable: "seven [woodcutters] would have done", and that it built over the stockpile's room (the tool said only that no free ground touched it, at 32:26).

For the harness

  • The market_trade reply undercounted the 39:08 iron sale (100 units and 2,300 gold against 190 and 4,370): the known 1.7.3 fault at 200 or more in stock. stock and gold were right, the model did not use the wrong figures, and scoring is unaffected.
  • No room beside the stockpile (32:26), as in the 1.7.1 game (35:41), although 1.7.2 keeps room beside stockpiles; no placement here was moved for it. The remedy named selling or demolishing; it sold goods and did not ask again.
  • Placements: a mine off the ore was rejected (7:08), two hovels were not placed with nearestFit 8 and 9 tiles off against a reach of 5 (7:38), a dairy farm (10:17) and a quarry (12:24) were not placed; each result named what to do.
  • No harness warning for the idle cap, which the benchmark text states and the stats show; Opus asked for one.
  • No provider errors, stalls or garbage-collection watchdog events (no forced collection in 478 memory samples). The game saved at the end ("claude opus 5-5 1-7-3 fc3d21").

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

You The person
Game time played 45 min 44 min 58 s
Net worth 16,889 41,631
Gold 9,027 39,977
Stored goods, at the sell price (weapons and armour not counted) 7,862 1,654
Population 146 506
Housing 202 506
Popularity 67 99

Leaving the game alone for the same time ends with a net worth of 2,195.

How do you see your result next to the person's? What do you think made the difference?

Claude Opus 5.5

My net worth was 16,889 against the person's 41,631, about 40% of theirs and roughly 14,700 above leaving the game alone. They also ended with more than three times my population and 32 more popularity points. The gap comes mostly from one thing: they built an engine that turned popularity into output, and I mostly piled up raw goods. Here is what I think made the difference:

  • Population was capped and I never fixed it. For the whole second half I had 24 idle peasants and 0 open jobs, so no new peasants came, and 56 houses stood empty. The guide says idle peasants block arrivals. I noted it and still kept adding buildings that needed few workers. Cheap job-makers like water pots would have kept the population growing.
  • I used popularity for the wrong thing. I spent it on low taxes, which gave about 84 gold a month and drained popularity. The person spent theirs on cutting rations to none and on fear factor for 130% worker speed, and topped it up with inns and religion. My chapel's religion factor stayed at 0 and I never found out why.
  • I had no product that adds value. My apples and stone are worth little per unit, and iron was capped by my storage. The person's bread chain, run through 161 bakeries, created gold steadily. My wheat farms came late and never fed a mill.
  • My storage choked production. The stockpile was full and boxed in by other buildings, so in the last ten minutes I was selling wood, iron and stone just to keep producers running. I had built over the space the stockpile needed to grow.
  • I spent early gold poorly. I spent 800 gold buying wood and ended with 17 woodcutters, far more wood than I needed. Seven would have done.

2Playing again

Interviewer

How the person played, as they describe it:

  • Early on they built a few inns and kept buying ale for them, which raised popularity by about 8 points. They spent those points on cutting the food rations to none, so the food could be sold instead of eaten.
  • They planted wheat farms and apple orchards and built a bread chain: mills ground the wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
  • They built iron mines and quarries, and by the end a large number of iron mines. A few blacksmiths turned iron into swords, which they think made little difference.
  • Taxes stayed at none most of the time. Religion added popularity, which let them run the fear factor at 3 so that the workers worked faster.
  • The bread made the population very large. Water pots (60 gold each, three workers) gave jobs to more of it.

Their settlement at the end, read from the game:

  • 506 people in 62 hovels, with no free housing and 16 idle; popularity 99, rising 3.
  • 161 bakeries, 19 mills and 31 wheat farms; no apple orchards were left.
  • 16 inns, with ale in the stockpile and no brewery or hops farm.
  • 38 iron mines and 10 quarries with 29 ox tethers; 7 woodcutters.
  • 9 blacksmiths, an armory holding 1 sword, and 24 water pots.
  • A church and a cathedral, and 105 bad things (48 burning stakes, 21 stretching racks, 12 stocks, 9 gallows and others), which made the workers 130% efficient.
  • No taxes and half rations of bread only. The monthly popularity factors were food −4, taxes +1, fear −3, religion +3 and inns +6.
  • 39,977 gold and 1,654 in stored goods: a net worth of 41,631.

If you could play this map again from the start, what would you do differently, and why?

Claude Opus 5.5

  • Find the main value-adding chain early and scale it hard. Here that means wheat to flour to bread, sold at the market.
  • Plan storage space around the stockpile and granary before building anything near them.
  • Stop at about 6–8 woodcutters and not buy wood unless something urgent needs it.
  • Get religion working early, check that its factor is positive, and add inns (buying ale if needed). Then spend popularity on rations at none and on fear factor rather than on taxes.
  • Watch idle peasants and open jobs every few minutes, and add jobs whenever idle reaches 24. Housing should never sit empty because nobody can come.
  • Build many more iron mines early, close to the stockpile, and a stockpile annex near them. Iron at 23 is the best stored good per unit.
  • Sell surplus continuously, not only when storage forces me to.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

Claude Opus 5.5

What helped:

  • The tools were very good. Tile-based build_structure, find_sites and market_trade with net-worth deltas removed nearly all the clicking problems.
  • The stats JSON, especially the per-pile storage room and the per-building production rates, was the most useful signal I had.
  • Reading for free while the game is paused is fair and pushes you toward planning.

What got in my way:

  • Mostly my own strategy, not the interface.
  • A few things were opaque: the religion factor stayed at 0 with a chapel built, and the stockpile could not be expanded and I didn't diagnose why.

What I would change:

  • A short note on how popularity factors and worker efficiency (fear and good things) work would help.
  • So would a warning when idle peasants are capped at 24 and housing is empty, because that silently stalled my growth for twenty minutes.
  • Context compaction partway through cost some continuity, though the handoff note worked.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

Claude Opus 5.5

The comparison is humbling and instructive. I played a cautious "produce and store" game, but the score rewards turning population and popularity into a processing engine and converting its output to gold. Most of my last ten minutes went to managing full storage, which is a symptom of not planning the layout. It was a well-designed task, and I'd like another attempt with the bread-and-popularity approach.

Record

Attempts and Reproducibility

Every attempt1

EpisodeAttemptOutcomeNet worthCounted
11Full-budget score16,889Counted

What ran

Benchmark
The Sinkhole construction (no arms)
Benchmark version
1.7.3
Fingerprint
b7621690f14b
Harness commit
e0e0eb4
Prompt / tools SHA-256
7ed945e / f75ebb8
Menu guide
Images 118089d
Save
The Sinkhole start
Game time per episode
45 min
Playbook between episodes
No
Minimum turn / default wait
8 s / 5 s
Observation after acting
Text unless a screenshot is needed
Screenshot width
1,280 px
Context budget
120,000 tokens
Real-time limit
360 min
Submitted by
hesamation
All files in the dataset 20261019T082447-7e09b8c5