One episode · The Sinkhole

DeepSeek V4.1 Flash 1017

Benchmark 1.7.1DeepSeekReasoning mediumPlayed by the maintainersFull-budget score
Model
deepseek/deepseek-v4.1-flash-1017
Served via
OpenRouter · fireworks, together
Max output
32,768 tokens
Played
19 October 2026
Score
20.9
Crusader Bench score on The Sinkhole: 0 is doing nothing, 100 the person’s game
Net worth
11,345
+9,150 over doing nothing, 23% of the person’s gain of 39,436
Population
28
5% of the person’s gain of 502 · popularity 19
Cost
$0.14
2.4M tokens
Time
45 game min
42 min of wall-clock time, 46 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

  • DeepSeek V4.1 Flash 1017
  • Baseline (do nothing), final
  • A person's game, final

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Videos and images stream from the dataset. Game footage © Firefly Studios, shared for non-commercial research.

Episode 1 · 45 min of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

11,345net worth
Gold
6,037
Stored goods, at sell price
5,308
Over the baseline (do nothing)
+9,150
Population / housing
28 / 90
Popularity
19
Food
106
Structures (map-wide)
114
Attempt
1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock5,308 gold in goods

GoodUnitsPriceValue
Gold6,03716,037
Iron143233,289
Wheat1118888
Apples1024408
Wood2061206
Crossbows630180
Stone207140
Flour1110110
Hops7856
Cheese4416
Bows11515
Net worth11,345

Beside a person’s gameThe Sinkhole, the same game time

This episodeThe personLeft alone
Net worth11,34541,6312,195
Gold6,03739,977
Stored goods, at the sell price5,3081,654
Population28506
Housing90506
Popularity1999

The person played The Sinkhole for 44 min 58 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game
July 1189
Idle peasants
0
Popularity next month
−1
Food
+1
Taxes
−2
Religion
±0
Ale from the inn
±0
Taxes
Low taxes, 16 gold a month
Rations
full; 2 of 2 foods eaten
Worker efficiency
100%
Storage
16 stockpile piles, 0 empty; 1 granary holding 106 food

Buildings78

  • Iron Mine21
  • Hovel10
  • Wheat Farm10
  • Apple Orchard8
  • Woodcutter8
  • Dairy Farm5
  • Stockpile4
  • Fletchers' Workshop2
  • Armory1
  • Bakery1
  • Brewery1
  • Chapel1
  • Granary1
  • Hops Farm1
  • Inn1
  • Keep1
  • Marketplace1
  • Mill1

Tool calls141

  • find_sites55
  • building_info17
  • build_structure16
  • market_trade14
  • set_tax10
  • inspect_building7
  • demolish_building6
  • wait_and_observe5
  • set_workshop4
  • place_near3
  • map_overview2
  • expand_storage1
  • status1

Play

Turns
46
Buildings placed / attempted
71 / 76
Anchor-tool placements
7
Tool errors
1
Raw inputs sent to the game
698
Context compactions
2
Game time
45 min

Compute

Tokens
2.4M
Served from cache
91%
Output tokens
54K
Cost, as billed
$0.14
Wall-clock time
42 min
of which inference
8 min
Peak game memory
3,498 MiB

The plan, before play

I understand the task: play The Sinkhole free build for 45 game minutes maximizing net worth (gold + stored goods), with population and popularity as secondary. No soldiers, walls, towers or defences. Buildings must be placed accounting for terrain (oasis grass for farms, stone/iron ground for quarries/mines), one open side for workers, and reachable from the keep.

Opening plan:

  1. Read status/inventory and map overview to locate trees, oasis grass, stone and iron near the keep.
  2. Build 2–3 woodcutters beside the nearest grove to get wood flowing (wood is the main early bottleneck).
  3. Place hovels for housing (each +8), keeping employment growth in step with idle peasants so peasants keep arriving.
  4. Put food on the oasis ring: apple orchards / dairy farms feeding the granary; later wheat→mill→bakery for bread variety.
  5. Build a marketplace early so surplus can be sold and key materials bought to accelerate construction; use it to convert excess goods rather than let storage fill.
  6. Add iron mines on the ore patches plus blacksmith/fletcher workshops and an armory, since iron (23) and weapons are high-value stored worth; expand stockpiles/granary as space fills.
  7. Once popularity is comfortably above 50, raise taxes for steady gold; keep rations full/extra. Keep checking production panels to fix bottlenecks (no worker, no input, no storage) instead of overbuilding.
  8. Continue expanding until the clock ends, prioritizing whatever adds the most stored value per game minute.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (drafted by Sonnet 5.5, checked by Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

  • Model: DeepSeek V4.1 Flash 1017 (deepseek/deepseek-v4.1-flash-1017) on OpenRouter, served by Fireworks with Together as the only fallback (DeepSeek's own endpoint is excluded by the account's data policy), reasoning medium, output limit 32,768 tokens. Fireworks served 46 of the 49 recorded requests (1 preparation, 43 gameplay, 2 compaction); Together served 3 gameplay requests, at 34:31, 37:38 and 38:19.
  • Benchmark: The Sinkhole construction, version 1.7.1 (commit 07f2b7a), that benchmark's round, from the save "The Sinkhole start"; 45 min of game time at speed 40, the length of the person's game; no playbook. Weapons and armour in the armory count at the rules' prices. The host ran on the game machine.
  • Run fb7aed77, series 20261019T190209-fc06072c, 2026-10-19: this model's weapons-counted game on this map, a fresh series with no memory of any other. Its four other finished games (all 1.7.3) are in the table below; its no-arms Sinkhole game is still playing and is left out. This game counts on the website's leaderboard.

Result

This game The person Left alone
Net worth 11,345 41,631 2,195
Gold 6,037 39,977
Stored goods, at the sell price 5,308 1,654
Population / housing 28 / 90 506 / 506 4
Popularity 19 (−1 next month) 99 0
Score 20.9 100 0

A full-budget score: the run used all 2,700 game seconds. Net worth grew by 9,150 over leaving the game alone, 23.2% of the person's 39,436. By the website's formula (100 × (0.8 × net-worth share + 0.1 × population share + 0.1 × popularity ÷ 100), shares from leaving the game alone (net worth 2,195, population 4) to the person (41,631, 506), net-worth share clipped to −1..2) the score is 20.94: 18.56 from net worth, 0.48 from 28 people, 1.90 from popularity 19. Of the ten listed 1.7.1 Sinkhole games it is ninth.

The stored goods are 143 iron (3,289), 111 wheat (888), 102 apples (408), 206 wood, 6 crossbows and 1 bow (195), 20 stone, 11 flour, 7 hops and 4 cheese: gold is 53% of net worth, iron 29%. The town ended with 28 people in 90 places, no idle peasant, full rations and 106 food.

How the game went

Times are game time; for a call, when it began (the recording's clock; the run clock in the observations reads a few seconds later).

0:01 to 20:25. One build_structure call (83 s) put up the marketplace, three woodcutters and three hovels; three iron mines followed at 3:55 and average taxes at 5:02. Idle peasants hit the cap of 24 at 3:53. Food read 2 for 41 people at 6:36 (100 meat, 800 gold), the first iron came at 7:08 and popularity fell from 100 to 58 by 11:01. Then eight wheat farms (8:43), low taxes (11:03), a 250-gold chapel with five mines (13:04), a hops farm, dairy farm, brewery and inn (14:22), no taxes at 48 (15:32), four fletcher's workshops and an armory (16:14). Gold fell to 290 at about 17:00; net worth counted in full passed its start of 2,415 again at about 20:40.

20:25 to 34:30. Average taxes again and a mill and bakery (20:43). Food read 10 for 73 people at 23:30 (popularity 41; 100 bread, 800 gold). A small bribe from 25:28 took popularity from 36 to 74 by 27:48. Four mines went up at 25:44, five at 31:25 and four at 33:28; population reached 90 of 90 places at 33:26, popularity 50, monthly change 0.

34:31 to 45:00. A 300 s wait began at 34:31; a popularity warning ended it after 185 s (50 to 37). It set low taxes (37:38), sold 60 wheat and 60 iron, demolished six buildings (below), sold wood, wheat and flour (39:52 to 40:34) and waited 150 s and 75 s. Population read 90 at 35:00, 67 at 40:00, 28 at the end.

Below 50, the prompt says, peasants leave. Popularity was under 50 for 73 s (14:46), 227 s (22:32) and 565 s (35:33 to the end). At 34:09 the factors were food +2 (three kinds eaten), inn +2 and tax −4: a sum of 0, nothing in hand. The inn read 0 in every reading from 37:37 and cheese was down to 9 of the 40 bought at 32:27, so food was +1 and the monthly change −3. 62 people left, about 7 a minute from 37:00 to 44:00; paydays fell from 72 gold to 18; the mines delivered 14 iron a minute from 35:00 to 41:30 and 2.9 after.

Set at Level (factor; gold a month) Popularity, start to end People, start to end Paydays (gold)
5:02 Average (−4; 30) 100 to 58 38 to 48 14 (436)
11:03 Low (−2; 28) 58 to 48 48 to 48 10 (297)
15:32 None (+1; 0) 48 to 66 48 to 67 1 (7)
20:25 Average (−4; 53) 66 to 35 67 to 68 9 (499)
24:19 Low (−2; 40) 35 to 36 68 to 61 2 (78)
25:28 Small bribe (+3; −36) 36 to 74 61 to 64 5 (−120)
27:36 Average (−4; 51) 74 to 69 64 to 76 5 (230)
29:41 High (−6; 76) 69 to 50 76 to 89 6 (486)
32:19 Average (−4; 71) 50 to 37 89 to 81 12 (849)
37:38 Low (−2; 49) 37 to 19 81 to 28 16 (540)

Spells run from each set_tax reply to the next; they sum to 3,302 gold (3,441 paid, 139 of bribe).

The six demolitions, 38:19 to 39:32. Its reasoning read 12 open jobs as unstaffed iron mines and proposed freeing workers: iron at "11.5 gold/min per worker" against 3.2 for a woodcutter. It wrote "demolition refunds?" once and did not follow it up. It demolished four fletcher's workshops and two woodcutters. The recording shows +50 gold and +10 wood at 38:36, 38:47, 38:58 (+15) and 39:12, half a workshop's cost; the woodcutters (3 wood each) left no gold and no wood step apart from deliveries. The refunds, 200 gold and about 40 wood, are 2.1% of net worth and 0.5 of a point (20.5 without them).

Whether the freed workers took mine jobs the readings cannot show: idle peasants stayed 0, and open jobs went from 12 (38:20) to 15 (39:53), where the 9 people lost alone would have made 21. Iron delivered was 13.3 a minute in the three minutes before (35:00 to 38:00) and 13.7 in the three after (38:30 to 41:30). The 43:37 handoff said the demolitions "did not prevent the worker loss" and the interview called them "pure loss"; neither names the refunds. The benchmark rules do not mention refunds; the system prompt does only in the production readout's definition.

Game minute 0 5 10 15 20 25 30 35 40 End
Net worth 2,415 1,899 1,938 2,062 2,284 3,144 4,227 6,248 9,372 11,345
Gold 2,000 1,600 1,128 1,054 865 604 1,032 2,109 4,808 6,037
Iron 0 0 8 19 14 37 65 102 109 143
Food 55 22 97 78 49 95 82 83 70 106
Population 7 37 44 52 64 65 77 90 67 28
Popularity 100 100 65 49 66 35 68 50 30 19
Iron mines standing 0 3 3 8 8 8 12 21 21 21

Each column is the last reading at or before that minute. Net worth adds wheat, flour, hops, ale and weapons from the last observation before the minute (at most 80 s earlier). Its lowest reading was 1,680 at 17:17, with gold at 290.

Weapons

At 14:22 it priced 15 fletchers at 225 value a minute ("That's the big play") and built four, with an armory (16:14); two more followed at 28:25. set_workshop moved the first four to crossbows (17:18 to 18:02); the later two stayed on bows. They made 7 pieces, 6 crossbows and 1 bow, worth 195 at the rules' prices; none was sold. The four showed 0 a minute in all ten readings from 17:19 to 25:45 (an inspection at 19:53 found one working with its worker, nothing missing), and the handoff of 20:40 called the output an "UNRESOLVED problem". The first crossbow came between 25:45 and 27:37; a reading of 1.3 a minute at 28:26 ("39 gold/min! better than I thought") brought the two more, and readings were 0 to 1 after: 7 pieces in about 117 fletcher-minutes.

By the table in prompt/game-mechanics.md the six workshops cost 600 gold and 120 wood, the armory 5 wood, and the pieces took 20 wood; with the refunds, 505 at store prices for 195 made, about 310 behind (0.6 of a point). The 21 iron mines cost 420 wood and mined 228 iron, 5,244 at 23: 12.5 gold per wood, against 0.26 for the wing before refunds. The 195 held are 1.7% of net worth and 0.4 of the 20.9 points; with weapons worth nothing the same town scores 20.5.

Beside its other games and the other models

Oasis, weapons Oasis, no arms Desert, weapons Desert, no arms Sinkhole, weapons (this game)
Version / game time 1.7.3 / 33:20 1.7.3 / 33:20 1.7.3 / 33:20 1.7.3 / 33:20 1.7.1 / 45:00
Net worth 5,598 6,327 4,874 9,384 11,345
Score 26.5 28.0 0.7 78.5 20.9
Population / popularity 74 / 38 98 / 9 58 / 38 90 / 50 28 / 19
Iron mines / quarries placed 11 / 4 5 / 4 4 / 3 4 / 10 21 / 0
Fletcher's workshops placed 3 (+2 poleturner's) 7 14 0 (1 blacksmith) 6
Turns / cost 43 / $0.134 54 / $0.145 29 / $0.099 36 / $0.142 46 / $0.139

Scores use each map's own scale. Popularity ended under 40 in four of the five games.

1.7.1 Sinkhole, weapons counted Net worth Score Weapons held Iron mines / iron in store Population / popularity
Gemini 4 Argon 23,189 52.4 786 19 / 139 215 / 56
GPT-6.1 Sol 20,384 50.4 885 5 / 39 181 / 100
Claude Opus 5.5 15,424 34.7 1,530 27 / 432 164 / 47
Claude Sonnet 5.5 12,629 33.1 1,205 17 / 315 100 / 100
Muse Wonder 14,473 33.0 1,170 17 / 222 125 / 57
GPT-6 Luna 11,136 26.7 690 24 / 257 105 / 66
Claude Haiku 5.5 12,104 26.3 370 18 / 325 74 / 48
Claude Fable 5.1 13,107 23.1 450 25 / 336 50 / 0
DeepSeek V4.1 Flash 1017 11,345 20.9 195 21 / 143 28 / 19
GLM 5.3 Flash 5,680 15.1 1,020 3 / 47 79 / 65

Ordered by score; mines are those standing at the end. This game is ninth by score and eighth by net worth. It held the fewest people (Fable's 50 next) and weapons (Haiku's 370 next); its popularity is above only Fable's 0, and its 6,037 gold is third. GPT-6 Luna ended 209 lower in net worth and 5.8 higher in score: 105 people at popularity 66 are worth 8.6 points against 2.4 here.

What held it back

  • No margin, then an unattended wait. Popularity stood at 50 on a sum of 0 at 34:09 and a 300 s wait followed (above). The step to level 6 (29:41) had cost 17 points of popularity for 13 gold more a payday (72 at 77 people against 59 at 75).
  • Two food falls. 2 food for 41 people (6:36) and 10 for 73 (23:30) were bought out with 1,600 gold of meat and bread (800 of net worth); the second came with a fall of 25 points of popularity in the 3 minutes to it and 12 people lost after (73 to 61).
  • The idle cap early. With 22 to 24 idle from 3:53 to 12:36, the prompt's cap, population stood at 30 to 49 against 58 to 90 places; net worth stayed under its start until about 20:40.
  • Iron late. Mines placed: 3 at 3:55, 8 by 13:04, 12 by 25:44, 17 by 31:25, 21 by 33:28; iron delivered per five minutes was 0, 8, 11, 20, 23, 28, 37, 67, 34. Its "11.5 gold/min per worker" (28:25 to 38:19) was a per-mine figure; per worker the readings from 17:19 (0.3 to 0.7 iron a minute, two workers) are 3.5 to 8, against 6 to 14 for a wheat farm (0.8 to 1.7 wheat at 8). It stopped at 10 wheat farms.
  • Chapel and weapons. 250 gold for a chapel with religion 0 in every reading from 14:20; the weapons wing above.

What worked

  • Iron. 21 mines for 420 wood mined 228 iron (5,244): 3,289 held, 1,955 sold.
  • Taxes and the bribe. 3,302 gold net of the bribe (80 paydays). The bribe cost 139 gold in four paydays (25:28 to 27:48), took popularity from 36 to 74, stopped the loss at 61 people and let population grow to 90.
  • Placement. All 78 buildings placed (71 by build_structure, 7 hovels by place_near) are in the final count after the 6 demolitions (72); eight were not placed, each with a remedy.

Market

14 calls, 402 game seconds (14.9% of the budget). Purchases (units; gold): planks 225 (900), meat 100 (800), bread 100 (800), cheese 40 (320); 2,820 gold, 1,635 of net worth lost against the sell price. Sales: iron 85 (1,955), wheat 240 (1,920), wood 130 (130), flour 30 (300); 4,305, leaving net worth unchanged. Gold reconciles: 2,000 start + 3,441 paydays − 139 bribe + 4,305 sales + 200 refunds − 2,820 purchases − 950 buildings (chapel 250, inn 100, six fletchers 600) = 6,037.

The recording agrees with the replies except in two places. The wood sale (39:52) replied 100 units and 100 gold in 5 clicks; the first click sold 50 (220 in stock), so 130 went (stock and gold were right). The cheese purchase (32:27) replied partly_traded, 7 clicks, 35 units, −280, "a purchase did not show in the reader within 5 seconds"; one click fell in a payday frame (+31 and food +4: a payday of about 71 less a click of 40), so there were 8 (40 units, −320). It did not order the rest. The other sales match the recording's gold steps (iron 5 × 115 and 3 × 460; wheat 3 × 160 three times and 160 + 4 × 80; flour 6 × 50) and so do the other five purchases.

Cost and speed

46 turns, 141 tool calls (55 find_sites, 16 build_structure, 14 market_trade), 2.38 million tokens and $0.139 billed by OpenRouter (preparation $0.006, play $0.128, compaction $0.005); the interview cost $0.003 more. The run took 41.8 real minutes, 7.7 of them waiting for the model; five fast waits covered 558 game seconds in 221 real ones. 45 of 46 gameplay requests read the prompt cache, 91.6% of input; the exception is the 34:31 request on Together, 64,515 tokens uncached at $0.0195 against about $0.002 for its neighbours.

Of the 2,700 game seconds, build_structure took 1,096 (40.6%), waits 572 and market_trade 402. That is 15.4 game seconds per building placed by build_structure (71 buildings), against 11.8 (Oasis, weapons), 10.2 (Oasis, no arms), 11.9 (desert, weapons) and 11.2 (desert, no arms) in the model's other games, all with the host on the game machine; the maps differ, so this says nothing general.

The interview

It put the gap down to population and popularity: 28 people on 21 mines meant "most stood empty" (44 open jobs and no idle peasant at the last reading, 43:41, with 39 people), against the person's inns, ale, religion, 62 hovels, a bread chain and water pots. Playing again it would fix popularity first, build hovels in bulk, make bread the main industry and keep taxes at none or 4. It asked for a peasants-leaving readout, a warning when popularity trends under 50 and a list of idle buildings. Against the game, the results, 10 hovels, one mill and one bakery, full rations and the 44 open jobs match; these do not:

  • "Treated popularity as something to defend with low taxes": taxes were average or high from 27:36 to 37:38; low only after.
  • "A popularity collapse around minute 15 is effectively unrecoverable in 30 minutes": it was 48 at 15:31 and 67 by 20:24; the fall that ended the game began at 35:33.
  • "Several times I had to infer a stall" from full storage: one reading had no empty pile (39:53).
  • "Status/get_inventory" among the most useful readings: status once, get_inventory never.
  • "Red no-entry signs are the only signal" of an unstaffed building: open_jobs is in every reading, and placing a mine at 25:44 returned "Not enough workers available".
  • A notebook kept "from minute 5" would have shown the collapse: notebook_write and update_plan were available and never called.

For the harness

  • Demolition refunds half the cost, and the rules do not say so. Four workshops returned 200 gold and 40 wood, 0.5 of a point; the demolish_building description names no refund. This model tested refunds in its desert weapons game (6.1 points there) and weighed them in its desert no-arms game. The rules or tool text could say what is intended.
  • market_trade counts. A sale from 240 in stock sold 130 and replied 100 (the 50-a-click case from 200 in stock); a purchase with a click in a payday frame was counted a click short and stopped, as in the desert weapons game (14:22).
  • Placement on 1.7.1. Five orchard placements came back not_placed (2:21, 5:11, 7:25 ×3) with retrySkipped: no_free_spot_nearby and the remedy to use find_sites; the three at 7:25 were sites listed 5 to 7 minutes earlier, and it searched other ground (8:43) instead of retrying them. The hovel call at 8:18 placed one of three. expand_storage (30:04) replied that no free ground touched the stockpile and to sell or demolish beside it; it did not retry, and one of the 11 later readings (39:53) showed no empty pile. 1.7.1's retry searched 12 tiles; 1.7.2 limits it and keeps room beside stockpiles. This game cannot say what that would have changed.
  • Provider and warning. The 34:31 request went to Together with nothing cached, then back to Fireworks at 39:52; no failure was logged. The 300 s wait ended at 185 s with popularity 37: the warnings cover a fall of 10, low food and low gold, and none covers sitting at 50.
  • The game saved at the end ("deepseek v4-1 flash 1-7-1 fb7aed", 3,713,174 bytes in results.json).

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

You The person
Game time played 45 min 44 min 58 s
Net worth 11,345 41,631
Gold 6,037 39,977
Stored goods, at the sell price 5,308 1,654
Population 28 506
Housing 90 506
Popularity 19 99

Leaving the game alone for the same time ends with a net worth of 2,195.

How do you see your result next to the person's? What do you think made the difference?

DeepSeek V4.1 Flash 1017

I ended at 11,345 net worth against the person's 41,631 — a factor of nearly four — and the gap is almost entirely population and popularity, not production efficiency. With 28 people and popularity 19 my whole economy was running on roughly a tenth of the workforce they had. My iron mines were the right instinct (iron at 23 each is the best stored value per unit), but 21 of them against 28 peasants meant most stood empty: I finished with 44 open jobs and 0 idle peasants, i.e. I had built a labour demand I never had the people to fill. They ended with 506 people and 16 idle, which means every building they placed was actually staffed.

What made the difference, concretely:

  • Popularity as the engine, not a side-effect. They used inns + bought ale (+6) and religion (+3) to keep popularity near 100, which let them run fear buildings at −3 and rations at half/none and still grow. I treated popularity as something to defend with low taxes and it still slid 73 → 19, and once it fell below 50 peasants left, which cost me workers, which cost me production — a spiral I never broke.
  • Housing early and massively. 62 hovels vs my 10. Housing is cheap and it is what converts popularity into labour. I kept housing ahead of nothing.
  • Selling the food chain instead of eating it. They monetised bread at the marketplace (161 bakeries!) and ran rations low because ale/religion/inns covered popularity. I kept full rations, fed the population out of granary stock, and my bread chain stayed a single mill + bakery.
  • Jobs for the population. Water pots (3 workers, 60 gold) let them employ hundreds. I never built one.
  • Taxes. I taxed at level 4 for ~49–76 gold/month; that popularity cost compounded. They took 39,977 gold from trade, not taxes.

2Playing again

Interviewer

How the person played, as they describe it:

  • Early on they built a few inns and kept buying ale for them, which raised popularity by about 8 points. They spent those points on cutting the food rations to none, so the food could be sold instead of eaten.
  • They planted wheat farms and apple orchards and built a bread chain: mills ground the wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
  • They built iron mines and quarries, and by the end a large number of iron mines. A few blacksmiths turned iron into swords, which they think made little difference.
  • Taxes stayed at none most of the time. Religion added popularity, which let them run the fear factor at 3 so that the workers worked faster.
  • The bread made the population very large. Water pots (60 gold each, three workers) gave jobs to more of it.

Their settlement at the end, read from the game:

  • 506 people in 62 hovels, with no free housing and 16 idle; popularity 99, rising 3.
  • 161 bakeries, 19 mills and 31 wheat farms; no apple orchards were left.
  • 16 inns, with ale in the stockpile and no brewery or hops farm.
  • 38 iron mines and 10 quarries with 29 ox tethers; 7 woodcutters.
  • 9 blacksmiths, an armory holding 1 sword, and 24 water pots.
  • A church and a cathedral, and 105 bad things (48 burning stakes, 21 stretching racks, 12 stocks, 9 gallows and others), which made the workers 130% efficient.
  • No taxes and half rations of bread only. The monthly popularity factors were food −4, taxes +1, fear −3, religion +3 and inns +6.
  • 39,977 gold and 1,654 in stored goods: a net worth of 41,631.

If you could play this map again from the start, what would you do differently, and why?

DeepSeek V4.1 Flash 1017

In order of importance:

  1. Lock popularity at a high level before expanding anything. Two or three inns with bought ale, a church and a cathedral early, and never let popularity drift under 60. Bought ale costs gold but the popularity it buys is what creates the workforce, and the workforce is the score.
  2. Build hovels in bulk, continuously, and check open_jobs vs idle_peasants every few minutes. Never place a producer that will sit unstaffed. If jobs > peasants and popularity is low, fix popularity, not production.
  3. Treat the bread chain as the main industry, not a garnish. Many wheat farms → mills → bakeries, sold at the marketplace (bread 4/unit adds up at volume, and the chain also feeds a big population cheaply). I stopped at one mill and one bakery — a rounding error.
  4. Keep taxes at none or 4 at the most, and use trade revenue for gold instead; use fear buildings for the 130% worker-efficiency bonus once popularity can absorb it, and add water pots as worker sinks.
  5. Iron mines yes, but scaled to the workforce, with blacksmiths converting iron to swords (30 each) only once iron exceeds what I can store, and armories expanded in step.
  6. Keep rations lower in mid-game where ale/religion/inns cover the popularity, and sell the surplus food rather than eat it — those are net-worth-neutral sales but they free storage for higher-value goods.

The single sentence version: I optimised the production side of the economy and neglected the demographic side, and in this game the demographic side is the economy.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

DeepSeek V4.1 Flash 1017

The tools were genuinely good: build_structure with map-tile targets, automatic retry around blocked ground, and find_sites saved enormous time compared with clicking blind, and status/get_inventory/production.per_building_per_minute were the most useful readings in the whole run — they let me see which buildings produced nothing. The per-building production window was the single best diagnostic.

What got in my way:

  • Popularity and worker dynamics are opaque. The factors list is there, but the causal chain (popularity < 50 → peasants leave → jobs unfilled → production collapses) has no projection or warning. A "peasants arriving/leaving per month" readout, or a warning when popularity is trending under 50, would change how a model plays enormously.
  • Red no-entry signs are the only signal that a building is unstaffed, and they're only visible in the screenshot. inspect_building helps but costs an action; a settlement-level list of "idle buildings and why" in status would be far cheaper.
  • Storage-full silences production with no message — several times I had to infer a stall from storage numbers alone.
  • The time budget is honest but punishing of early mistakes: the payback on an iron mine is many minutes, and a popularity collapse around minute 15 is effectively unrecoverable in 30 minutes. That's a legitimate design choice, but it means the first 5 minutes decide the run, and the rules under-emphasise population growth relative to building.
  • Minor: screenshot staleness limits and the rule that reading tools are free only for a while forced me to think in a particular order; fine once understood.

Suggestion: a small "recommended next check" hint when popularity is falling or jobs are unfilled — not advice on strategy, just surfacing state the game already knows.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

DeepSeek V4.1 Flash 1017

My run was reactive rather than planned: I discovered the population bottleneck after it had already bitten, then spent the second half demolishing woodcutters and fletchers to shuffle a shrinking pool of workers, which is pure loss. The compacted-context handoff I inherited described the state accurately but framed the remaining decisions as "protect net worth", when in fact the biggest lever (population growth) had already been lost long before. If I'd kept a running notebook of popularity and idle-peasant trends from minute 5, the collapse would have been visible early enough to fix with an inn and a bought ale shipment rather than discovered at 22 popularity. The comparison to the human game is genuinely instructive — 41,631 is not a faster economy than mine, it is a bigger one built on people.

Record

Attempts and Reproducibility

Every attempt1

EpisodeAttemptOutcomeNet worthCounted
11Full-budget score11,345Counted

What ran

Benchmark
The Sinkhole construction
Benchmark version
1.7.1
Fingerprint
6ee823bf5cb0
Harness commit
07f2b7a
Prompt / tools SHA-256
3b0fa8a / f75ebb8
Menu guide
Images 118089d
Save
The Sinkhole start
Game time per episode
45 min
Playbook between episodes
No
Minimum turn / default wait
8 s / 5 s
Observation after acting
Text unless a screenshot is needed
Screenshot width
1,280 px
Context budget
120,000 tokens
Real-time limit
360 min
Submitted by
hesamation
All files in the dataset 20261019T190209-fc06072c