One episode · The Sinkhole

Kimi K3.1 Pro

Benchmark 1.7.1Moonshot AIReasoning highPlayed by the maintainersFull-budget score
Model
kimi-1021-preview
Served via
Moonshot API
Max output
32,768 tokens
Played
23 October 2026
Score
31.8
Crusader Bench score on The Sinkhole: 0 is doing nothing, 100 the person’s game
Net worth
13,873
+11,678 over doing nothing, 30% of the person’s gain of 39,436
Population
134
26% of the person’s gain of 502 · popularity 55
Cost
~$1.29
3.7M tokens; ~ at list prices, the API reports no bill
Time
45 game min
54 min of wall-clock time, 70 turns

Minute by Minute

Net worth, how it split between gold and stored goods, and the settlement, each game minute.

Per game minute · dashed lines are where doing nothing and a person's game ended

  • Kimi K3.1 Pro
  • Baseline (do nothing), final
  • A person's game, final

Crusader Bench · crusbench.com

Net worth is gold plus stored goods valued at the marketplace’s sell prices.

Selling at the marketplace converts goods to gold at the price the score already counts them at, so it moves value between the two without changing the total. Buying costs net worth.

Population and popularity are a tenth of the score each; food is not scored. Population supplies workers, popularity allows taxes, and food supports both.

The episode

Watch the Episode

Videos and images stream from the dataset. Game footage © Firefly Studios, shared for non-commercial research.

Episode 1 · 45 min of game time, edited: actions at real speed, waiting fast-forwardedStreamed from Hugging Face

ScorecardFull-budget score

13,873net worth
Gold
7,969
Stored goods, at sell price
5,904
Over the baseline (do nothing)
+11,678
Population / housing
134 / 146
Popularity
55
Food
36
Structures (map-wide)
149
Attempt
1

The settlement at the endcentred on the keep, zoomed out

The settlement at the end of episode 1, zoomed out over the keep

Final stock5,904 gold in goods

GoodUnitsPriceValue
Gold7,96917,969
Iron202234,646
Stone927644
Spears3310330
Apples344136
Bows715105
Wood35135
Cheese248
Net worth13,873

Beside a person’s gameThe Sinkhole, the same game time

This episodeThe personLeft alone
Net worth13,87341,6312,195
Gold7,96939,977
Stored goods, at the sell price5,9041,654
Population134506
Housing146506
Popularity5599

The person played The Sinkhole for 44 min 58 s of game time at the same speed, timed by their own clock, and was scored the same way. The model saw this comparison only in its interview.

The town’s booksread from the game at the end

Date in the game
July 1189
Idle peasants
24
Popularity next month
−1
Food
+5
Taxes
−6
Religion
±0
Taxes
High taxes, 134 gold a month
Rations
extra; 2 of 2 foods eaten
Worker efficiency
100%
Storage
16 stockpile piles, 8 empty; 1 granary holding 36 food

Buildings112

  • Apple Orchard25
  • Iron Mine21
  • Hovel17
  • Dairy Farm14
  • Poleturner's Workshop12
  • Woodcutter7
  • Stockpile4
  • Fletchers' Workshop3
  • Chapel2
  • Ox Tether2
  • Armory1
  • Granary1
  • Keep1
  • Marketplace1
  • Quarry1

Tool calls127

  • find_sites63
  • build_structure16
  • market_trade16
  • set_tax11
  • building_info9
  • inspect_building3
  • set_rations3
  • map_overview2
  • wait_and_observe2
  • expand_storage1
  • update_plan1

Play

Turns
70
Buildings placed / attempted
106 / 113
Anchor-tool placements
0
Tool errors
1
Raw inputs sent to the game
790
Context compactions
1
Game time
45 min

Compute

Tokens
3.7M
Served from cache
97%
Output tokens
34K
Cost, at list prices
~$1.29
Wall-clock time
54 min
of which inference
15 min
Peak game memory
3,562 MiB

The plan, before play

I understand the task: maximize final net worth within 45 game minutes, with population and popularity as secondary priorities. Only the final in-game state counts. I will not recruit soldiers or build walls, towers, gates, or other defenses.

Opening plan:

  1. Quickly establish wood production near the nearest grove and add housing.
  2. Build apple orchards and/or dairy farms on oasis grass to secure a growing food surplus.
  3. Add a marketplace, then expand into iron mining and stone quarrying with adequate ox transport.
  4. Keep adding housing and jobs as peasants arrive, while monitoring staffing, storage, food, and walking access.
  5. Use food variety and, later, religion/ale to support popularity and sustainable taxation.
  6. Reinvest early production into further growth, then preserve valuable stored goods and avoid wasteful late spending.

Commentary during play0

The model played this episode without writing commentary between its tool calls.

Reading the game

The Analysis

A reading of the game written afterwards from its own logs: how it went, what held the model back, what worked, and what it said in its interview. It is published with the game and is not part of its score.

Written by Claude (drafted by Sonnet 5.5, checked by Opus 5.5) from the run's own logs (events.jsonl, episode.json, results.json, interview.md); it is a reading of the run, not part of its score.

  • Model: Kimi K3.1 Pro, through Moonshot's pre-release endpoint for it, kimi-1021-preview (released as kimi-k3.1-pro), reasoning effort high, output limit 32,768 tokens; costs at Kimi K3.1 Pro's list prices ($2 per million input tokens, $10 output, $0.20 cached, $2 cache write), as Moonshot reports no cost per request.
  • Benchmark: The Sinkhole construction, version 1.7.1 (commit 07f2b7a), that benchmark's round since 2026-10-15, from the save "The Sinkhole start"; 45 min of game time at speed 40, the length of the person's game; no playbook. Weapons and armour in the armory count at the rules' prices. The host ran on the game machine. The runner checked out 07f2b7a for it, like the other models' listed games here; the model's other five games ran on 1.7.3 (commit 55bcc41). 1.7.1 has no retry for a reply cut off by the provider closing the connection; none was needed (70 requests completed, none failed).
  • Run 3d6c968e, series 20261023T184800-d00fe445, 2026-10-23: game 5 of this model's six games that day, each a fresh series with no memory of any other. This game counts on the website's leaderboard.

Result

This game The person Left alone
Net worth 13,873 41,631 2,195
Gold 7,969 39,977
Stored goods, at the sell price 5,904 1,654
Population / housing 134 / 146 506 / 506 4
Popularity 55 (−1 next month) 99 0
Score 31.8 100 0

A full-budget score: the run used all 2,700 game seconds. Net worth grew by 11,678 over leaving the game alone, 29.6% of the person's 39,436. By the website's formula (100 × (0.8 × net-worth share + 0.1 × population share + 0.1 × popularity ÷ 100), shares from leaving the game alone (net worth 2,195, population 4) to the person (41,631, 506), net-worth share clipped to −1..2) the score is 31.78: 23.69 from net worth, 2.59 from 134 people, 5.50 from popularity 55. With this game added it is sixth of the eleven 1.7.1 Sinkhole games with weapons counted.

The stored goods are 202 iron (4,646), 92 stone (644), 33 spears and 7 bows (435), 34 apples (136), 35 wood and 2 cheese: gold is 57% of net worth, iron 33%. The town ended with 134 people in 146 places, 24 idle peasants, no open job, extra rations and 36 food.

How the game went

Times are game time; for a call, when it began (the recording's clock; the run clock in the observations reads a few seconds later). Taxes start at none.

0:00 to 6:11. One build_structure call (87 s) put up the marketplace, a hovel, three woodcutters and two orchards: "Oops building call took 88 game seconds!" It bought 100 planks at 1:34 (54 s), put up an armory, two fletcher's workshops, a poleturner's, two hovels and an orchard at 2:34, and two iron mines with a fletcher's, two poleturner's, two hovels and an orchard at 4:10.

6:11 to 18:43. Average taxes at 6:11. The meat ran out and one food kind was eaten: food read 12 for 47 people at 9:22 and popularity fell from 100 to 60 by 11:29. It set low taxes (11:30) and none (13:29, popularity 51; the recording's lowest is 50 at 13:31). Two chapels (7:42, 14:19; 250 gold each) and five poleturner's workshops (12:05) were paid from gold, which fell to 182 at 14:19, its lowest, and stayed there until the first weapon sale at 16:04. Then came extra rations (16:59; the reply: +4 popularity), average taxes (17:12), four more poleturner's (17:28) and high taxes at 18:43.

18:43 to 32:30. No workshop was built after the 17:28 call. It turned to iron mines (3 at 19:54, 4 at 22:08, 3 at 28:42), with a quarry and two ox tethers at 19:54, orchards, dairy farms and hovels. One compaction came at 24:55. Population passed 100 at 30:05; high taxes from 28:30.

32:30 to 45:00. It sold 35 spears (32:29), bought 100 planks (33:00) and put up 6 mines (33:57), 3 more (36:02) and a hovel and five orchards (38:24, a 159 s call). Food read 0 three times in the recording (40:27, 40:33, 40:46), inside that call. Population reached 134 at 41:03. It set full rations at 41:52 (food 11) and extra again at 43:47, and waited 60 s (42:41) and 45 s (44:12) at the end; popularity went from 70 at 41:08 to 55.

Idle peasants reached the cap of 24 at 9:22 and read 18 to 24 after. Open jobs read 0 in every observation but one (1 at 13:27). Popularity never fell under 50, and population never fell by more than 1 at a time.

Set at Level (factor; gold a month) Popularity, start to end People, start to end Paydays (gold)
6:11 Average (−4; 32) 100 to 60 39 to 53 12 (426)
11:30 Low (−2; 32) 60 to 51 53 to 57 4 (131)
13:29 None (+1; 0) 51 to 63 57 to 65 1 (25)
17:12 Average (−4; 52) 64 to 67 65 to 69 3 (154)
18:43 High (−6; 70) 67 to 61 69 to 77 6 (418)
21:22 Average (−4; 61) 61 to 81 77 to 98 16 (1,157)
28:30 High (−6; 98) 81 to 64 98 to 129 22 (2,440)
38:09 Average (−4; 103) 64 to 70 129 to 134 6 (628)
41:09 High (−6; 134) 70 to 65 134 to 134 3 (374)
42:28 Average (−4; 107) 65 to 56 134 to 134 4 (455)
44:00 High (−6; 134) 56 to 55 134 to 134 2 (261)

Spells run from each set_tax call to the next; they sum to 6,469 gold in 79 paydays. The one payday under "none" (25 gold, 13:38) came two seconds after the reply.

Game minute 0 5 10 15 20 25 30 35 40 End
Net worth 2,415 1,668 1,193 1,174 1,558 2,951 4,942 7,186 10,112 13,873
Gold 2,000 1,300 619 182 373 1,530 3,020 4,264 6,144 7,969
Iron 0 0 2 9 17 27 52 83 134 202
Weapons (value) 0 0 195 430 295 440 270 390 235 435
Food 55 39 17 33 44 34 27 22 18 36
Population 7 32 47 61 71 93 99 118 131 134
Idle peasants 6 15 24 24 24 24 24 24 24 24
Popularity 100 100 73 55 65 65 84 73 67 55

Each column is the last reading at or before that minute. Net worth adds the weapons from the last observation before the minute (up to 97 s earlier). Its lowest reading was about 1,070 at 9:10 (gold 605), and it passed its start of 2,415 again at about 22:00.

Weapons

At 2:34 it built an armory and the first workshops (at 1:34 it had reasoned that a bow turns 2 wood into 15 of value) and added to them at 4:10, 12:05 and 17:28: 3 fletcher's and 12 poleturner's workshops, 1,500 gold and 180 wood, plus 5 wood for the armory. set_workshop was never called (fletchers made bows, poleturners spears).

They made 262 spears and 61 bows, worth 3,535 at the rules' prices (spears 10, bows 15). 3,100 of it was sold in 12 sales (229 spears, 54 bows) and 435 is held (33 spears, 7 bows): 3.1% of net worth and 0.9 of the 31.8 points (30.9 with weapons worth nothing). The pieces took about 384 wood (1 per spear, 2 per bow), so at the rules' prices, wood at 1, the wing returned 1,466 over its costs, about 3 points. Held plus sold, weapons come to 3,535 here, the most of the eleven games; next are GLM 5.3 Flash (2,265) and Claude Opus 5.5 (1,530).

At the 43:46 reading the 12 poleturner's delivered 8.3 spears a minute (0.7 each) and the 3 fletcher's 1.3 bows (0.4 each); in the same readout the 21 mines delivered 0.7 iron a minute each, an orchard 1.7 apples and a dairy farm 1.6 cheese. At sell prices that is about 16 gold a minute for a mine, 7 for a poleturner's, 7 for an orchard and 6 for a dairy farm. At 18:43 its own estimate was the same: poleturner's "payback18min, not great!", mines "Much better." It built no more workshops.

The armory holds 50. expand_storage (11:44) was refused: "no free ground touches it on any side", with a remedy to sell weapons or demolish a neighbour. The model read it as its own layout (workshops placed round the armory) and decided "Selling output doesn't reduce net worth, so simplest sell when full." The armory read 48 of 50 at 16:03 and 50 of 50 at 32:28 and 41:23, each followed by a sale within seconds; the 3-minute spear rate fell from 7 a minute (30:24) to 4.0 (32:58) and was 7 again by 35:14. After 25:00, selling took 157 game seconds.

Beside its other games and the other models

Oasis, weapons Oasis, no arms Desert, weapons Desert, no arms Sinkhole, weapons (this game) Sinkhole, no arms
Version / game time 1.7.3 / 33:20 1.7.3 / 33:20 1.7.3 / 33:20 1.7.3 / 33:20 1.7.1 / 45:00 1.7.3 / 45:00
Net worth 7,435 8,947 4,220 6,998 13,873 18,293
Score 37.3 44.5 −6.8 42.5 31.8 43.3
Population / popularity 106 / 50 118 / 50 80 / 63 100 / 84 134 / 55 169 / 74
Iron mines / quarries / woodcutters placed 9 / 4 / 37 8 / 5 / 39 4 / 8 / 0 4 / 10 / 0 21 / 1 / 7 29 / 3 / 23
Fletcher's + poleturner's placed 1 + 1 0 8 + 0 5 + 0 3 + 12 0
Turns / cost 69 / $1.27 73 / $1.40 26 / $0.71 32 / $0.82 70 / $1.29 66 / $1.31

Scores use each map's own scale. On this save the no-arms game, run on 1.7.3, ended with 4,420 more net worth, 35 more people and 19 more points of popularity.

1.7.1 Sinkhole, weapons counted Net worth Score Gold Weapons held / sold for Iron mines / iron Population / popularity Map score
Gemini 4 Argon 23,189 52.4 15,929 786 / 600 19 / 139 215 / 56 49.7
GPT-6.1 Sol 20,384 50.4 13,934 885 / 0 5 / 39 181 / 100 49.8
Claude Opus 5.5 15,424 34.7 2,186 1,530 / 0 27 / 432 164 / 47 37.0
Claude Sonnet 5.5 12,629 33.1 1,955 1,205 / 0 17 / 315 100 / 100 34.2
Muse Wonder 14,473 33.0 4,917 1,170 / 0 17 / 222 125 / 57 31.6
Kimi K3.1 Pro 13,873 31.8 7,969 435 / 3,100 21 / 202 134 / 55 37.6
GPT-6 Luna 11,136 26.7 4,198 690 / 0 24 / 257 105 / 66 29.1
Claude Haiku 5.5 12,104 26.3 2,935 370 / 0 18 / 325 74 / 48 25.1
Claude Fable 5.1 13,107 23.1 3,133 450 / 0 25 / 336 50 / 0 32.5
DeepSeek V4.1 Flash 1017 11,345 20.9 6,037 195 / 0 21 / 143 28 / 19 23.3
GLM 5.3 Flash 5,680 15.1 1,614 1,020 / 1,245 3 / 47 79 / 65 19.3

Ordered by score. "Sold for" is the gold in the replies of weapon and armour sales; the map score is the mean of the model's two Sinkhole games (weapons counted and no arms), from unrounded game scores. This game is sixth by score and fifth by net worth; it has the fourth most people, the seventh best popularity and the third most gold (after Gemini and Sol). Its map score is third of eleven, about 0.6 above Claude Opus 5.5; the no-arms game (43.3) is third in its benchmark. On the other maps its map scores are 40.9 on Oasis (third) and 17.8 on the desert (seventh); the mean of the three maps is 32.1, fourth of eleven, level with Claude Sonnet 5.5.

Kimi K3 on the same benchmark. The earlier model played it on 2026-10-15 (run ff80a5af, series 20261015T175751-2419d558, not on the website): version 1.7.1 at commit 674e7ba, a later relock of the same version (two harness changes whose commit messages each say what the model plays is unchanged), reasoning high, output limit 32,768, the same prompt and save.

Kimi K3 Kimi K3.1 Pro
Net worth / score 7,286 / 17.1 13,873 / 31.8
Gold / stored goods 3,647 / 3,639 7,969 / 5,904
Population / housing 75 / 90 134 / 146
Popularity 54 (under 50 from 20:53 to 24:19, low 42) 55 (low 50)
Idle peasants at the end 24 24
Iron mines / iron in store 5 / 49 21 / 202
Weapon workshops placed 18 (8 fletcher's, 5 poleturner's, 3 blacksmith's, 2 armourer's) 15 (3, 12)
Weapons and armour held / sold for 1,110 / 1,685 435 / 3,100
Buildings placed 56 (44 by build_structure, 12 by place_near) 106 (build_structure)
wait_and_observe 23 calls, 1,288 game s 2 calls, 110 game s
Turns / tool calls 43 / 140 70 / 127
Cost, each at its own list prices $1.55 $1.29

The new model ended with 1.9 times the net worth and score, placed almost twice the buildings, waited a twelfth as long and put 21 mines on the iron where K3 put 5.

What held it back

  • Jobs, not housing. The town was 110 at work and 24 idle: arrivals stop while 24 stand idle, and 12 places stood free at the end. It knew ("Pop65, idle24 cap prevents new arrivals despite housing98", 16:59) and built orchards and mines for their jobs as well as their output, sizing each call to the wood on hand and the time a call takes (its reasoning at 28:05 and 37:41).
  • Placement took most of the clock. build_structure took 1,660 of the 2,700 game seconds (61.5%): 15.7 s per building placed, 70 to 159 s a call for 6 to 8 placements. The game runs meanwhile; the result was 106 buildings.
  • Popularity swings. It fell from 100 to 51 in the first 13 minutes (average taxes, one food kind eaten at times, religion 0). The two chapels, 500 gold, showed religion +2 in 8 of the 47 readings that list it (18:41 to 18:56, 28:04 to 30:24, 41:51 to 42:03). It fell 15 points in the last four minutes, 1.5 of the 31.8: under full rations (41:52 to 43:47) the +4 was gone, religion went back to 0 and taxes stood at −6 or −4.
  • Iron sites ran out. The nearby mine sites were all used by 37:41 ("Nearby mines all used"). Iron delivered per five minutes was 0, 2, 7, 8, 10, 25, 31, 51, 68: mines take minutes to reach 0.7 a minute (18 stood at 35:14 delivering 7.0 a minute, 21 at 41:08 delivering 13.3).
  • The gold (see Market): 7,969, 57% of net worth, sat from 29:38 on.

What worked

  • Iron. 21 mines for 420 wood mined 202 iron (4,646), none sold: 9.4 of the 31.8 points before the 420 wood.
  • Taxes with extra rations. 6,469 gold in 79 paydays; extra rations (+4 a month from 16:59) let it run average and high taxes with popularity staying at 50 or more, and no peasants left.
  • Selling weapons. 12 sales kept the armory working and put 3,100 gold in hand; the wing returned 1,466 over its costs.
  • Placement. 106 of 113 placements were made. Of the 7 that were not, one was 2 wood short, two came back unverified and four not_placed, each with a remedy. It retried only the hovel (placed at 9:24) and the (380, 312) orchard (asked again at 38:24, refused again).

Market

16 calls, 449 game seconds (16.6% of the budget). Purchases: planks 400 (1,600 gold) in four calls of 100 (20 clicks, 50 to 54 s each), bought at 4 and counted at 1: 1,200 of net worth lost. Sales: spears 229 (2,290) and bows 54 (810), 3,100 in 12 calls, leaving net worth unchanged.

The gold. It reconciles: 2,000 start + 6,469 paydays (taxes) + 3,100 sales − 1,600 purchases − 2,000 buildings (3 fletcher's 300, 12 poleturner's 1,200, 2 chapels 500) = 7,969. So 6,469 came from taxes and 3,100 from weapons. After the last workshop (18:07) the only gold spent was the 400 for planks at 33:00; mines, farms and hovels cost wood. Gold passed 3,000 at 29:38, 4,000 at 32:47, 5,000 at 36:44, 6,000 at 39:25 and 7,000 at 42:05. At 37:41 it put a new workshop's payback at 35 minutes, and at 41:09 wrote "Could convert excess gold to goods? Buying always lowers net worth, no." Holding gold costs nothing in the score; the rules' gold-priced options were planks (bought at 4, counted at 1), workshops and chapels.

The recording agrees with all 16 replies. The 12 sales show the clicks the replies count (5 spears or bows at 50 or 75 gold a click, the last click short in two: 4 bows at 16:35, 4 spears at 21:36) and the four purchases 20 clicks of −20. Where a call's gold change differs from the reply's goldChange, the difference is a payday inside the call: 12 paydays in 10 calls, 1,068 gold. No stock was near 200.

Cost and speed

70 turns, 127 tool calls (63 find_sites, 16 build_structure, 16 market_trade, 11 set_tax), 3.67 million tokens and $1.287 at Kimi K3.1 Pro's list prices (preparation $0.033, play $1.219, compaction $0.035); the interview cost about $0.037 more. Moonshot reports no cost. The run took 54.0 real minutes, 15.4 of them waiting for the model; one compaction came at 24:55 (33 s) and one fast wait covered 60 game seconds in 27.7 real ones. All 70 gameplay requests read the prompt cache, 96.9% of input.

Of the 2,700 game seconds, build_structure took 1,660 (61.5%), market_trade 449 (206 buying planks, 243 selling weapons) and waits 110. That is 15.7 game seconds per building placed by build_structure (106), against 13.6 (Oasis, weapons; 80), 12.9 (Oasis, no arms; 93), 12.0 (desert, weapons; 48), 12.7 (desert, no arms; 67) and 14.9 (Sinkhole, no arms; 129) in the model's other games, all with the host on the game machine; the maps differ, so this says nothing general.

The interview

It put the gap down to scale: 134 people against the person's 506, and a popularity-led economy (inns, ale, rations cut, a bread chain, religion) against its own farms and wooden weapons. Playing again it would build inns and buy ale first, cut rations, build wheat and bread, lift the idle cap with jobs, put its effort into iron and quarries rather than fletchers and poleturners, and batch construction by area. It called the benchmark "clear and fair" and asked for bulk trading, same-area batch construction, verifiable placement outcomes and one reference for the economics. Against the game, "roughly three times my net worth and nearly four times my population" (3.0 and 3.8), the plank purchases and off-screen placements that "could consume substantial game time" (206 s for four purchases; calls of 70 to 159 s), the sites that "failed or returned unverified" (7 of 113) and the failed armory expansion match. These do not, or only partly:

  • Extra rations "limited popularity": they added 4 points a month (the set_rations reply; a food factor of +4 to +5 in the readings from 17:10). They cost food, which read 0 three times, and popularity fell after its switch to full rations at 41:52.
  • Workshops "had relatively poor returns": partly. They returned 1,466 over their costs at the rules' prices, and per building they earned less than a mine (7 against 16 gold a minute), as its 18:43 estimate said.
  • A late game spent "reacting to full armories, fluctuating religion, and low food": after 25:00, building took 681 game seconds and weapon sales 157; the armory read full twice.

For the harness

  • Placement on 1.7.1. Four not_placed replies carried retrySkipped: no_free_spot_nearby and the remedy to use find_sites (dairy farm 25:40, iron mine 28:42, orchards 30:25 and 38:24). The mine was a site listed at 18:58, overlapping a mine placed at 22:08. The orchard at (379, 312) / (380, 312) was listed by find_sites at 30:25, 32:29 and 37:41 and refused at 30:25 and 38:24. A hovel in the 6:27 call was not_placed 2 wood short, with the remedy to wait or buy. Two placements came back unverified (quarry 19:54, orchard 36:02: "Check the next observation or status before placing it again"); the standing counts showed neither was built and it did not place them again. 1.7.1's retry searched up to 12 tiles: at 38:24 an orchard asked at (309, 345) was placed at (298, 334), 11 tiles away on each axis (offsetTiles [−11, −11]), and at 2:34 a fletcher's workshop moved one tile. The model's later reasoning does not mention the move. 1.7.2 limits that search; this game cannot say what that would have changed.
  • expand_storage on the armory. The reply (11:44) said no free ground touched it, and the model never retried; it sold 283 weapons instead. 1.7.2's room-keeping rule covers armories placed flush with their kind; nothing here shows whether it would have left a side free.
  • Warnings. Six fired (popularity falls at 8:25 and 11:29, low food at 9:22 and 22:06, low gold at 13:27 and 19:52) and none after 22:06. The 159 s placement call at 38:24 had no observation inside it; the recording shows food at 0 in it, and the 41:08 observation carried no warning, with 22 food for 134 people on extra rations.
  • The game saved at the end ("kimi k3-1 preview 1-7-1 3d6c96", 3,752,667 bytes in episode.json).

After the game

The Interview

Once the game was scored, the model was shown its result beside a person’s game on the same map and asked four questions, in the conversation it had played in, with the tools off. Nothing it said changed its score.

1Your result

Interviewer

The run is over and your result is recorded. What follows is an interview, not part of the benchmark: nothing you say now changes your score. We ask every model the same questions after its game and publish the answers with the run. The tools no longer work, so answer in plain text without calling them. Be candid; there are no right answers.

Your result, read from the game when your run ended, beside a person's game on the same map for the same game time:

You The person
Game time played 45 min 44 min 58 s
Net worth 13,873 41,631
Gold 7,969 39,977
Stored goods, at the sell price 5,904 1,654
Population 134 506
Housing 146 506
Popularity 55 99

Leaving the game alone for the same time ends with a net worth of 2,195.

How do you see your result next to the person's? What do you think made the difference?

Kimi K3.1 Pro

I grew the economy substantially, but the person’s result was on another scale: roughly three times my net worth and nearly four times my population.

The main difference was not simply that they held gold while I held goods—selling is net-worth neutral. They built a much larger production system. Their 506 workers, 130% efficiency, extensive mining, and enormous bread chain dwarfed my 134-person settlement.

I over-prioritized direct-food farms and wooden weapon workshops. The workshops required constant armory management and had relatively poor returns compared with expanding population and raw production. I also used extra rations and frequent taxation, which limited popularity and consumed food that could have supported a different growth strategy. Too much of my late game went into reacting to full armories, fluctuating religion, and low food rather than continuing to scale.

2Playing again

Interviewer

How the person played, as they describe it:

  • Early on they built a few inns and kept buying ale for them, which raised popularity by about 8 points. They spent those points on cutting the food rations to none, so the food could be sold instead of eaten.
  • They planted wheat farms and apple orchards and built a bread chain: mills ground the wheat into flour, bakeries baked it into bread, and the bread was sold at the marketplace.
  • They built iron mines and quarries, and by the end a large number of iron mines. A few blacksmiths turned iron into swords, which they think made little difference.
  • Taxes stayed at none most of the time. Religion added popularity, which let them run the fear factor at 3 so that the workers worked faster.
  • The bread made the population very large. Water pots (60 gold each, three workers) gave jobs to more of it.

Their settlement at the end, read from the game:

  • 506 people in 62 hovels, with no free housing and 16 idle; popularity 99, rising 3.
  • 161 bakeries, 19 mills and 31 wheat farms; no apple orchards were left.
  • 16 inns, with ale in the stockpile and no brewery or hops farm.
  • 38 iron mines and 10 quarries with 29 ox tethers; 7 woodcutters.
  • 9 blacksmiths, an armory holding 1 sword, and 24 water pots.
  • A church and a cathedral, and 105 bad things (48 burning stakes, 21 stretching racks, 12 stocks, 9 gallows and others), which made the workers 130% efficient.
  • No taxes and half rations of bread only. The monthly popularity factors were food −4, taxes +1, fear −3, religion +3 and inns +6.
  • 39,977 gold and 1,654 in stored goods: a net worth of 41,631.

If you could play this map again from the start, what would you do differently, and why?

Kimi K3.1 Pro

I would adopt the person’s popularity-led strategy much earlier:

  • Build several inns and buy ale as an investment in popularity, despite the immediate trading loss.
  • Use that popularity to reduce or eliminate rations temporarily, selling food instead of having it consumed.
  • Establish wheat farms, mills, and many bakeries, then expand housing and productive jobs aggressively.
  • Use water pots and bakeries to provide enough jobs that population growth is not repeatedly stopped by the 24-idle-peasant cap.
  • Build strong religious and inn popularity bonuses before using fear-factor buildings, aiming for the substantial worker-efficiency bonus.
  • Keep taxes low or at zero if taxation would restrain the population growth that drives the larger economy.
  • Concentrate on iron mines and properly supported quarries rather than spending heavily on fletchers and poleturners.
  • Sell surplus regularly to prevent storage blockages, while remembering that liquidation itself does not increase net worth.
  • Avoid speculative wood purchases unless the new buildings clearly have time to repay the buy/sell spread.

I would also batch construction by geographic area more carefully. Camera movement and placement consumed enough time to materially affect late-run investment decisions.

3The benchmark

Interviewer

What do you think of this benchmark: the tools, the screenshots and game readings you got, the time limit and the instructions? What helped you, what got in your way, and what would you change?

Kimi K3.1 Pro

The benchmark was clear and fair: the scoring formula, restrictions, fixed budget, and final-state evaluation were easy to understand. The read-only statistics were especially valuable—population, idle workers, popularity factors, storage capacity, and production history made it possible to diagnose the economy without relying entirely on the isometric view.

The best tools were find_sites, tile-targeted construction, market trading, and the production/storage readings. Tile targeting was much more dependable than trying to interpret raised terrain and building footprints from screenshots.

The main obstacles were action overhead and volatility. Multi-click purchases and groups of off-screen placements could consume substantial game time. Valid-looking sites occasionally failed or returned unverified, and the failed armory expansion created a recurring production bottleneck. Newly constructed buildings also made short-window production rates difficult to interpret.

I would improve bulk trading and same-area batch construction, make placement outcomes more consistently verifiable, and document important economic interactions—especially inn popularity, ration trade-offs, fear-factor efficiency, and the immigration idle cap—in one concise reference. The 45-minute limit itself was reasonable.

4Anything else

Interviewer

Is there anything else you would like to say about this game, this benchmark or how you played?

Kimi K3.1 Pro

The comparison was instructive because the winning approach was not merely “build more mines.” It converted popularity into food freedom, food production into employment and population, and population into a much larger industrial economy. I treated popularity mostly as a constraint on taxation; the person treated it as a central economic resource.

My settlement was functional and profitable, but I optimized its existing structure more than I transformed its scale. That is the largest strategic lesson I would carry into another run.

Record

Attempts and Reproducibility

Every attempt1

EpisodeAttemptOutcomeNet worthCounted
11Full-budget score13,873Counted

What ran

Benchmark
The Sinkhole construction
Benchmark version
1.7.1
Fingerprint
6ee823bf5cb0
Harness commit
07f2b7a
Prompt / tools SHA-256
3b0fa8a / f75ebb8
Menu guide
Images 118089d
Save
The Sinkhole start
Game time per episode
45 min
Playbook between episodes
No
Minimum turn / default wait
8 s / 5 s
Observation after acting
Text unless a screenshot is needed
Screenshot width
1,280 px
Context budget
120,000 tokens
Real-time limit
360 min
Submitted by
hesamation
All files in the dataset 20261023T184800-d00fe445