The Agent

A Turn

A turn is one model reply plus the tool calls in it.

1. The game is paused while the model thinks. Before every model request the host pauses the game and waits for the reader to confirm the pause. Thinking costs no game time.
2. The model replies with text and tool calls.
3. Tools run one at a time, in order. Reading tools (status, get_inventory, find_sites, map_overview, flat_view, the lookup tools and the memory tools) run with the game still paused. At the first tool that acts, waits or takes a screenshot, the host unpauses the game and waits for the reader to confirm it; the game then keeps running for the rest of the reply. So a reply should read first and act after. The two minimap tools, go_to_tile and navigate_minimap, are the exception among tools that act: called while the game is still paused, they move the camera with it paused (minimap clicks work on a paused game) and unpause it only for the screenshot they return.
4. The host makes sure the model sees the result. If the turn acted without observing afterwards, or called no tools at all, the host lets the default wait pass (5 game seconds) and sends an observation. A reply that called no tools always gets a screenshot. Otherwise the run setting fallbackObservation decides: with "when-needed" (the default) it is a screenshot only if the turn sent input that works from the screenshot (any game_action, inspect_building, or a build_structure placement given by x/y; a placement given by tile does not count), and otherwise the same stats, game events and run_clock as text, with no image (see Text-only observations); with "always" it is a screenshot every time, as in every run before 1.3.0. A turn that ends with a fresh observation, or only read, goes straight to the next request (but see the limit on reading-only turns below).
5. A turn that runs the game lasts a minimum time, the minimum turn length (8 game seconds by default, about 6 real seconds at the fixed speed), counted from the start of the request. observe and wait_and_observe (also with screenshot: false) wait until the turn has run that long before they look, so an observation the model asks for always shows the game at least that far into the turn, and the result says how long the host waited. A turn that ran the game for less is topped up: the host lets the rest pass and sends an observation, chosen as in step 4; a turn that acted without looking waits for the longer of this and the default wait. This caps the number of model requests per game minute, and so the cost of a run: at 8 seconds, a 25-game-minute run has at most 187 turns that run the game. Turns that only read stay free, but only for a while: from the 12th reading-only turn in a row, the host lets the longer of the default wait and the minimum turn pass after each one and sends an observation (text under "when-needed", since a reading-only turn sends no input; a screenshot under "always"), so they fall under the same cap. The briefing tells the model so.
6. Every request shows the time left. Screenshots carry run_clock; when a reply's last tool result has no screenshot, the host adds a line such as "Game time left: 23 min 41 s of 25 min."

Each reply can take at most 8 actions. An action is anything that sends input to the game: a click or key (game_action), each building placed by build_structure, a demolition, or a camera, panel, tax or trade tool. Looking, waiting, reading stats, looking things up and taking notes are free. The tool reference marks which tools count.

What the Model Sees

• Screenshots of the game window only, as JPEG at the window's own size (tested at 1920 × 1080).
• A JSON summary with each screenshot: the game clock and budget, the date, gold, population and housing, popularity in the game's own points, tax level, food, stored goods, placement mode, camera position, any open building panel, troops (in military benchmarks only), and game messages since the previous screenshot.
• Tool results, mostly text. Placement tools report what happened to each building instead of returning a screenshot.

See Context for the full layout and how old images are dropped.

What the Model Can Do

The higher-level tools exist because raw clicking is slow and error-prone for a model. For example, build_structure opens the right menu, selects the building, clicks the target, checks with the reader whether the building was actually placed, and tries nearby spots if it was silently blocked.

Rules the Harness Enforces

• No pausing or game menu. P and Escape are refused. Space (flatten the landscape) is not offered; flat_view uses it and restores the view.
• Game window only. Every input goes to the game window, at pixel coordinates in the latest screenshot. The bridge checks the game is the active window and has not changed size.
• Fresh images only. Input based on a screenshot more than 30 seconds old is refused. Time the game spent paused by the host does not count, so slow thinking does not expire a screenshot. Two exceptions need no screenshot at all: find_sites, and a build_structure placement given by map tile instead of pixels, which read the live camera themselves right before acting (see Tools).
• Camera guards. Placement and inspection tools targeted by pixel stop if the camera moved since the screenshot they target. A tile-targeted placement has no such screenshot to drift from: the host moves the camera to the tile when it is off screen, and reports not_on_screen (no terrain click) only if it cannot bring it into view.
• Single-player only. See the harness.

Benchmark rules such as "no soldiers or defences" are instructions in the prompt, not blocked in code. build_structure only offers economic buildings, but game_action can click anything a player could.

Memory

The model's conversation is trimmed as it grows (see Context), so the agent has memory it controls:

• Plan (update_plan): a checklist of up to 20 steps, each pending, in progress or completed. Each call replaces the whole plan.
• Notebook (notebook_read, notebook_write): free text up to 8 KB, replaced whole. Writes must quote the current revision number, so the agent cannot overwrite notes it has not read.
• Notification history (notification_history): every message the reader saw on screen during the run, up to 50 per call.
• Playbook (playbook_read, playbook_write, playbook_edit), in the episodes of a learning series only: notes up to 8 KB that, unlike everything else, carry over to the next episode. After each episode the host shows the agent its final result and asks for the playbook's next version (see Benchmark).

The plan and notebook survive context compaction. The model sees them when it reads or changes them and in the summary after a compaction; they are not repeated every turn. The dashboard shows both to the operator live.

When Things Go Wrong

• Provider errors. Timeouts, rate limits, overloads, server errors, empty replies and streams the provider broke off are retried up to 5 times, waiting 2, 4, 8, 16 and 30 seconds, with the game paused. This holds for gameplay, preparation, compaction and the playbook request after an episode. Other errors end the run.
• Cut-off replies. A reply that hits the profile's output limit (8,192 tokens by default) is discarded, tool calls included (none of them runs), and retried up to twice per turn with a reminder to think briefly and keep tool arguments short.
• Pause toggles. If the reader does not confirm a pause or unpause within about 3 seconds, the host reads the pause state again and presses P again only if the game has not changed. After three failed attempts the run ends with an error.
• No BEGIN. A preparation reply without BEGIN, or cut off at the output limit, is retried; after three replies the run fails with the game still paused.
• Failed tool calls are not retried by the host. The model gets the error and decides what to do.
• Stuck loops. Four replies in a row with no tool calls end the run with an error; the briefing says so, and the host repeats it after each such reply. In the last game minute the fourth instead lets the rest of the budget pass without another request, and the run ends at its full budget (GLM 5.3 Flash's run ended 5 game seconds short this way, 2026-10-11). Replies that only read (no action, wait or screenshot) cost no game time, so a model could otherwise read forever without using the budget: from the 12th in a row, each is followed by a host wait (see A turn).
• Host shutdown. Stopping the host (Ctrl-C or a terminate signal) ends a run as stopped but, unlike the dashboard's Stop, still pauses the game and takes the final reading, waiting up to 10 seconds; a second Ctrl-C exits at once. An unexpected host error ends the run the same way. A failed background task is only logged.
• Run setup. If a run cannot be set up, it is recorded as error and the host stays ready for the next one.

Models

The model must accept images and tool calls through an OpenAI-compatible chat completions endpoint, or be a Claude model on Anthropic's own Messages API. The default profile is Moonshot's Kimi K3. Moonshot and OpenRouter endpoints are tested; other OpenAI-compatible endpoints are handled like Moonshot's and are untested. Each profile sets its reasoning level, its output limit per reply (8,192 tokens by default) and, on OpenRouter, which upstream providers may serve it; every run records these. Requests time out after 90 seconds. See the harness for adding and configuring models.