The Harness and Dashboard

Start the Host

npm run dev

Open http://127.0.0.1:4317. The server serves both the dashboard and its API, and listens only on 127.0.0.1. On macOS, start it from a terminal: a server started inside another app, such as an editor's preview pane, can be denied local network access and fail to reach the game machine.

The server reads its prompt and harness files once at startup. Restart it after editing anything in prompt/ or harness/.

Connect the Game

Start the game on the game machine and load a single-player map, then click Connect game. The host opens SSH sessions that start:

• the window bridge, which captures the game window and sends input to it;
• the reader stream, sampled every 100 ms;
• the memory guard (the header shows MEMORY GUARD ON);
• the control monitor, a terminal window on the game machine.

The dashboard then shows the game and current stats. Outside runs the game preview refreshes every 5 seconds, every 2 seconds while you have manual control, and at once after a manual action. Stats that are older than 1.5 seconds, or unavailable in menus and loading screens, are shown as unavailable, never as zero.

Model Profiles

A profile is a display name, a model ID, an endpoint and a key. Home → Configure lists them; + Add creates one.

• Endpoints. https://openrouter.ai/api/v1 and Moonshot's endpoint use the OpenAI-compatible chat completions API. https://api.anthropic.com uses Anthropic's own Messages API, for Claude models by their Anthropic IDs (such as claude-haiku-5-5), with adaptive thinking: the reasoning level becomes Claude's effort (minimal and low both become low). Any other endpoint is treated like Moonshot's.

• The default profile, Kimi K3, reads MOONSHOT_API_KEY (and optionally MOONSHOT_MODEL, MOONSHOT_BASE_URL) from .env. Restart after changing .env.

• Keys typed into the dashboard are saved in harness/runtime/config/models.json, a git-ignored file readable only by your user. It is plain JSON, not an encrypted vault. The API never returns saved keys, and leaving the key field blank keeps the existing one.

• Endpoints must use HTTPS (plain HTTP only for localhost) and cannot contain credentials or a query string. Changing a profile's endpoint requires entering the key again.

• A profile with saved runs keeps its model ID and endpoint, so its runs stay attributable. Create a new profile for a different model.

• Model settings (Configure → model settings), recorded in every run:

  Setting	Values	Meaning
  Reasoning	default, off, minimal, low, medium, high, max	default sends no reasoning setting and lets the endpoint decide. off is OpenRouter only. Moonshot takes default (none sent, which Kimi K3 treats as max) or Kimi K3's levels low, high and max, sent as its reasoning_effort; Kimi K3 has no medium. On Anthropic a level is Claude's effort. max is the endpoint's highest effort, as Artificial Analysis reports models: Claude's max, OpenRouter's xhigh
  Max tokens	1,024 to 131,072 (default 8,192)	Output tokens per reply, reasoning included. Used for gameplay, preparation and compaction (compaction never below 8,192)
  Providers, allow fallbacks	Provider names in order of preference	OpenRouter only: which upstream providers may serve the model. Empty lets OpenRouter choose for each request
  Prices	US dollars per million tokens: input, output, cache read, cache write; optionally higher ones for prompts over a number of tokens	Not OpenRouter, which reports what it billed. The provider's list prices, which the host applies to each request's tokens to record its cost (see Context). Cache write is the 5-minute rate; a prompt counts input, cache reads and cache writes. A run cannot start without them. Runs record the prices they used

  Profiles created before these settings existed keep what the harness used to send: low reasoning on OpenRouter, the endpoint's default elsewhere, 8,192 tokens, no provider preference.

• A profile can instead read OPENROUTER_API_KEY or ANTHROPIC_API_KEY from .env, by creating it through the API with envKey set to that name; the endpoint must be OpenRouter's or Anthropic's.

• Configure → Test saved model sends one short, billed request.

Manual Control

Take control lets you click, right-click, drag, scroll and press the allowed keys in the live view (focus the view to type). P · Pause and Esc buttons are available. Manual and agent input never mix: starting a run, leaving the page or opening settings turns manual control off.

Start a Run

On Home, fill in:

Start run is refused unless the game is connected, the model has a key (and prices, unless it is on OpenRouter), the memory guard is active and the reader confirms a single-player map within 3 seconds. The run is named Model · Benchmark · YYYYMMDD-HHMMSSZ (UTC).

The API also accepts a turn limit of 1 to 12 (maxTurns) for short diagnostics; the dashboard does not offer it.

Watch and Stop a Run

While a run is active the dashboard shows the agent's latest screenshot, stats, the agent's messages and tool calls, its plan and notebook, the phase, the time left, inference timing and compactions. The dashboard takes no screenshots of its own during a run: the preview changes when the agent observes. Extra captures would queue behind the agent's inputs and cost it game time.

Stop a run with Stop in the dashboard, by disconnecting, or with the control monitor on the game machine (below). Stopping aborts the model request and waits, and cancels queued input; an input already being sent finishes its key or button release.

If the host restarts, unfinished runs are marked interrupted. Runs cannot be resumed.

What a Run Saves

Every run gets its own folder, harness/runtime/runs/<name>-<time>-<id>/, readable only by your user. The sidebar lists runs by model; select one to see its record and logs. Export logs and Download full events download them.

Harness version. Every run records which code produced it, in run.json under harness: the git commit the host started from, whether the files the benchmark fingerprint covers had uncommitted changes (and a hash of those changes; docs, tests, the dashboard and the dataset tools do not count), and hashes of the exact system prompt and tool definitions. The run page shows it, and npm run report has a Harness column, so runs made with different code are easy to tell apart. run.json also records the benchmark version under benchmark: the version, the fingerprint of the files it covers, and a hash of the private preparation images.

Known API keys and fields named like keys are replaced with [redacted] before anything is logged; a long base64 string or a data:image URL is left alone instead of scanned for one, the same 512-character rule the dataset tools use. Treat the runs folder as private anyway: it holds full screenshots and model output.

Run Videos

With Record video on (or --record for episodes), the game machine records the game window while the agent acts, and the host renders an edited MP4 when the run ends.

• Recording. A separate SSH session captures the game window only, scaled to 1600 × 900: 10 frames per second for 2.5 seconds after each input, 4 otherwise. Recording holds while the game is paused for thinking. It runs independently of the agent's own screenshots and clicks, and a recorder failure never stops the run. The frames are JPEGs kept in <run>/recording/frames/, indexed by <run>/recording/frames.jsonl: a 2-game-minute run recorded at the earlier 1440 × 810 took 370 MB; at 1600 × 900, measured offline, a 25-game-minute run writes about 7,300–8,600 JPEGs (roughly 400 KB each) for about 3 GB.
• The video (<run>/video.mp4, 1920 × 1080 H.264 with no audio track, one chapter per turn) plays actions at real speed with a drawn pointer, click rings, building names and key badges; fast-forwards idle stretches; cuts paused frames; and shows each thinking pause briefly, the paused game beside the model's reasoning as it types out. The game fills 1600 × 900; beside it are the plan and the turn's tool calls, below it game time and the main stats. It ends on the result: the final overview beside the score. A 2-game-minute run renders to about 1:40 and 11 MB. The dataset's copy is scaled to 720p (Publishing).
• Compaction. Once a render succeeds, the host queues tools/video/compact-recording.py on the same run, right after it, still serialised (compaction is CPU-heavy): it re-encodes recording/frames/*.jpg, in frames.jsonl order, into one recording/frames.mp4 (libx264, CRF 18, one video frame per recorded frame), checks with ffprobe that the result holds exactly as many frames at the same size, writes recording/frames-video.json recording what it did, and only then deletes the JPEGs. Offline tests measured the video 10–25× smaller than the JPEGs at SSIM ≥ 0.987. A failed render is never compacted; a compaction that fails verification is only logged and leaves the JPEGs in place — nothing is deleted unless the result checks out. render-video.py reads recording/frames.mp4 once the JPEGs are gone, decoding it with one sequential ffmpeg pipe (restarted on the rare backward seek); a re-render afterwards, by hand or by npm run package, works the same way.

Re-render or change the edit by hand (needs Pillow, ffmpeg with libx264, and ffprobe):

npm run video -- <run folder>                        # writes <run>/video.mp4
npm run video -- <run> --idle-speed 16 --max-idle 2  # compress idle play harder
npm run video -- <run> --max-think 0 --outro 0       # no thinking pauses, no result

Other options: --speed, --min-think, --think-rate, --fps, --max-seconds (for previews) and --encoder.

Compact a run recorded before this existed, or one the host's automatic compaction missed, by hand:

npm run compact-recordings -- <run folder> [<run folder> ...]  # named runs
npm run compact-recordings -- --all                            # every run under harness/runtime/runs,
                                                                 # skipping ones with no frames/ or already compacted

Safety Boundaries

Host server. Listens on 127.0.0.1 only, rejects other Host headers, and requires a dashboard header (and a matching origin, if sent) on every request that changes something. Keys stay on the host.

Game machine. The host starts each helper over SSH with key login; no service listens on the network. The window bridge:

• attaches to exactly one game process of your own user (Steam app 3024040) and its window, matched by title, window class and process, and refuses if anything is ambiguous or changes;
• captures that window's own image, not the desktop, so other windows covering the game never appear in screenshots;
• sends input events addressed to the game window only, with no system-wide input injection, pointer warping, shell commands or arbitrary keyboard shortcuts;
• checks the game is the active window and has the screenshot's size before every input, and rejects coordinates outside it;
• exposes only finite clicks, drags and key presses, never a held key.

This bridge is written for the tested GNOME, Xwayland and Proton setup and refuses states it does not recognise. It limits what the tools can do; it is not a sandbox against other code running as your user.

Single-player only. The reader rejects multiplayer, map editor and spectator states. Runs cannot start without its confirmation, and if it reports a multiplayer game while connected, the host disconnects.

Memory guard. The game's garbage collector can stall and let memory grow without end (see Status); the reader's collector watchdog breaks a stall, and the guard is the backstop. Every connection runs a guard that closes the game if it uses more than 7 GiB or the machine has less than 2 GiB available. A run cannot start without it. If it closes the game, the run ends with that reason and the last memory reading. During a run, events.jsonl gets a memory_sample every 5 seconds (the game's memory, the reader's managed-heap reading, the game clock and pause state) and a gc_watchdog event for each collection the reader forced; run.json counts them under progress.memory.forcedCollections.

Control Monitor

Connecting opens Crusader Arena — Game control monitor, an xterm on the game machine. It shows the model and run, the phase, turns and tokens, current stats, screenshots sent, model requests and every input with its acknowledgement or failure. It only uses private files under $XDG_RUNTIME_DIR/crusader-arena/ and adds no network listener.

Click the game again before giving input: the bridge refuses input while the monitor or any other window has focus. The monitor never appears in the agent's screenshots.

Run Overlay

For people watching the game machine's screen, the monitor also starts an overlay (tools/ubuntu/overlay.py): a panel at the top centre of the screen with the model and benchmark, the game time used and left as a progress bar, and a line of figures (net worth, gold, people, popularity, turns, tool calls and the run's cost so far). It shows only while a run has a game clock, reading the same status file as the monitor, which the host updates every second with the session's live game time; it exits 30 seconds after the bridge goes.

The agent never sees it. It is a separate window above the game, and the agent's screenshots and the run videos copy only the game window's own image (its compositor pixmap), not the screen. It cannot take focus (an override-redirect window) and lets clicks through (an empty input region), and the bridge sends input to the game window directly. A recording of the whole screen, such as a stream, shows it. It runs on the control venv's Python, drawing the panel with Pillow and showing it with python-xlib, so it needs nothing beyond the bridge's own setup.

A round played on a version made before the overlay (1.5.6 and earlier) has a host that sends no game clock and a monitor that starts no overlay. For those, npm run episodes copies this checkout's overlay.py to the game machine and feeds it itself (overlay-feed.ts): once a second it reads the round's host state and sends the panel's figures over its own SSH connection to overlay.py --feed, which keeps the latest in overlay-feed.json and starts the overlay when a run's clock appears, and exits after 60 seconds without a line; the overlay takes the display from the game process, as the bridge does. While that feed is fresh (4 seconds) its figures replace the status file's; the bridge's own status still decides whether the overlay shows. That host updates the run's game time only when a turn or tool changes it, so in between the feed runs the clock on from the reader's game tick. Tool calls are counted from the run's events.jsonl.

Development Checks

npm test                                                  # harness tests
npm run build                                             # type check and production build
npm start                                                 # serve the built dashboard
python3 -m unittest discover -s tools/video               # video timeline and a tiny render (needs ffmpeg)
python3 -m unittest discover -s tools/runner              # the runner's menu screen probe (needs Pillow)
python3 -m unittest discover -s tools/ubuntu -p 'test_*.py'  # game-machine helpers (run on Ubuntu)

The harness tests drive real agent turns against fake models, game devices and clocks, including long runs, repeated compaction, stops, deadlines and failures.