Setup

What You Need

• The game. Your own Steam copy of Stronghold Crusader: Definitive Edition (Steam app 3024040). The reader was mapped against one game build (Steam build 24816905): it checks four game files against fingerprints in src/windows/probe.cpp and refuses anything else. After a game update it stops working until it is re-verified.
• A game machine with Steam and Proton. Tested: Ubuntu 26.04 with GNOME on Wayland (the game runs under Xwayland), Proton Experimental, Steam from the Snap and from the regular package. The window bridge needs X11 access to the game window, so other desktops and native-Wayland setups need their own bridge.
• A host machine with Node.js 22.12 or newer (tested with 24.4.1), Python 3 and OpenSSH. Tested on macOS. Pillow lets npm run episodes recognise the game's main menu instead of waiting out fixed timeouts (about 105 s more per launch without it). Run videos also need Pillow and ffmpeg with libx264; the recording is later compacted into a smaller video with the same ffmpeg install's ffprobe, which verifies it. Generating the guide images needs macOS.
• SSH key login from the host to the game machine.
• A model API key: Moonshot, OpenRouter or another OpenAI-compatible provider whose model accepts images and tool calls.
• A 16:9 game window. Menu positions were measured at 1920 × 1080; tools that click menus refuse other aspect ratios.

1. Prepare the Game Machine

Put a copy of this repository on the game machine. Any path works; this guide uses ~/codebox/CrusaderBench.

cd ~/codebox/CrusaderBench
bash tools/ubuntu/setup-dev.sh --install

This installs the compilers (including the MinGW cross-compiler for the Windows reader), CMake and Python with apt (asks for sudo), runs the native tests and builds the reader and its test programs. Later builds can drop --install. It does not touch Steam, the game or its Proton prefix. After this first setup, npm run episodes keeps the copy's helpers and reader sources in step with the version each run plays and rebuilds the reader when they change (see Rounds).

sudo apt install python3-pil xterm
bash tools/ubuntu/setup-control.sh

This creates .venv-control with a pinned Python X11 binding for the window bridge. xterm is for the control monitor.

In Steam, start the game once through Proton and set a 16:9 window, ideally 1920 × 1080.

Check the reader. Start the game, load a single-player map, then run:

python3 tools/ubuntu/run-proton.py probe            # finds the game and checks the build fingerprints
python3 tools/ubuntu/run-proton.py watch --samples 5  # five readings as JSON lines

See The reader for what the readings contain.

2. Prepare the Host

npm ci --ignore-scripts
cp .env.example .env

Fill in .env:

The harness uses SSH in batch mode and refuses unknown hosts, so connect once by hand and accept the game machine's host key after checking its fingerprint:

ssh player@192.168.1.20 true

For run videos, install Pillow and ffmpeg on the host (on macOS: brew install ffmpeg and python3 -m pip install pillow); brew install ffmpeg also installs ffprobe, which the recording's later compaction uses to verify it.

3. Guide Images

Before timed play, the agent studies a guide to the construction menus. The button names and roles are part of the code, so runs work without any images: the agent then gets the guide as text. The image version shows what the buttons look like and was used for all recorded runs, but it is made from game screenshots, so it is not included. The files live in .internal/ui-reference/, which git ignores:

The combined guide is preferred as a JPEG: at quality 90 it has about a third of the PNG's bytes at the same image size and model token cost, and it is resent in every request body. A checkout with only the older construction-trays.png still works; the host prefers the .jpg when both exist. To convert one by hand (Pillow, already needed by the video renderer):

python3 -c "from PIL import Image; Image.open('.internal/ui-reference/construction-trays.png').convert('RGB').save('.internal/ui-reference/construction-trays.jpg', quality=90)"

To make them, put one source PNG per menu page in .internal/ui-reference/legends/, named after the file of each of the ten pages in harness/server/visual-atlas.ts. Each source must be exactly 2620 × 1080 pixels, or the script stops. Put a 1920 × 1080 game screenshot with that menu page open at its left edge; the remaining 700 pixels of width are not used. The script draws no boxes or numbers itself, and the finished guide says its numbers match red boxes on the crops. So draw a red box for each of the page's labels, numbered in their order, at its box position (x, y, width and height in screenshot pixels). Then run, on macOS with the Xcode command-line tools (the script compiles a small AppKit program with clang):

node --import tsx tools/make-construction-trays.mjs

It crops the menu tray (the bottom 230 pixels from x 795 to 1575) from each source into trays/ (PNG), and lays out all ten crops with their labels into construction-trays.jpg (JPEG, quality 90), written at 80% of the layout's size (1530 × 2270): the model pays for its pixels in every request, and at 80% its smallest text stays readable. Only the tray is used, so the boxes must lie inside it.

4. First Run

1. Start the game on the game machine and load a single-player map.
2. On the host, from a terminal: npm run dev, then open http://127.0.0.1:4317.
3. Click Connect game. The live view, stats and MEMORY GUARD ON should appear, and the control monitor opens on the game machine.
4. Add or test a model under Home → Configure, then start a run. See The harness.

For unattended benchmark episodes, see The benchmark.

Known Issues

• Start the host from a terminal on macOS. Started inside another app, it can be denied local network access and fail with "No route to host".
• NVIDIA graphics on hybrid laptops. The game renders on the NVIDIA GPU by default, and the test laptop (Intel and NVIDIA RTX 2060 graphics, NVIDIA driver 610.57.04, GNOME Wayland) runs it that way, with no launch options. With Ubuntu's kernel 7.0.0-31 the game stalled at the end of its loading screen whenever it ran on the NVIDIA GPU, with both Vulkan and OpenGL; with 7.0.0-38 it loads normally (see Status). If it stalls there, run it on the integrated GPU with Mesa's device selection variables as Steam launch options: MESA_VK_DEVICE_SELECT=<vendor>:<device> MESA_VK_DEVICE_SELECT_FORCE_DEFAULT_DEVICE=1 %command%, with the integrated GPU's PCI IDs from lspci -nn. On the integrated GPU the game's graphics memory comes out of system RAM: at the main menu the test laptop had about 3 GB less memory available than with the NVIDIA GPU.
• Garbage-collector stalls. The game's garbage collector can stop collecting, and its memory then grows 300–470 MiB a minute, paused or running (see Status). The reader's collector watchdog breaks a stall while the reader runs, and the dashboard always runs the memory guard. For manual sessions without the reader, run python3 tools/ubuntu/monitor-memory.py --terminate-on-limit yourself.
• Restarting the reader. If a reader call is interrupted, restart the game before trying again; the reader leaves a marker and refuses to reattach to that game process.