When I started leaving coding agents alone, I expected a bad abstraction, a weak test or a plausible implementation of the wrong thing.
Over two months, failures came from somewhere else. An executor inherited the wrong package manager. A preview server survived its worktree. A correct write boundary stopped work that the task had scoped incorrectly. An old instruction taught an agent a convention the repository had already retired. Green checks certified changes that still needed a reader.
None of those failures was fixed by a better prompt. Each exposed a missing property of the harness around the model, and each became a rule I could enforce.
The first two parts of this series covered validation receipts and provenance accounting; this part covers what those guarantees did not prevent: a correctly recorded pass can still inherit the wrong world.
An unattended agent finds every gap between the environment you think you provided and the one that actually exists.
The environment the agent inherits
The devcontainer image shipped pnpm 10 while package.json pinned a newer
version. In an ordinary shell, pnpm switched itself to the pinned version. The
process that launched my executors set
npm_config_manage_package_manager_versions=false, so their bare pnpm
commands used the image’s copy instead and failed with
ERR_PNPM_UNSUPPORTED_ENGINE.
A developer sees that once and fixes the shell. The unattended executors tried to make progress. They fell back to Corepack; one wrote shims into its worktree, then waited for approval to remove them. Two coordinator attempts ended on that wait.
The problem was not that pnpm failed. The problem was that the command meant something different inside the executor’s pane.
My rule now is that every tool an agent runs is already in the image at the pinned version. No agent installs a package manager, browser or shell tool into a live container. A structural test keeps the image and manifest together:
pinned=$(sed -n 's/^[[:space:]]*"packageManager":[[:space:]]*"pnpm@\([^"+]*\).*/\1/p' "$repo_root/package.json")
image=$(sed -n 's/^ARG PNPM_VERSION=\(.*\)$/\1/p' "$dockerfile")
check "the image's pnpm ($image) is the pinned one ($pinned)" test "$pinned" = "$image"
# shellcheck disable=SC2016
check "the image installs that version globally" grep -q '"pnpm@${PNPM_VERSION}"' "$dockerfile"The test matters more than the Dockerfile fix. Without it, the same drift returns on the next package-manager upgrade.
The state the agent leaves behind
Three times, a preview process outlived the agent that started it. Removing its worktree did not remove the process. The next visual suite found the port busy.
There was a quieter version too. Two executors ran visual tests in separate
worktrees but shared port 4322. Playwright’s reuseExistingServer let the second
executor connect to the first one’s preview. Its tests were green against the
wrong build.
The first fix was one deterministic port per checkout:
import { createHash } from "node:crypto";
function portOf(checkout: string): number {
if (!checkout.includes("/.worktrees/")) return 4322;
const offset = createHash("sha1").update(checkout).digest().readUInt16BE(0) % 400;
return 4400 + offset;
}
export const previewPort = process.env.VISUAL_TEST_PORT
? Number(process.env.VISUAL_TEST_PORT)
: portOf(process.cwd());The second was lifecycle ownership. Closing a pass must end what its worktrees
started, not only delete the worktrees. This sketch is illustrative and
Linux-only: /proc exposes the current working directory of each process.
import { readdirSync, readlinkSync } from "node:fs";
export function processesUnder(worktree: string): number[] {
return readdirSync("/proc").flatMap((entry) => {
if (!/^\d+$/.test(entry)) return [];
try {
const cwd = readlinkSync(`/proc/${entry}/cwd`);
return cwd === worktree || cwd.startsWith(`${worktree}/`) ? [Number(entry)] : [];
} catch {
return []; // the process exited, or belongs to another user
}
});
}
export function reapWorktree(worktree: string): void {
for (const pid of processesUnder(worktree)) process.kill(pid, "SIGTERM");
}Ports, databases and cache directories are state. If parallel agents can share one accidentally, isolation is only a folder name.
The boundary the task draws
One task changed the author’s title on the About page and in every article bio. It allowed the About screenshots to change, but not the article screenshots. The articles rendered the same bio, so their baselines changed too. The agent stopped at its write boundary.
Another task added links to article headings. Astro’s processing order required
a direct github-slugger dependency, but the task did not allow changes to the
manifest or lockfile. The agent stopped again.
Both stops were correct. The tasks were wrong.
I now scope a task by its effects, not by the first source files I expect it to edit. The list includes what the change renders and what it installs:
### BRAND-006 — The author's title follows lrji.co
- files: `src/pages/about.astro`, every article footer bio, `CONTENT.md`, the
blog skill, and the screenshots of every page that renders the bio.A task that an executor cannot finish in its environment does not enter a pass. It is excluded explicitly instead of becoming a predictable overnight block:
{
"backlog": {
"root": "backlog/",
"excludedTaskIds": ["BRAND-005"],
},
}A stop at the boundary is not agent failure. It is feedback that the task did not describe the whole change.
The conventions the agent copies
A shared skill still showed a Skill-Reviewed: commit trailer after that
convention had been retired. The surrounding repository rules were correct,
but the example was concrete and the executor copied it into its source commit.
The pass cherry-picked that commit onto the base branch, trailer included. Nothing there objected: the check that rejects the trailer had existed for a month, but it ran when I pushed, not when the pass integrated. So the pass closed green, and the push failed afterwards. I amended the message by hand.
The gate held, but in the wrong place. A rule the push enforces has to run where the pass integrates too, or every pass can deliver a branch that cannot ship.
Agents learn from examples as much as from imperative rules. I treat those examples as code now. When a convention changes, the instructions that describe it are part of the change, and the check runs during integration as well as before a push:
if git log -1 --format=%B "$commit" | grep -Eq '^Skill-Reviewed:'; then
printf 'commit %s carries the retired Skill-Reviewed trailer\n' "$commit" >&2
exit 1
fiDocumentation drift is usually described as a maintenance problem. With an agent, it becomes executable behavior.
What only review caught
The receipts were green for four defects that review still found:
- A push gate ran the full quality suite and then invoked a pre-push hook that ran it again. Every push would build the site and run the visual suite twice.
- A ported dependency-audit script had lost its
cdto the repository root, so it audited whichever project owned the caller’s current directory. - A heading link used
#as a text node. Astro included it in the heading text, so every table-of-contents entry ended with#. - A navbar mark reproduced the favicon’s outline but omitted its filled core.
Format, lint, unit tests, build and visual tests had all run. They were not the right assertions for every semantic detail of those changes.
The heading bug is a good example. The correction was tiny: keep the link empty and draw the visible mark in CSS, so the heading’s text remains clean.
export const headingLink = (id: string) => ({
type: "element" as const,
tagName: "a",
properties: { className: ["heading-link"], href: `#${id}` },
children: [],
});A validation receipt proves that the selected checks ran against the recorded commit. It does not prove that the selected checks were sufficient. Review stays. Small, single-purpose commits make it cheaper.
When not to build all of this
This harness is for concurrent writers working while I am absent. It would be too much machinery for an agent I watch in one checkout. In that case, a clean tree, the repository’s normal checks and review give immediate feedback without port allocation, process discovery or an integration ledger.
The threshold is not “AI is writing code.” It is loss of supervision. Add the extra controls when several executors share infrastructure, when a run must survive the night, or when its result will be integrated before anyone can reconstruct what happened. Below that threshold, keep the loop short and use the controls the repository already has.
The mechanisms are replaceable too. The /proc scan above fits one Linux
container where every executor runs as the same user. Separate containers can
own their processes through the container runtime. CI jobs get teardown from
their runner. A service with parallel tests may allocate databases by run ID
instead of checkout path. Keep the property, not my implementation: one owner
for every resource, and cleanup at the same lifecycle boundary that created it.
Do not copy a private harness into a small repository because the checklist looks rigorous. Start with the failure you can actually have, then make its prevention executable.
Before you leave them alone
This is the pre-flight list I use now:
- The tools agents run are in the image, pinned, with a test that catches drift from the manifest.
- Nothing an agent runs needs interactive approval from someone who may be asleep.
- Parallel checkouts never share a port, database or cache directory.
- Closing a pass ends the processes its worktrees started.
- Every task lists what the change renders and installs, not only what it edits.
- A task agents cannot finish in their environment is excluded, not left to block.
- The rules enforced before push also run where a pass integrates.
- Agent instructions are reviewed whenever the conventions they describe change.
- Someone reads the diff before it ships.
The model was the most reliable part of this system. Before asking it to work unattended, spend the effort on the environment, state and boundaries around it.
Mohammed Hamdoune is a Staff Software Engineer specializing in software architecture and AI-assisted delivery. He writes about real-world engineering decisions at sidlynx.me and founded LRJI, a software architecture consulting firm at lrji.co.
