Back to Home
AI Development

The Coding Agent Harness: Hooks, Verification Gates and Allowlists You Set Up Once

One verification command, hooks that enforce it, typecheck and lint failures routed back to the agent, and permission allowlists so it stops asking you.

13Labs Team13 August 20269 min read
AI coding agentsClaude Code hooksverificationdeveloper toolingagent supervision

Contents

What is an agent harness, and what does it actually stop?

An agent harness is the four pieces of setup that turn a run you supervise into a run you check afterwards: one verification command the agent must run and pass, lifecycle hooks that fire it whether or not the agent remembers, a permission allowlist so safe commands stop prompting you, and environment variables and secrets configured once so the agent stops asking. The reason it matters is stated plainly in Anthropic's own Claude Code documentation: "Claude stops when the work looks done. Without a check it can run, 'looks done' is the only signal available, and you become the verification loop: every mistake waits for you to notice it" (Claude Code documentation, code.claude.com/docs/en/best-practices, retrieved 13 August 2026). You are not a bottleneck because the model is bad. You are a bottleneck because nothing else in the loop can say pass or fail. This is a live problem for builders, not a theoretical one. Four registrants for 13Labs buildDay sessions named it independently. One, G-Den, described the blocker as "just having to babysit codex, I think I need to come up with some hooks and stored environmental variables." Another, Oskar, wrote that "harness engineering i find difficult to do agentically." Khalid is building what he calls a "Bot Harness" and is stuck on "Maintaining project with Bot." A fourth, Sumyat, wants to learn "how others work with AI in terms of hardness, verification, quality, observation" (the word is almost certainly a typo for harness, quoted as written). Everything below is single-agent setup, done once per repository. Isolating several agents running at the same time is a different job, covered in our guide on running parallel agents without conflicts.

What is the one command the agent must run and pass?

Define a single script that runs typecheck, lint and test in sequence and exits non-zero on the first failure, then make that the only check anyone mentions. In a Node project that is one line in package.json: - `"verify": "tsc --noEmit && eslint . && vitest run"` Collapse three commands into one because an agent given three optional checks will run one of them. Anthropic's documentation describes the check as "anything that returns a signal Claude can read in the conversation: a test suite, a build exit code, a linter, a script that diffs output against a fixture, or a browser screenshot compared against a design" (Claude Code documentation, code.claude.com/docs/en/best-practices, retrieved 13 August 2026). Simon Willison chose his tooling on exactly this property. "This run-the-code-in-a-loop pattern is so powerful that I chose my core LLM tools for coding based primarily on whether they can safely run and iterate on my code." - Simon Willison, "Here's how I use LLMs to help me write code", simonwillison.net, 11 March 2025. Two practical constraints. Keep the command under about 90 seconds, because a slow gate gets skipped by humans and burns context for agents. And make its output short: the agent has to read the failure inside its context window, so a test runner that prints 400 lines of passing output before the one failure is actively working against you. Armin Ronacher put the general rule as "Tools need to be fast. The quicker they respond (and the less useless output they produce) the better" ("Agentic Coding Recommendations", lucumr.pocoo.org, 12 June 2025).

How do hooks turn a rule the agent can ignore into one it cannot?

Hooks are shell commands Claude Code runs itself at fixed points in the session, so they execute regardless of what the model decides. Anthropic draws the distinction directly: "Unlike CLAUDE.md instructions which are advisory, hooks are deterministic and guarantee the action happens" (Claude Code documentation, code.claude.com/docs/en/best-practices, retrieved 13 August 2026). The same page's advice on when to reach for them is one line: "Use hooks for actions that must happen every time with zero exceptions." As of 13 August 2026 the hooks reference lists 31 events, including `SessionStart`, `UserPromptSubmit`, `PreToolUse`, `PermissionRequest`, `PostToolUse`, `PostToolUseFailure`, `PostToolBatch`, `SubagentStop`, `PreCompact`, `Stop` and `SessionEnd` (code.claude.com/docs/en/hooks). Three of those carry most of the weight for a verification harness. - `PostToolUse` with matcher `Edit|Write` runs your formatter or typechecker every time a file changes. - `Stop` runs your verification command and blocks the turn from ending until it passes. - `PreToolUse` with matcher `Bash` blocks a command before it runs. The `Stop` hook is the one that changes how a run feels. Anthropic describes it as a deterministic gate where "a Stop hook runs your check as a script and blocks the turn from ending until it passes," with one important limit: "Claude Code overrides the hook and ends the turn after 8 consecutive blocks." That cap is the harness admitting it cannot fix everything, and it is deliberate. Hooks live in `settings.json` under a `hooks` key, at `~/.claude/settings.json` for you, `.claude/settings.json` for the repository (check this one in), and `.claude/settings.local.json` for machine-specific overrides. Matchers are exact strings, pipe-separated lists like `Edit|Write`, or unanchored JavaScript regular expressions such as `mcp__memory__.*`.

How do you make a failure go back to the agent instead of back to you?

Exit code 2 is the mechanism. In Claude Code, a hook exiting 2 is a blocking error, and what it blocks depends on the event, which is why the event you pick matters more than the script you write. | Hook event | Exit 2 blocks? | What happens | | --- | --- | --- | | `PreToolUse` | Yes | Blocks the tool call before it runs | | `PostToolUse` | No | Tool already ran; stderr is shown to Claude | | `Stop` | Yes | Prevents Claude stopping, continues the conversation | | `SubagentStop` | Yes | Prevents the subagent stopping | | `PostToolBatch` | Yes | Stops the agentic loop before the next model call | | `Notification` | No | Exit code and stderr are ignored | Source: Claude Code hooks reference, code.claude.com/docs/en/hooks, retrieved 13 August 2026. The practical shape is this. Your `Stop` hook runs `npm run verify`. If it passes, exit 0 and the turn ends. If it fails, print the failing output on stderr and exit 2, and the agent keeps working with the error text in front of it. Nothing reaches you until either it passes or the eight-block cap trips. For finer control, hooks can write JSON to stdout instead of relying on exit codes. The `hookSpecificOutput` object carries `hookEventName`, a `permissionDecision` of `allow`, `deny` or `escalate`, a `permissionDecisionReason`, and `additionalContext`, which injects text the agent reads. Anthropic's documentation notes that when valid JSON is returned for events using the standard decision model, "Claude Code ignores the exit code and the JSON alone decides the outcome." Aider shipped the same idea without hooks. Its documentation states that with `--auto-test` and `--test-cmd` set, "Aider will try and fix any errors if the command returns a non-zero exit code" (aider.chat/docs/usage/lint-test.html, retrieved 13 August 2026). Different plumbing, identical contract: non-zero exit plus readable output equals self-correction.

How do you stop the agent prompting you on commands you already trust?

Write the safe commands into `permissions.allow` in `.claude/settings.json` and commit it, so every developer on the repository gets the same silence. Anthropic is blunt about why prompt fatigue is a real failure mode rather than an annoyance: "After the tenth approval you're not really reviewing anymore, you're just clicking through" (Claude Code documentation, code.claude.com/docs/en/best-practices, retrieved 13 August 2026). A starting allowlist for a typical web project: - `"Bash(npm run verify)"` - `"Bash(npm run test *)"` - `"Bash(git status)"` and `"Bash(git diff *)"` - `"WebFetch(domain:docs.claude.com)"` Three matching rules are worth knowing before you write your own, all from the permissions reference at code.claude.com/docs/en/permissions (retrieved 13 August 2026). First, a trailing space before the wildcard enforces a word boundary: `Bash(ls *)` matches `ls -la` but not `lsof`, while `Bash(ls*)` matches both. Second, compound commands are decomposed, not string-matched: "Claude Code is aware of shell operators, so a rule like `Bash(safe-cmd *)` won't give it permission to run the command `safe-cmd && other-cmd`." Every subcommand must match its own rule. Third, environment runners are the trap. `direnv exec`, `devbox run`, `npx` and `docker exec` are not stripped like `timeout` or `nice` are, so `Bash(devbox run *)` would approve `devbox run rm -rf .`. Write the runner and the inner command together. The wider lever is the sandbox. Claude Code's sandboxed Bash tool uses Seatbelt on macOS and bubblewrap on Linux and WSL2, and in auto-allow mode it "runs it inside the sandbox and approves it automatically, without asking your permission," with explicit deny rules still honoured and root or home-directory deletions still prompting. That is a real reduction in interruptions, because the operating system is enforcing the boundary rather than a string match.

How do you set up environment and secrets once so it stops asking?

Put non-secret configuration in the `env` block of `.claude/settings.json` so it is present in every session, and keep real credentials out of the agent's reach entirely. G-Den's phrase "stored environmental variables" is exactly the right instinct: most repeated questions from an agent are a missing variable, not a missing instruction. For rotating credentials, `apiKeyHelper` runs a script that returns a key. Claude Code calls it after 5 minutes by default or on an HTTP 401, tunable with `CLAUDE_CODE_API_KEY_HELPER_TTL_MS` (code.claude.com/docs/en/iam, retrieved 13 August 2026). That is the clean path to a vault or 1Password without pasting a long-lived key into a file. For secrets the agent should never read, use deny rules and sandbox credential settings rather than trusting the model to look away: - `"deny": ["Read(./.env)", "Read(./.env.*)", "Read(./secrets/**)"]` - Sandbox `credentials.files` entries such as `{ "path": "~/.aws/credentials", "mode": "deny" }` - Sandbox `credentials.envVars` entries such as `{ "name": "GITHUB_TOKEN", "mode": "deny" }` One caveat from the sandboxing docs is easy to miss: "There is no built-in credential deny list, so only the files and variables you list are restricted." Nothing is protected by default. A `mask` mode on Linux and WSL2 goes further, showing sandboxed commands a placeholder while the sandbox proxy substitutes the real value on outbound requests to allowed hosts, so `gh` and `npm` keep working without the agent ever seeing the token. The other half of this is making the development environment observable to the agent. Armin Ronacher's example is the sharpest one published: "In debug mode (which the agent runs in), the email is just logged to stdout. This is crucial! It allows the agent to complete a full sign-in" ("Agentic Coding Recommendations", lucumr.pocoo.org, 12 June 2025). An agent that can read the sign-in link out of a log does not need to ask you for it.

What is the equivalent setup in Codex, Cursor and Aider?

Every serious coding agent shipped some version of this in 2025 and 2026, with different names for the same four jobs. If you are not on Claude Code, the table below is where to look. | Tool | Enforcement layer | Permission control | Verification loop | | --- | --- | --- | --- | | Claude Code | Hooks in `settings.json`, 31 events, exit 2 blocks | `permissions.allow` / `deny` / `ask`, `defaultMode`, OS sandbox | `Stop` hook running one command | | Cursor | `hooks.json` at `.cursor/hooks.json`, events including `beforeShellExecution`, `afterFileEdit`, `stop` | Hook response `permission` field of `allow`, `deny` or `ask` | `afterFileEdit` plus `stop` hooks | | Codex CLI | `[hooks]` in `config.toml`, behind a feature flag | `approval_policy` of `untrusted`, `on-request` or `never`; `sandbox_mode` of `read-only`, `workspace-write` or `danger-full-access` | AGENTS.md instructions plus sandboxed shell | | Aider | `--auto-lint` (on by default) and `--auto-test` | Repository map and confirmation prompts | `--lint-cmd` and `--test-cmd`, errors fed back on non-zero exit | Sources: code.claude.com/docs/en/hooks; cursor.com/docs/agent/hooks; learn.chatgpt.com/docs/config-file/config-basic; aider.chat/docs/usage/lint-test.html, all retrieved 13 August 2026. Cursor's blocking contract is close enough to Claude Code's to port a script across with almost no changes: "Exit code 2 - Block the action (equivalent to returning permission: "deny")". Codex takes the coarser route, leaning on `sandbox_mode` and `approval_policy` rather than per-event hooks, which means less granularity but less to configure. One warning that applies to all of them. Hooks run arbitrary shell commands with your credentials and your filesystem access, at machine speed, without confirmation. Read any hook you copy from a blog post or a plugin before you enable it.

Where should the harness still stop and fetch you?

Keep four categories on `ask` or `deny` permanently, no matter how good the rest of the harness gets: anything that writes to production, anything that destroys data, anything that spends money, and anything that touches credentials. A green verification run says the code compiles, lints and passes the tests you wrote. It says nothing about whether the migration is reversible or whether the feature is the one the customer asked for. The concrete rules: - `"ask": ["Bash(git push *)", "Bash(npm publish *)", "Bash(*prisma migrate deploy*)", "Bash(vercel --prod*)"]` - `"deny": ["Bash(rm -rf *)", "Read(./.env)", "Bash(aws *)"]` Content-scoped ask rules such as `Bash(git push *)` still force a prompt even when a command would otherwise run sandboxed and auto-approved, so this survives turning the sandbox on. Root and home-directory removals such as `rm -rf /` still prompt even in `bypassPermissions` mode, as a circuit breaker. The evidence for keeping a human on those calls is not sentimental. In Stack Overflow's 2025 Developer Survey, 66 per cent of developers named "AI solutions that are almost right, but not quite" as a top frustration, 45.2 per cent said debugging AI-generated code is more time-consuming, and only 3.1 per cent said they highly trust the accuracy of AI output. Google Cloud's 2025 DORA report, published 24 September 2025, found 90 per cent of respondents using AI at work, 30 per cent reporting little or no trust in AI-generated code, and a continuing negative relationship between AI adoption and delivery stability. METR's randomised controlled trial, published 10 July 2025, found 16 experienced open-source developers completing 246 issues took 19 per cent longer when allowed to use AI tools. Those numbers argue for the harness, not against it. Automated gates take the mechanical part of verification off you so the attention you have left goes to the calls only you can make. Deciding what to check in the moment, when no automated gate exists, is a separate skill covered in our guide on what to do when the AI says it works. "Most people try to fix a bad agent run by writing a longer prompt. The fix is almost always a gate, not a paragraph. If the rule matters, make the machine enforce it, and if the machine can't enforce it, that is exactly the decision you should still be making yourself." - Callum Holt, Founder, 13Labs

Frequently asked questions

Do hooks slow the agent down? Yes, and that is the trade. A `PostToolUse` hook running a formatter adds a second or two per edit. A `Stop` hook running a full verification command adds however long that command takes to every turn end. Keep the verification command under about 90 seconds, and scope file-level hooks with matchers like `Edit|Write` rather than firing on every tool. What is the difference between a hook and an instruction in CLAUDE.md? Enforcement. Anthropic's documentation states that "Unlike CLAUDE.md instructions which are advisory, hooks are deterministic and guarantee the action happens." An instruction file competes for the model's attention against everything else in context. A hook is executed by the tool itself and cannot be argued with. Can a hook block the agent forever if my verification command is broken? No. Claude Code overrides a blocking `Stop` hook and ends the turn after 8 consecutive blocks, so a permanently failing check costs you eight wasted turns rather than an infinite loop. Test the command manually before you wire it into a hook. Is a permission allowlist a security boundary? Not on its own. Rules match command strings, and Claude Code decomposes compound commands so each subcommand needs its own rule, but environment runners like `devbox run` and `docker exec` are not unwrapped, so a broad rule can approve more than you intended. For a real boundary, use the OS-level sandbox or a container. Where do I put the config so my team gets it too? Commit `.claude/settings.json` at the repository root for shared hooks and permissions, keep personal overrides in `.claude/settings.local.json` and gitignore it, and leave `~/.claude/settings.json` for preferences that follow you across projects. Managed settings deployed by an administrator override all three.

Build the harness once, then stop watching every run

buildAcademy teaches builders the verification and enforcement setup that makes a coding agent trustworthy without supervision, the same habits we run in every buildDay session.

See buildAcademy