← Back to notebook

The Harness Is Scaffolding. The Model Is the Building.

Anthropic deleted a piece of Claude Code's harness.

It was called the "sprint construct." It decomposed long tasks into chunks so the model could work coherently across them. It was load-bearing. Then Opus 4.6 shipped and it became drag.

4.6 plans more carefully and sustains agentic tasks for longer. The model had absorbed some of the work harness was doing.

This is harness engineering. Every component is a bet on what the current model can't yet do. Every model release deletes another layer. The fastest way to see where agents are going is to look at what the harness is doing today, because that's what the next model will absorb.

At Pokee AI, we built models for on-prem and on-device, where memory, threads, and GPU budget are all capped and the harness has to be lean. The cloud is the inverse: abundant memory, heavier harness, most of the published research. We study cloud harnesses closely because what they do externally is what an on-device model has to absorb internally.

What follows is the map: what a harness is, its 8 components, how one loop iteration works, and how five production harnesses differ in practice.

Agent = Model + Harness

The cleanest one-liner comes from LangChain's Vivek Trivedy: "If you're not the model, you're the harness."

A harness is everything that turns a stateless next-token predictor into something that actually does work. If the LLM is the CPU, the harness is the rest of the computer: memory, I/O, the scheduler, the filesystem, permissions.

The phase shift:

  • 2023: Prompt engineering. What do I tell the model?
  • 2025: Context engineering. What do I show the model?
  • 2026: Harness engineering. What world does the model act in?
  • The arc: each phase exists because the model couldn't yet do that work on its own. Each phase shrinks as the model catches up.

The leaderboard of coding harnesses now includes Codex (OpenAI), Claude Agent SDK (Anthropic), Pi (earendil-works, ~56K stars), Hermes Agent (Nous Research), and OpenCode (160K+ stars). Five harnesses, one underlying capability layer, five very different behaviors. That's the point.

The 8 Components of a Production Harness

Every production agent harness, regardless of framework, decomposes into 8 core subsystems. Here's what they are and what each one actually does.

1. Orchestration and Planning

The outermost while-loop plus the strategy for how work gets decomposed:

Every harness implements some version of this. Claude Agent SDK wraps it in query(), framing the loop as gather context, take action, verify work, then repeat. OpenAI's Agents SDK runs it through a Runner class with turn limits, while Pi simply loops until its tool calls stop.

Two orchestration strategies dominate: ReAct, where the model reasons and acts step by step, and Plan-and-Execute, where it plans first then executes (often in parallel). LLMCompiler is a strong Plan-and-Execute example. It turns multi-tool tasks into an optimized execution plan, allowing independent function calls to run concurrently. This delivered up to 3.7x faster latency and 6.7x lower cost than sequential ReAct. The tradeoff is brittleness: a bad upfront plan can waste downstream work, which is why modern systems keep plans high-level and let execution adapt as the task unfolds.

When a task outgrows a single loop, the harness can spawn subagents. The common shape is orchestrator-worker: a lead decomposes the task, spawns two to ten workers in parallel, and synthesizes their summaries. Anthropic found this beat a single agent by 90.2% on research tasks, though it burns roughly 15x the tokens.

2. Tool Interface

Tools are the functions the model can invoke, each with a name, a description, a parameter schema, and an execution handler.

The critical design question is how many. SWE-agent showed that agent-specific tool design (its Agent-Computer Interface) dramatically outperforms raw system access. Yet Vercel went the other way, cutting 80% of their specialized tools, keeping a single bash tool, and watching success climb from 80% to 100%.

These two reconcile if you start with Unix primitives (read, write, edit, bash, glob, grep) and add a specialized tool only when you can name the exact failure mode it prevents. Anthropic's rule of thumb is to keep three to five tools always loaded. If a human can't tell which tool to use, neither can the agent.

3. Context and Prompt Assembly

A 200K context window isn’t 200K of working memory.

The model can only use what it sees clearly. And as Stanford’s “Lost in the Middle” showed, stuff buried in the middle of a long context is much easier to miss.

So prompt assembly matters. Every run rebuilds the prompt from system instructions, docs, history, tool results, and the latest user message. The important rules need to be placed where the model is most likely to notice them: durable constraints at the top, current task at the bottom.

Near the limit, Claude Code compacts older context into structured summaries, so the gist survives but some detail gets lost.

4. State and Memory

Memory is what persists beyond the current context window, and it's usually organized in three layers:

  • Layer 1: Pointer index (~150 chars per entry), always loaded. Stores locations, not data.
  • Layer 2: Topic files. Fetched on demand, never all in context at once.
  • Layer 3: Raw records. Only accessible via search.

Within a session, progress.md tracks the steps while git tracks the code. More elaborate setups like LangGraph model state as a graph with checkpoints that support branching, retries, and human-in-the-loop. The key insight is that memory isn't a bigger context window. It's a decision about what survives between runs.

5. Output Parsing and Structured Calls

The model returns text. The harness classifies it: tool call, final answer, reasoning trace, or error?

Most frameworks use structured output (function calling), so parsing is deterministic. But you still have to handle the edge cases: malformed tool calls (retry with feedback), multiple calls in one response (run them in parallel or in sequence?), and refusals (detect and surface them). Pi's design is the cleanest here: four tools, explicit structured output, zero ambiguity.

6. Guardrails and Permissions

Probabilistic outputs meet deterministic consequences. A hallucinated DELETE against production costs an outage.

Permissions sit on a spectrum. A permissive setup auto-approves reads and blocks destructive writes, which is fine for local dev. A restrictive one confirms every call, which is what you want in production. OpenAI's Agents SDK formalizes this as input, output, and tool guardrails with tripwires that short-circuit the loop. Claude Agent SDK uses three-phase permissions: ask-once, always-allow, and block.

The non-negotiable part: the harness can kill the loop at any point, and the model never gets to decide its own permission level.

7. Execution Environment

This is where tools actually run. Codex gives each task its own git worktree with full observability (logs, metrics, traces) so the agent can see what its work did. Claude Agent SDK sandboxes bash and runs security checks on every command. Pi is local-first, with no network access unless you grant it.

The environment determines blast radius: in a local sandbox, a bad call corrupts one directory. In a shared environment, it corrupts production. The harness decides which world the agent lives in.

8. Feedback, Error Handling, and Verification

For error handling, feed failures back as context and let the model reason its way to a different approach. Budget two or three retries, then escalate. Never retry the same call forever, or you get a "doom loop."

Verification happens at three levels: automated (tests, linters, type-checkers), visual (Playwright clicking buttons and navigating pages), and LLM-as-judge (grading against a rubric). Each catches different failures.

How a Single Loop Iteration Actually Works

Let's trace one complete cycle through a production agent loop. This is what happens between "user sends a message" and "agent takes one action":

Step 1: Prompt Assembly. The harness gathers: system prompt + project docs (AGENTS.md) + conversation history (compacted if needed) + any new tool results from the previous iteration + the user's latest message. It checks total token count. If over threshold, it fires compaction first.

Step 2: LLM Inference. The assembled prompt goes to the model. The model returns either a text response (final answer), one or more tool calls, or a mix of both.

Step 3: Output Classification. The harness parses the response. Is it done? (Text only, no tool calls = done.) Does it want to act? (Tool calls present = not done, continue loop.) Is it malformed? (Retry with error feedback.)

Step 4: Tool Execution. For each tool call, the harness validates parameters against the schema, checks permissions (ask user if needed), executes the tool, and captures output and any errors.

Step 5: Result Packaging. Tool outputs get formatted as context for the next iteration. Large outputs may get truncated or summarized. Errors get structured with actionable information.

Step 6: Context Update. New tool results get appended to conversation history. If context is now over the compaction threshold, fire a compression pass. Update any external state (progress files, git commits).

Step 7: Loop Decision. Was there a tool call? If yes, go back to Step 1. If no (model returned only text), the loop terminates and the response is delivered.

That's it. Every agent you've ever used runs some version of this loop. The difference between a toy demo and a production system is how robust each step is.

Co-Evolution: The Harness Gets Thinner

The sprint construct is one deletion in a longer pattern. As models improve, harnesses shrink.

The decision framework: for any harness component, ask "If the model were perfect at this, would I still need this code?" If yes (permissions, audit logs, tool execution), it's permanent infrastructure. If no (context anxiety compensation, output format correction, retry loops for refusals), it's scaffolding that should be deleted the moment the model outgrows it.

The harness gets thinner. It never reaches zero. There will always be permissions, observability, tool execution, and human-in-the-loop checkpoints. The model can get infinitely smart, but it still needs an interface to the world.

The on-device case is where this future already lives. When memory, threads, and GPU budget are all capped, capability has to live in the model. That's the bet we're making at Pokee AI: study the cloud harness closely, then push as much of its work as possible back into the model itself.

Sources: Anthropic "Build Agents That Run for Hours" (AI Engineer, May 2026), "Is Grep All You Need?" (arXiv:2605.15184), LLMCompiler (ICML 2024), ReAct (ICLR 2023), "LLMs Cannot Self-Correct Reasoning Yet" (ICLR 2024), Anthropic "Building Effective Agents" (Dec 2024), ACON research, SWE-agent ACI (NeurIPS 2024), Pi Agent, Hermes Agent (Nous Research), GEPA (ICLR 2026), OpenCode, Vercel tool reduction experiment, Stanford "Lost in the Middle," Chroma context rot study, Can Boluk edit-tool format study.

No comments yet