Harness engineering

Knowledge base · Dwayne Donaldson · September 22, 2026 · 7 min read

Agentic engineering · part 1 of 10 · Next: Loop engineering

Everyone benchmarks models. Almost nobody benchmarks the thing around the model, which is where most of the variance lives. Two teams hand the same weights the same task. One gets a demo that impresses a room for four minutes. The other gets a system that runs unattended for six hours and files a pull request.

Here is a complete agent:

while True:
    reply = model(messages)
    if not reply.tool_calls:
        break
    messages += [reply, run(reply.tool_calls)]

Five lines. It genuinely works. Hand it a calculator tool and a maths question and it will answer, and you will feel briefly like a wizard. Hand it a shell tool and a repository and it will delete something you needed.

Everything between those five lines and a system you would let near a production repo is the harness, and the harness is where the discipline lives.

What is actually in there

Start from the five lines and ask what each one quietly assumes.

model(messages) assumes somebody decided what is in messages. That is a system prompt, a tool catalogue, a conversation history, retrieved documents, memory, and some policy for what to throw away when they stop fitting. Nothing in the loop tells you how to pick.

run(reply.tool_calls) assumes the call is allowed. Nothing checks. It also assumes the tool returns something the model can read, that a 40 MB file does not arrive in one piece, and that a failure comes back as a message rather than an exception that kills the process.

while True assumes something eventually breaks the loop. if not reply.tool_calls is that something, and it is a terrible judge. The model stops calling tools when it believes it is finished, which is not the same event as being finished.

And the whole thing assumes one process, one machine, one uninterrupted run. Kill it at iteration 30 and everything is gone.

Nobody sits down to write a runtime. It accretes, one incident at a time, which is why most of them are undocumented and live in one person’s head.

Fill in those gaps honestly and you have written a context assembler, a tool registry with schemas and error contracts, a permission layer, a compaction strategy, a persistence format, a termination policy, a retry and replan policy, a trace log, and some way of resuming. That is a runtime.

The part that decides whether your work survives

There are two kinds of harness code and they age in opposite directions.

The first kind compensates for something the model is bad at. Reminders not to forget the file path. A regex that repairs malformed JSON. A retry that rephrases the prompt when the model refuses for no reason. Three paragraphs explaining that yes, it really does have permission to read that directory. This code is real work and it is load-bearing right now. It is also on a timer. The next model release deletes it, and if you built your product on it you spend that release day pulling out scaffolding instead of shipping.

The second kind gives the model something it cannot have alone. A filesystem. A test suite it can run and read the output of. State that outlives the context window. Permission to act, bounded by a policy. Six hours of wall clock. A colleague process to delegate to. No model release makes these unnecessary, because they are not weaknesses being patched, they are capabilities being granted.

The practical test when you are about to add something to the harness: if the model got twice as good tomorrow, would this code still be here? If the honest answer is no, write it anyway, but write it somewhere you can find it again, in one file, with a comment saying which model behaviour it exists for. The teams that do this upgrade in an afternoon. The teams that do not have compensation tangled through forty files and never quite dare.

Determinism is a budget too

Every decision in the system is made either by code you wrote or by a sample from a probability distribution. The second kind is expensive, slow, and different every time. The first kind is free, instant and testable.

So a good harness is constantly asking whether a decision really needs the model. Which tool to call, probably yes. Whether the output of that tool is valid JSON, absolutely not, that is a parser. Whether the tests passed, that is an exit code, and asking a model to read the test output and tell you whether it passed is paying tokens for a worse ==.

This sounds obvious written down. It is not obvious at three in the afternoon when the fastest way to make the demo work is one more instruction in the system prompt. Prompt text is the cheapest thing in the world to add and the most expensive thing to own. It never shows up in code review, it has no tests, it degrades silently, and six months later nobody can tell you which sentence is still doing something. The system prompts I would least like to inherit are not badly written. They are archaeological.

The tax nobody puts on the invoice

Every harness feature costs tokens before the user types anything. Tool schemas, the skill catalogue, the permission rules, the memory index, the formatting instructions. On a long-running agent that fixed cost is paid on every single model call, forty or four hundred times in a run.

Which produces the actual design constraint of the field: the harness must be rich enough to be useful and small enough to disappear. Most of what is good in the last two years of agent design is a different answer to that one question. Progressive disclosure, where a capability announces itself in one line and loads its full instructions only when invoked. Sub-agents, which do expensive work in a context window you then throw away, keeping only the summary. Compaction. Retrieval at the point of need instead of pre-loading. They look like unrelated features. They are all the same trick, which is to keep the catalogue and pay for the content only on use.

Why this is engineering and not prompting

Because it has the properties. Failure modes you can enumerate, tradeoffs you can name, components you can test in isolation, a cost model, an upgrade path, and incidents.

The reason it took a while to be treated that way is that the first version of everyone’s harness is five lines, works immediately, and feels like it is mostly the model doing the work. It is, right up until the thing has to run unattended, and then you discover that you have been maintaining a runtime for months without calling it one. The harness is the product. The model is a dependency you do not control, and the job is to build something that gets better when it improves rather than something that was hiding its flaws.