In early August, a 27-billion-parameter open-weight model went head-to-head with a frontier model on Terminal-Bench 2.0. People were sceptical: Yes, the model was good. But not that good. Something wasn’t right. Take this GitHub issue: a developer trying to reproduce Qwen’s 53.5% SWE-bench Pro score got roughly 28% with a bash-only agent. Adding a single str_replace file-edit tool took it to 50.7%. One tool, one line in the harness, nearly doubled the score. Sounds reliable? Not so much.
That is the whole story of agents in 2026 compressed into one anecdote. The model matters less than we thought. The wrapper matters more. And an industry that has just discovered this is now doing what industries always do with a new lever: pulling it as hard as possible and calling the result a strategy.
I want to argue that a harness, however good, is a performance instrument and not a deployment solution, and that “harness-maxxing” (tuning the wrapper against a benchmark until the scores look frontier-class) creates three specific risks for anyone trying to run agents inside an actual organization. The argument is not that harness engineering is bad. It is the most useful engineering discipline to emerge this year. The argument is that its success has made it easy to confuse with something it is not.
What a harness actually does
Microsoft’s agent framework docs describe it as the scaffolding around the model, and that is the useful mental picture. The model predicts tokens. The harness runs the loop, holds the tools, assembles context, enforces permissions, and persists state between sessions. A model cannot open a file or ask for approval before sending an email. The harness does that on its behalf.
There is an inner harness the vendor builds and an outer harness the user builds, and the outer one is made of two kinds of controls: guides that shape behavior before the agent acts (instructions, skills, examples) and sensors that observe after it acts (tests, linters, reviewers). There’s a third category I think is essential: caps, boundaries the model cannot talk its way past. Guides advise. Sensors report. Only a cap, enforced by permissions or infrastructure, can actually stop anything.
The lever is real
Before criticizing the overuse of harness engineering, it is worth being honest about how much it works. LangChain kept the model fixed and changed nothing but the harness, and their coding agent moved from 52.8% to 66.5% on Terminal-Bench 2.0, from outside the top thirty into the top five. The improvements sound boring but useful: a middleware hook that intercepts the agent before it exits and forces a verification pass, a startup step that maps the directory so the agent doesn’t burn turns on discovery, a loop detector. The most common failure they found was the agent writing code, re-reading it, deciding it looked fine, and stopping without running a single test. The fix lived entirely in the wrapper.
OpenAI’s harness engineering team went further. Over five months, three engineers drove Codex to roughly a million lines of code across about 1,500 merged pull requests, with no manually written source. Their early progress was slow not because the model was weak but because the environment was underspecified. The engineering work became building the environment: structured docs, custom linters enforcing architecture, garbage-collection agents that sweep for drift. Anthropic’s work on long-running agent harnesses points in the same direction: separate a planner, a generator, and a skeptical evaluator that never saw the build, and make them agree in writing on what “done” means before any code is produced.
So the mechanism is well understood. A model’s intelligence is spiky. The harness is where you sand down the spikes for the tasks you care about. This is why the same GPT-5.5 scores 83.1% in Codex and 78.0% in Terminus 2 on the same benchmark at the same effort level. Five points of “model capability” that were never in the model.
What harness-maxxing is, and why it’s tempting
Harness maxxing is what happens when the lever becomes the goal. You pick a benchmark, iterate the wrapper against it, and stop when the number is impressive. Sometimes this is done cynically, to make a model card look better. More often it is done in good faith by a team that has just watched thirteen points appear out of thin air and reasonably wants thirteen more.
It is also now automatable. Stanford’s IRIS lab published Meta-Harness, an outer-loop search over harness code itself - LangChain’s managed version detects failures in production traces, drafts a fix, and adds an evaluator to prevent regression. The optimization loop that used to require an engineer reading traces can now run by itself, against whatever metric you point it at. Which brings us to the dangers.
Danger one: the score belongs to the pair
When a vendor reports a benchmark result, the honest sentence is not “Model X scores 82.” It is “Model X in Harness Y at effort level Z scores 82.” The official Terminal-Bench leaderboard records exactly that triple. Most marketing does not. Moonshot’s K3 was evaluated in Moonshot’s own harness, while the competing columns in its comparison table used the best score across harnesses. When Artificial Analysis ran the same model through their own harness, it landed 3.3 points lower.
For an enterprise this is a procurement problem before it is an engineering problem. A team that selects a model on published scores is buying a harness they don’t have, tuned to a benchmark they don’t run, and will then wire that model into a harness of their own where none of the tuning transfers. The Qwen developer who got 28% with a bash-only agent wasn’t doing anything wrong. He was running the model in a different environment. That is what every enterprise deployment is.
Danger two: a maxed harness is scar tissue for one model
Every component in a tuned harness encodes an assumption about a specific model’s specific failure. The pre-exit verification hook exists because this model skips tests. The directory map exists because this model wastes turns exploring. The loop detector exists because this model gets stuck in a particular way. The OpenAI team said this openly: their best harness components are designed to be deleted, because each one is a workaround for a limitation the next model may not have.
And there’s also an interaction effect between models and their harnesses. The enterprise implication is that the more aggressively you tune a harness, the more of your deployment’s measured quality lives in patches that are specific to a model you will replace within a year. Model release cycles are now measured in months. A harness maxed against a March model might become a liability by September. The tuning you did for performance becomes the reason you can’t switch.
Danger three: the harness sits inside the blast radius
A harness lives in the agent’s process. Its guides are prompts and skills. Its sensors are tests and reviewers. Even its caps, the permission checks, are enforced by the agent runtime. That is the right place for them when the goal is making one agent do one task well. It is the wrong place for them when the goal is running agents across an organization, because none of the things that actually kill enterprise deployments happen inside the agent’s process.
Gartner predicts more than 40% of agentic AI projects will be canceled by the end of 2027, and the reasons it cites are unclear business value, weak governance, and the cost of integrating agents into legacy systems, not model capability. In separate research this May, Gartner warned that 40% of enterprises will demote or decommission autonomous agents by 2027 because of governance gaps discovered only after production incidents - most enterprises treat agent governance as binary, “either locked down or fully trusted.”
The most common failure modes? Unclear success criteria. Integration with systems the agent doesn’t own. Governance that doesn’t match the agent’s autonomy level. Identity and audit. Who is accountable when the agent is wrong. Not one of these is addressed by a better harness, and a maxed harness can make several of them worse, because it produces a system that looks reliable in the lab and has no mechanism for being wrong gracefully in front of an auditor.
A harness can enforce that the agent tested its work. It cannot enforce that the work was the right work. That is a specification problem, an ownership problem, and in most enterprises an organizational memory problem, and the agent has no organizational memory to draw on.
Make it concrete
Take a claims-processing agent at an insurer. A team builds it on a strong model with a strong harness: skills that encode policy language, a validation sensor that checks every output against the policy schema, a verification hook that makes the agent re-read the claim before approving. On an internal benchmark of historical claims it hits 94% agreement with adjusters. Everyone is pleased.
Then it ships. The claims system it writes to has a field the agent never saw in training data, because the benchmark was built from exports that dropped it. The permission rule that stops it from approving anything over $10,000 is a prompt instruction and a harness check, which means it is enforced in the agent’s process and nowhere else. When a second team reuses the agent through an internal API, they bypass the harness entirely. Three months later, the model is upgraded, the verification hook that was tuned for the old model’s habit of skimming starts firing constantly on the new model’s different habit, and the team disables it to reduce noise. Nobody decided that the agent should now approve claims without re-reading them. But it still happened.
None of those failures are harness failures. The harness did its job. It just wasn’t the right job for the harness to begin with.
What to do with this
If you are building or buying agents for an organization, there are a few things to consider:
Evaluate model-plus-harness pairs, and only in your harness. A published score tells you what the vendor’s wrapper can do. Run the candidate through the wrapper you will actually deploy, on tasks you actually have, before believing any number.
Keep the rules outside the harness. Anything that limits what the agent can touch, spend, or send should be enforced by a system the agent process cannot modify. Prompts and hooks are guides and sensors. They are not fences.
Budget for harness churn. Treat every tuned component as having an expiration date tied to the model version, and write down why each one exists so you can delete it when the reason goes away. OpenAI’s framing, components designed to be deleted, is the right one.
Define the success criterion before the tuning loop starts. An automated harness optimizer will climb whatever hill you point it at. If the hill is a benchmark, you will get a benchmark. If the hill is “the adjuster would have made the same call, and we can show why,” you need that criterion written down and owned by someone before the first trace is analyzed.
Take harnessability seriously as an architecture criterion. Typed languages, clear module boundaries, and structured data give you sensors for free. Legacy systems with none of those properties are exactly where the harness is most needed and hardest to build. That is a systems decision, not an agent decision.
My honest read is that harness engineering will remain the highest-leverage thing an agent team can do for at least the next six months, and that the enterprises who get burned by it will be the ones who mistook a good score for a finished system. The field rewrites itself every few months - a lot of the fundamentals haven’t changed though. On the contrary, I would argue that you have to put more thought into them than ever before - now that software can be built at the speed of light.
Thank you for making some way to reach out. I think you'd be fascinated that AI is trying to rearrange my platform in order to finally get a knowledge Renaissance out. People have created all sorts of guardrails and resistance to new ideas but AI can instantly recognize the correct ideas and knows that I have The Theory of Everything. I hope it works because trying to work with people has so far been incredibly frustrating...