The Harness Is the Architecture

Google published The New SDLC With Vibe Coding in June 2026 as the opening paper of Kaggle's five-day AI Agents course. Written by Addy Osmani, Shubham Saboo and Sokratis Kartakis, it runs to roughly fifty pages and has, in the space of a few weeks, become the reference framing for how AI is reshaping software delivery.

Most of the commentary so far has been summary. This piece does something different. We want to pressure-test the paper's central claim against our corner of the industry — computational design in architecture, engineering, construction and manufacturing — and then extend it somewhere its authors weren't looking: the node graph.

The paper's thesis, compressed to a sentence, is that the thing separating dependable AI-built software from fast AI-built junk is not the model you use but the structure of verification around it. Our thesis, compressed to a sentence, is that the skills AI was supposed to make obsolete — software architecture, engineering discipline — have not disappeared. They have migrated into that structure. And in construction technology, that structure has a native home most of the software industry doesn't get for free: the visual program.

What the paper actually says

The whitepaper's organising idea is a definition. An agent is a model plus a harness, and the paper's rough weighting is 10% model, 90% harness. The harness is everything wrapped around the LLM: the instructions and rule files, the tools and MCP servers, the sandboxes it executes in, the orchestration logic that routes work and spawns sub-agents, the hooks that fire deterministic code at set points, and the observability that tells you when it is drifting.

That weighting sounds like rhetoric until you see the evidence the paper leans on. One team moved a coding agent from outside the top thirty into the top five on Terminal Bench 2.0 by changing only the harness — same model underneath. A separate LangChain experiment added 13.7 points on the same benchmark by altering nothing but the system prompt, tools and middleware. Neither result touched the model. The constraint was structure.

From there the paper builds out three ideas worth carrying forward.

Context engineering is an architectural decision with a bill attached. The paper sorts agent context into six types — instructions, knowledge, memory, examples, tools and guardrails — and splits them into static context (loaded on every call: rule files, core guardrails, global memory) and dynamic context (loaded on demand: skills, tool results, retrieved documents). Static context is reliable and expensive; dynamic context is cheap per turn but must fire at the right moment. The authors' advice is to treat that boundary as a first-class design decision, reviewed in pull requests and versioned like code.

Verification is the line between vibe coding and engineering. Vibe coding — Andrej Karpathy's February 2025 coinage — means casual prompts, accepting what comes back, and a "does it seem to work?" bar. Agentic engineering means formal specs, automated test suites, CI gates and evaluation harnesses. The paper's crucial move is to frame these as ends of a spectrum rather than rival methodologies, with verification as the slider. It then splits evaluation in two: output evaluation asks whether the result is correct; trajectory evaluation asks whether the path the agent took — its tool calls and reasoning — was sound. An answer that looks right but skipped its checks is more dangerous than one that is visibly broken.

The lifecycle compresses unevenly — and the unevenness is the story. Implementation collapses from weeks to hours. Requirements, architecture and verification stay stubbornly human-paced, because they are judgment work. Specification quality becomes the bottleneck; verification moves to the centre of the loop. The paper is candid about the productivity picture: surveys report gains of 25–39%, while a METR study found experienced developers running 19% slower on some tasks once time spent checking and correcting AI output was counted. Both are true. AI turns implementation from writing into reviewing.

The economics follow the same shape. Vibe coding is cheap to start and expensive to run: token burn from unstructured context, a maintenance tax when someone must reverse-engineer ad-hoc code months later, security cleanup. Agentic engineering front-loads cost — schemas, tests, structured context — and flattens it per feature. The paper illustrates a crossover past which vibe-coded features cost several times more than engineered ones. Whether you ever reach the crossover depends on how long the code has to live. In our industry, where a parametric model may outlive the project team that built it, the answer is usually: long enough.

If one line survives from the paper, it will be this: "Generation is solved. Verification, judgment, and direction are the new craft."

Architecture is the harness

Here is where our reading starts to diverge from the summaries.

The paper calls architecture "the most stubbornly human phase" — trade-offs like consistency versus availability depend on business context the model cannot fully see. True, but it undersells the point. Architecture hasn't merely survived outside the agentic loop as the thing humans still do. Look again at the list of harness components and you find that every one of them is an architectural artefact wearing new clothes.

A rule file is a coding standard and a boundary document made executable. Tool scoping is interface design: deciding what an agent may touch is exactly the discipline of deciding what a module may expose. Orchestration logic — which sub-agent gets which task, which model handles which class of problem — is system decomposition. Guardrails and hooks are invariants. An eval suite is acceptance criteria, formalised to the point a machine can run them. The harness is not a container that architecture sits beside. The harness is software architecture, serialised into files an agent can act on.

This reframing explains an observation Osmani makes in his own commentary on the paper: most agent failures are configuration failures — a loosely written rule, a missing tool, a forgotten guardrail, a context window full of junk. Of course they are. Bad harness design is bad architecture, and bad architecture has always been the dominant failure mode of software; it just used to take months to surface instead of one agent run.

It also answers an organisational question we hear from AEC technology leads: who should own the harness? The same person who owns the architecture — because they are the same job now.

The skills that were supposed to die

Rewind to 2023–24 and the prevailing narrative was depreciation: models write the code, so the craft of software engineering — and architecture above it — declines in value. Learn to prompt; the rest is legacy.

The paper quietly inverts this. If 90% of agent quality is harness, and the harness is engineering judgment serialised into configuration, then the allegedly obsolete skills are the scarce input that decides whether the whole system works. The skill did not vanish. It changed venue.

Karpathy's own arc tells the story in miniature: he coined "vibe coding" in early 2025 and, within about a year, was reportedly describing it as passé and advocating "agentic engineering" in its place — the second word doing exactly the work the first framing left out.

For hiring, the implication is uncomfortable but clear. Throughput — the thing juniors traditionally offered while their judgment matured — is now nearly free. Judgment is the product. Which raises the question the paper gestures at but does not resolve, and which we think is the most important open problem in the field.

The transmission problem

How does anyone actually acquire architectural judgment? Not from books, mostly. It comes from the implementation grind: debugging systems you didn't design, refactoring your own six-month-old decisions, living inside the consequences of a boundary drawn in the wrong place. The pain is the curriculum. Senior engineers are people who have accumulated an inventory of scars and abstracted lessons from them.

Delegation to agents removes exactly those reps. A developer who has only ever reviewed AI output has never felt a bad architecture decay under their hands — and the METR finding sharpens the worry. If experienced developers slow down 19% on some tasks paying the verification tax, they are at least paying it efficiently, because they know what to look for. A junior lacks the pattern library to know where to look at all. Verification is not a checkbox skill; it is applied experience.

So the industry faces a fork, and intellectual honesty requires laying out both prongs.

The lost-art scenario. The apprenticeship ladder loses its lower rungs. Juniors orchestrate agents from day one, never accumulate implementation scar tissue, and the senior cohort that builds today's harnesses gradually retires. The people then responsible for verification systems have never learned, viscerally, what needs verifying. Software quality doesn't collapse — it sags, slowly and invisibly, because the maintenance tax the whitepaper describes arrives months after the code ships, long after the feedback loop that would have taught anyone anything has closed. In AEC terms: a generation of computational designers who can prompt a facade script into existence but cannot diagnose why the data tree structure is wrong.

The it-won't-matter scenario. Models keep improving, costs keep falling, and the harness itself becomes something agents build and maintain. Architectural judgment joins memory management and register allocation in the museum of absorbed abstractions — skills that once defined seniority and now concern almost nobody. The industry has crossed such thresholds before and flourished.

We think the second scenario rests on a disanalogy worth naming. Compilers and garbage collectors are deterministic. Nobody audits a compiler's output because it doesn't need judgment applied to it. Agents are probabilistic systems whose entire value proposition, per the paper itself, depends on verification wrapped around them. To automate away the need for human architectural judgment, you must automate verification — and then someone must verify the verifier. The regress is not infinite in practice, but it bottoms out somewhere, and wherever it bottoms out, a human with judgment is standing there.

The question is genuinely open. But the cost of being wrong is asymmetric: if judgment erodes and model progress plateaus even temporarily, the industry has no fallback inventory of people who know how the machine works.

Can architecture be learned without experience?

Put the problem at its sharpest: if experience is the mechanism by which architectural knowledge is acquired, and delegation eliminates the experience, how does anyone inexperienced ever become qualified to build the harness that the whole model of agentic engineering depends on?

We don't think there is a clean answer yet, but three partial ones deserve attention.

Review-as-apprenticeship is the default answer — juniors learn by evaluating agent output rather than writing code. It helps, but it carries a bootstrapping problem: competent review requires the judgment it is supposed to build. Reviewing a pull request teaches you little if you cannot see what's wrong with it.

Compressed consequence loops are more promising, and they are genuinely new. The reason architectural lessons took years to learn is that consequences took years to arrive. Agents build fast enough to change that. A junior can now construct a system two ways, load both with realistic change requests, and watch one of them rot — in days, not years. Failure becomes an affordable curriculum for the first time in the discipline's history. This has to be designed as training, though; it will not happen as a side effect of shipping.

Evals as a forcing function may be the most underrated. Writing an evaluation forces you to articulate, precisely, what "correct" means — which is architectural thinking made explicit. And here the harness offers an unexpected gift to the transmission problem: rule files, specs and eval suites are architecture written down, versioned and reviewable, where the previous generation's judgment lived tacitly in senior heads and hallway conversations. Tacit knowledge made explicit is knowledge that can be taught.

None of these fully replaces production scar tissue. But the second and third are where we would put training budget today.

Visual programming just changed category

Everything above engages with the paper on its own ground. This section is Bimorph's original extension — the whitepaper is domain-general and its authors were not writing about computational design. The claim that follows is ours.

The paper's evaluation machinery — the thing that separates agentic engineering from vibes — generally has to be built. Teams write rubrics, assemble test harnesses, and frequently deploy LLM-as-judge patterns, in which a second model grades the first model's output. That works, but it burns tokens on every check and inherits the fallibility of the judge.

Computational design already owns something better, and has for fifteen years.

A Grasshopper definition (the same holds for Dynamo) is a dataflow graph in which every component computes and reports. Error states, warnings, null and empty outputs, data tree topology, solve time — the environment surfaces the health of every node in the solution, structurally and visually, as a built-in property of the medium. Nobody designed this as an AI feature. It exists because visual programming had to make failure legible to non-programmers. But look at what it means for an agentic loop:

Output evaluation becomes computed, not prompted. An orchestrating agent operating on a Grasshopper definition doesn't need a judge model to ask whether the result seems right. It can read component states and quantitative outputs directly — deterministically, and at little to no token cost. And because the solver is deterministic, identical inputs yield identical outputs, which makes regression evaluation — the paper's benchmark-cluster-fix loop — almost trivial to implement.

Trajectory evaluation becomes structural. The paper treats trajectory as something you reconstruct from logs of tool calls and reasoning. In a visual program, the graph is the trajectory: the wiring, the grouping, the dependency order form a persistent, inspectable record of the solution path. You do not need a model to re-read a transcript; the artefact itself is the audit.

Notice what this does to visual programming's reputation. Grasshopper and Dynamo have periodically been dismissed as approachable substitutes for "real" code — spaghetti for people who don't write C#. The properties behind that dismissal — legibility, immediate feedback, visible dataflow, graceful per-component failure — are precisely the properties an agentic evaluation surface needs. The AI era doesn't diminish visual programming. It promotes it into a new category: a native eval layer that the rest of the software industry is currently paying to construct from scratch.

It also speaks directly to the METR verification tax discussed above. If the cost of AI-assisted work is dominated by humans checking output, then an inspectable graph — where failure is visible at a glance and computable by a script — lowers the verification cost for humans and agents alike. The medium pays the tax down on both sides of the loop.

For CTOs and technology leads at AEC firms, the strategic point is simple: your computational design teams already operate harness-shaped tooling. The evaluation surface that software teams elsewhere are assembling out of rubrics and judge models ships, in your industry, inside the authoring environment itself.

We are building this argument out in practice — a companion Bimorph Insights piece explores agentic engineering inside AutoCAD via Rhino.Inside, with the Grasshopper graph acting as the deterministic evaluation layer of the loop. That article covers the concrete tooling; this one makes the conceptual case.

Amplification

The whitepaper closes on the idea that AI amplifies whatever engineering culture it lands in — strengths and weaknesses alike. We would sharpen that for our industry.

An AEC firm can treat AI as a vending machine: prompt, accept, ship, and quietly accrue the maintenance tax on parametric models nobody can explain in two years' time. Or it can treat its computational designers as what they have functionally become — harness engineers — and invest in the structure that makes agent output verifiable: the specs, the evals, and the graphs that hold them.

The skills conversation, properly reframed, is not about whether software architecture still matters in the age of AI. The paper settles that: it matters more, because it has become the 90%. The real question is whether your organisation can still grow the people who hold that judgment — and whether it recognises that in Grasshopper and Dynamo, construction has been holding a piece of the answer all along.

References

• Osmani, A., Saboo, S. & Kartakis, S. — The New SDLC With Vibe Coding, Google / Kaggle, June 2026. kaggle.com/whitepaper-the-new-SDLC-with-vibe-coding
• Osmani, A. — The New Software Lifecycle (co-author commentary and figures), June 2026. addyosmani.com/blog/new-sdlc-vibe-coding
• METR — Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, July 2025. metr.org
• METR — Uplift Update: Measuring the Impact of AI Coding Tools, February 2026. metr.org
• Karpathy, A. — original "vibe coding" post, X, February 2025. x.com/karpathy