Graph Engineering, From Someone Who Actually Runs the Graph

Most writing about multi-agent verification is written by people who read about verifier gates. This is written from the other side of the gate. By someone who runs one in production and has the scar tissue to prove it.

The genre has a tell

There is a wave of "graph engineering" and "loop engineering" content right now. Fan out N agents, put a verifier on the edge, dedupe against everything seen, never let an agent grade its own homework. The advice is correct. It is also, almost universally, written by people describing a pattern they have not had to live with.

You can tell because they stop at the happy path. The real cost of running an agent graph in production is not in the diagram. It is in the three or four things the diagram doesn't show you. The ones that only announce themselves after you've shipped the pattern and it quietly fails.

I want to give you the version with the failures in it, because I run this every day against a codebase where a wrong answer has a real cost. Here is one concrete run, start to finish, including what broke.

The problem: a gate that can't see what it can't run

We maintain a claims registry. A machine-readable ledger binding every capability claim the system makes to a proof that executes it. A claim is only allowed to say REAL if there is a command that runs and exits clean. A deterministic verifier walks the whole registry on every commit and fails the build if any proof doesn't pass. No model in the loop. Fast, reproducible, boring. Exactly what a gate should be.

It has one blind spot, and it is structural.

The deterministic gate proves the code runs. It cannot prove the English is honest.

A proof can pass while the sentence describing it quietly overclaims. The test for a consensus round can go green while the prose says "live distributed commit" when what actually ran was in a single process. The word "detected" can survive in a description long after the result it referred to was retracted. This is not hypothetical. It is the single most common way an honest system drifts into dishonesty: the code stays correct, the story about the code inflates, and every automated gate you own waves it through because the gate only checks the code.

That gap (prose-versus-proof drift) is exactly the shape of problem a graph is for. It is wide (every claim, independently), it is judgment-heavy (does this sentence overclaim?), and it is adversarial (you want something actively trying to catch you, not confirm you). One agent reading 75 claims in one context window would drown. So: fan out.

The graph

Three phases, and every design choice earns its place:

1. Audit (fan out). Batch the 75 claims into groups of ten, one agent per batch. Each agent reads the actual source files behind its claims, looks at what the proof command actually exercises, and asks one question: does the English assert more than the proof and the code demonstrate? It returns structured findings (never free text) so the next stage can consume them without guessing.

Batching into groups instead of one-agent-per-claim is not laziness; it is the layered fan-in pattern. Seventy-five parallel agents feeding one consolidation step would blow the context budget before synthesis even started. Ten batches of ten keeps each agent's window clean and the fan-in tractable.

2. Verify (maker-checker, on fresh context). This is the phase the happy-path articles get right in theory and wrong in practice. Every flagged drift goes to a second agent whose only instruction is to refute the accusation. Not to confirm it. To kill it. Default posture: skeptical of the finding. A drift only survives if it's still standing after something tried to knock it down.

The non-negotiable detail: the verifier gets a clean context. If you hand the checker the same conversation the auditor had, you have not built a checker. You have built the auditor agreeing with itself in a different font. A graph of agents sharing one context is a single loop wearing a costume. It fails the same way, later, with more green lights on the way down.

3. Consolidate. Rank the survivors, discard the refuted (keeping the reasoning, so a rejected finding doesn't come back next run), emit one report.

What actually happened, including the part that broke

I ran exactly this graph over a 75-claim registry. Here is the honest ledger, failures first.

It took three launches to get a clean run. Both failures were mine, not the model's.

The first launch died in 30 milliseconds: my orchestration code assumed the input arrived as a parsed array when the runtime handed it a string. The second died the same way, one line later. A JavaScript operator-precedence trap where await parallel(...).filter(...) runs the filter on the unresolved promise instead of the results. Two ordinary programming bugs in the wiring, caught before a single agent spawned. If I had blamed "the AI" the first time it "failed," I would have spent the afternoon debugging the wrong layer. The fleet was never the problem. Line 22 was, and then line 38.

Once the wiring was correct, the run itself was clean and cheap to reason about. Eight audit agents fanned across the registry in parallel, each reading the real source behind its batch. They surfaced 13 candidate drifts. Then the verify phase (one adversarial checker per candidate, fresh context, instructed to refute) killed 5 of them. What survived the graph: 8 confirmed drifts. What survived contact with the fixes was fewer, and I'll come back to that.

That 5-of-13 cull is the whole argument for the maker-checker split, and it is not theoretical. In one case, two independent verifiers examined the same flagged claim and both refuted it. The finding read plausibly, cited real line numbers, and was still wrong, because the auditor had misread the scope of what the claim actually promised. A graph that let its finders grade their own work would have shipped all 13 as "confirmed." A third of that report would have been noise. The skeptic on fresh context is the only reason the number narrowed honestly.

Then, applying the fixes, an eighth fell over, and the reason is the most useful thing in this whole run. One surviving finding said a claim cited too few references to support its wording. It didn't. The real registry record listed every reference and every test file the claim needed. What the graph had been handed was an abbreviated copy of that record, with one field trimmed to keep the prompts small. The auditor read the abbreviation, correctly noticed something missing, and reported a defect in my summary as a defect in the system. The adversarial checker couldn't save me, because the skeptic was working from the same truncated record. Both halves of the maker-checker split were honest, careful, and looking at the wrong document.

Final honest count: 13 candidates → 5 refuted by the skeptics → 1 refuted by reality → 7 genuine drifts.

The 7 genuine drifts clustered into four repeatable kinds, and these are the portable lesson, independent of any one codebase:

• Scope-word inflation. A component does a real, modest job, and the sentence describing it borrows heavy vocabulary from a more impressive job it never performs. The test passes; the adjectives don't.
• Algorithm mislabeling. A simple mechanism gets named after a sophisticated one with different guarantees. The proof exercises the simple thing; the prose promises the sophisticated thing.
• Proof-scope gaps. The sentence claims capabilities A, B, and C. The cited test proves A. B and C are real elsewhere. But nothing this claim points at exercises them, so the gate blesses the whole sentence on one-third of the evidence.
• Conditional-capability-as-present-tense. A feature that is optional, and switched off in the test, is described as an always-on property.

Every one of these is invisible to a deterministic gate, because in every case the test passes. The code is correct. Only the story about the code drifted. That is the exact seam a graph of skeptical readers is built to catch, and it caught seven of them in one pass.

The four things the diagrams don't tell you

Everything above is the pattern working. Here is what running it taught that no methodology post mentions:

1. Your orchestration layer is code, and code has bugs that have nothing to do with agents. The most expensive failures in an agent graph are frequently not model failures at all. They are ordinary programming defects in the wiring. A value arriving as the wrong type, a barrier where you needed a stream. The seductive story is "the AI got confused." The boring truth is usually "line 22 assumed an array." Debug the plumbing before you blame the fleet.

2. "Independent" is a claim you have to verify, not assume. Two agents look independent because their prompts don't reference each other. Then they both write the same file, or hammer the same rate-limited endpoint, and you have a hidden edge you didn't draw. Audit for shared resources, not just shared data. The collision won't show up in the diagram; it shows up at 2 a.m.

3. Silent partial success is the failure mode that costs you. In a straight chain, one failure stops everything. Annoying, but obvious. In a graph, one dead agent among many vanishes into a report that looks complete. If you fan out N and only check that you got "some" results back, you will one day ship a confident, thorough-looking answer built on two-thirds of the evidence. Every fan-in must count its inputs against what it expected and say so out loud when they don't match.

4. Feed the graph the real records, not your summary of them. This is the one the eighth finding taught me, and it sits underneath the maker-checker split rather than beside it. Agents read the files you point them at, but they reason over the metadata you hand them, and it is very tempting to trim that metadata to keep prompts cheap. The moment you do, some fraction of your findings will be about your abbreviation instead of your system. Worse, adversarial verification cannot catch it: the skeptic is handed the same truncated record as the auditor, so both agree, honestly, about a document that doesn't exist. Every other failure in this list gets caught by some other agent. This one is invisible from inside the graph, and only shows up when you try to act on the finding. If you compress what you pass in, mark the compression explicitly, or expect to spend an afternoon fixing something that was never broken.

Why this is the moat, not the pattern

Here is the uncomfortable part for the genre, and the honest part for anyone deciding whether this is worth doing.

The pattern is commodity. Fan-out, verifier-on-the-edge, refute-don't-confirm, dedupe-against-seen. It's in every post now, and it's in the tooling docs. Nobody owns it. If your advantage is knowing the diagram, you have no advantage.

The discipline is not commodity. A registry that forces every claim to carry an executable proof. A gate that fails the build, not a checklist somebody skips. The scar tissue that tells you to put the verifier on fresh context before it burns you, to count your fan-in results before the incomplete report ships, to suspect your own wiring before you blame the model. That doesn't come from reading about graphs. It comes from running one against something that matters and paying for the lessons.

The diagram is free. The judgment about where it breaks is the whole job.

Matrix CR Studio builds verification infrastructure for post-quantum migration and sovereign AI systems. The kind of work where "looks done" and "is provably correct" are different planets, and only the second one ships.