Let's understand Conditional Co-Ablation — a method for finding the hidden backup parts of a neural network's reasoning. We'll build it up from the very beginning, assuming no background in interpretability.
Along the way we'll see why the standard way of measuring importance has a blind spot — and how one change to the question fixes it.
First, what are we even looking at? A language model reads text and predicts the next word.
Mechanistic interpretability is the effort to reverse-engineer how it does that — not just what it answers, but the internal computation that produces the answer.
The model is built from small, repeated parts. The ones we'll focus on are attention heads — each a tiny unit that moves and mixes information between words.
For any one task, only a handful of these heads actually do the work. That small, connected set of components is called a circuit — the model's wiring for a single behavior.
How do researchers find a circuit? With a simple, powerful tool: ablation.
You switch off one component and watch how much the answer changes. A big drop in the correct answer means it mattered; a small drop means it didn't.
Do this for every head, rank them by their effect, and the important ones light up. This single-ablation score is the workhorse of the whole field.
Here's a concrete circuit. On a name-copying task, a few heads read the correct name and write it to the output. Ablate them, and the model should fail.
But something strange happens. You delete an important head — and the answer barely changes.
The reason is self-repair. The moment a key component is removed, another component that was sitting silent wakes up, takes over its job, and re-routes the answer.
This is redundancy. The network keeps dormant backups and heals itself under damage — a little like a power grid rerouting around a failed line.
Now watch what that does to our measurement. The primary head, when deleted, looks unimportant — because the backup hid the damage.
And the backup looks unimportant too — because on the intact model it was silent. A single-ablation score misreads both sides of the redundancy at once.
Deleting a group is not the sum of deleting its parts. This is a genuine blind spot — and it is not just a curiosity.
Interpretability isn't only about understanding. It's used to attribute behavior to components, to knock out unwanted capabilities, and to prune models smaller.
Every one of those tools is built on the importance score. When the score is blind to backups, all of them inherit the same blindness.
For safety this is the sharp edge: try to remove a capability by deleting the heads that seem to carry it, and a dormant backup can quietly restore it.
So CoAx changes the question. Don't ask how important a unit is on its own.
Ask a conditional one: once the main circuit is already removed, how much does this unit's effect grow?
A true backup does almost nothing by itself — its effect alone is near zero. But once its partner is gone, its effect becomes large. That growth is the CoAx score.
It needs no labels and no gradients — only forward passes that measure the change in the output. Silent alone, load-bearing once its primary is gone: that is exactly the signature of a backup.
For the simplest case — a pair of heads — there is a companion signal called synergy.
Ablate each head alone and little happens. Ablate them together, and the effect is far larger than the sum of the two. That gap flags a mutually-compensating pair.
Measure this across every pair of heads, and a hidden structure appears: the main heads and their backups, wired in parallel — a module a one-head-at-a-time score can never see.
Now, the evidence. We start where we actually have ground truth — a circuit with eight documented backup heads.
We rank all the heads by each method, and check how well the known backups rise to the top. A score of one is a perfect ranking; a half is random guessing.
Single-ablation scoring sits below chance, at 0.33 — worse than a coin flip, because self-repair actively hides the backups.
Even gradient-based methods, built to be smarter, top out around 0.82. CoAx reaches 0.91 — moving the backups from the blind spot to the top of the ranking.
But a high score isn't proof. Are these heads really backups? We test them causally.
As we remove the main heads one by one, the heads CoAx found wake up — their activity climbs and they start writing the correct answer — while random heads stay flat.
And the decisive test: freeze these backups so they cannot respond, and 55 percent of the self-repair disappears. They don't just correlate with the healing — they cause it.
Because the backups are load-bearing, recovering them repairs the downstream tools that were blind to them.
Take attribution — measuring how much a circuit really drives the answer. The primaries alone seem to move it by only 0.22, because self-repair masks the rest.
Add the recovered backups, and the true effect reappears, several times larger. The redundancy had been hiding most of the real signal.
Capability knockout tells the same story: adding CoAx's backups reaches 0.70, matching the 0.72 you get from the documented backups, while a first-order top-up overshoots and damages other behavior.
And a repair-aware pruning order, which re-measures importance as heads are removed, stays closest to the full model — from a 124-million-parameter network up to 7 billion.
Is this one lucky circuit, or a general phenomenon? The same label-free method transfers to a second circuit — induction — across eight further models and six architecture families.
And it is honest about its limits. Where redundancy is shared among heads that fire together, simpler input-side methods can find it too; and on circuits dominated by feed-forward layers, this head-level signal does not transfer.
Step back, and the lesson is about measurement itself. A component's importance is not a fixed property — it is conditional on what else is present.
The parts that matter most are often the ones that stay silent until they are needed. Measure them in isolation, and you will miss them every time.
CoAx recovers exactly those hidden parts — label-free and gradient-free — and hands back a truer picture of how a model actually computes.
The code, the paper, and an interactive page are linked on screen. Thanks for watching.