Let's understand AnchorPath — a way to compress large language models far more aggressively, without ever touching their weights. We'll build it up from scratch, assuming no background.
The whole story is one idea: why compression breaks down at high sparsity, and how re-anchoring along the way fixes it.
Start with the goal. A large language model is enormous — billions of parameters, organized into repeated blocks of attention heads and feed-forward channels.
Pruning makes it smaller by deleting whole components — entire attention heads, whole channel groups — so the model genuinely runs faster and uses less memory.
The catch: we want to do this training-free. No retraining, no fine-tuning. We only have a little unlabeled text to calibrate on.
So we measure damage simply: how far do the pruned model's predictions drift from the original dense model's? Keep the components whose removal barely moves the output.
How do you decide what to cut? The standard recipe is one-shot second-order pruning.
It builds a single mathematical map — a local estimate of how much accuracy each component costs to remove — measured at the full, dense model.
Then it uses that one map to choose everything to delete, all the way down to the target budget.
This works beautifully when you remove only a little. Near the dense model, the map is accurate.
But push it further, and something dramatic happens. Quality doesn't just degrade — it collapses.
Here's the signature on a real model. Remove ten percent, and perplexity — where lower is better — barely moves. At twenty, thirty percent, it creeps up.
But at forty and fifty percent it explodes: from about twenty, to ninety, to nearly fifteen hundred. Each extra ten percent hurts more than the last — the collapse accelerates.
And this is not one bad method. Every one-shot pruner, no matter how clever its importance score, hits the same wall.
Why? Because the map was only ever accurate near the dense model — and we have walked far away from it.
Picture the true cost of pruning as a curve that bends upward. One-shot fits a single approximation at the start and extends it in a straight, optimistic line.
Near the start they agree. Far out, the gap between the real curve and the approximation grows — and it grows faster than linearly. That gap is the collapse.
There is a second, subtler problem. Once the model is no longer the original, removing a component changes damage that has already accumulated — something the starting-point map simply cannot see.
Both problems are about where the map was drawn, not which components it ranked. So no importance score can escape them.
AnchorPath's fix is to stop drawing one map for the whole journey.
Instead, trace the path. Prune a little, then redraw the map at the new, partially-pruned model. Prune a little more, redraw again.
The key is to separate two roles that one-shot secretly merges into one.
The anchor — the original dense model — stays fixed. All damage is always measured against it, so we always know how far we've drifted.
The expansion point — where we redraw the map — moves along the path with us. That is what keeps every step accurate.
Separating them revives a signal one-shot throws away: it flags components that have become costly precisely because of the damage already done.
Concretely, we split the budget into K steps. At each step we re-estimate the local curvature at the current model, greedily fill that slice of the budget, physically remove those components, and move on.
Here is the elegant part. With K equal to one — a single step — AnchorPath is exactly one-shot pruning. Bit for bit.
So every improvement comes purely from tracing the path more finely. With more steps, the errors add up gently instead of compounding — they add, rather than multiply.
And it stays cheap. Instead of redrawing everything each step, we only re-check the components most affected by what we just removed — so the whole trace costs about four and a half times a single shot, offline, and nothing extra to deploy.
So does it work? Take the headline model, LLaMA-3.1-8B, at fifty percent sparsity.
One-shot's perplexity has exploded past four thousand. AnchorPath holds it at 245 — sixteen point eight times lower — and it beats every training-free baseline.
On another model, the gap is nearly two orders of magnitude: a perplexity of sixteen thousand collapses to just sixty-two.
This is not cherry-picked. Across seven models from five different families, AnchorPath is the lower envelope — the best of every training-free method — at high sparsity.
And the effect is smooth and predictable. As we add steps — one, two, four, eight — perplexity falls monotonically: fifteen hundred, twelve hundred, eight hundred, down to about three hundred ninety. More tracing never hurts.
Because only the mask changes — no weight is ever touched — we can see exactly why it wins.
AnchorPath removes dozens of attention heads that one-shot never touches — heads that look cheap from the dense model, but whose true cost only appears once the model has moved.
It is a compression that is fully inspectable, and it gives you the exact same fast, small model as one-shot at that budget — just a far better one.
In deployment terms: a seven-billion-parameter model drops to under four billion, with a one-point-six times faster prefill and a third less memory — and since the pruned model is just a mask, there is no runtime overhead.
The paper is careful about scope. No training-free pruner survives untouched far beyond moderate sparsity; there, AnchorPath's job is to hand the most recoverable model to a later recovery stage. Generation tasks need that stage too.
So the lesson reaches beyond pruning. The collapse was never about picking the wrong components.
It was about trusting a local map far past where it was drawn. The fix is to keep a fixed reference, and redraw the map as you go.
One-shot is simply the single-step version of that idea. Trace the path, and the collapse dissolves — with no training at all.
The project page, the tutorial, and the code and paper — coming soon — are linked on screen. Thanks for watching.