1
00:00:00,600 --> 00:00:12,936
Let's understand AnchorPath — a way to compress large language models far more aggressively, without ever touching their weights. We'll build it up from scratch, assuming no background.

2
00:00:13,276 --> 00:00:21,964
The whole story is one idea: why compression breaks down at high sparsity, and how re-anchoring along the way fixes it.

3
00:00:22,714 --> 00:00:33,778
Start with the goal. A large language model is enormous — billions of parameters, organized into repeated blocks of attention heads and feed-forward channels.

4
00:00:34,118 --> 00:00:44,726
Pruning makes it smaller by deleting whole components — entire attention heads, whole channel groups — so the model genuinely runs faster and uses less memory.

5
00:00:45,066 --> 00:00:53,754
The catch: we want to do this training-free. No retraining, no fine-tuning. We only have a little unlabeled text to calibrate on.

6
00:00:54,094 --> 00:01:05,254
So we measure damage simply: how far do the pruned model's predictions drift from the original dense model's? Keep the components whose removal barely moves the output.

7
00:01:06,004 --> 00:01:11,524
How do you decide what to cut? The standard recipe is one-shot second-order pruning.

8
00:01:11,864 --> 00:01:22,112
It builds a single mathematical map — a local estimate of how much accuracy each component costs to remove — measured at the full, dense model.

9
00:01:22,452 --> 00:01:28,956
Then it uses that one map to choose everything to delete, all the way down to the target budget.

10
00:01:29,296 --> 00:01:36,448
This works beautifully when you remove only a little. Near the dense model, the map is accurate.

11
00:01:37,198 --> 00:01:44,422
But push it further, and something dramatic happens. Quality doesn't just degrade — it collapses.

12
00:01:44,762 --> 00:01:56,330
Here's the signature on a real model. Remove ten percent, and perplexity — where lower is better — barely moves. At twenty, thirty percent, it creeps up.

13
00:01:56,670 --> 00:02:09,606
But at forty and fifty percent it explodes: from about twenty, to ninety, to nearly fifteen hundred. Each extra ten percent hurts more than the last — the collapse accelerates.

14
00:02:09,946 --> 00:02:17,842
And this is not one bad method. Every one-shot pruner, no matter how clever its importance score, hits the same wall.

15
00:02:18,592 --> 00:02:25,456
Why? Because the map was only ever accurate near the dense model — and we have walked far away from it.

16
00:02:25,796 --> 00:02:36,788
Picture the true cost of pruning as a curve that bends upward. One-shot fits a single approximation at the start and extends it in a straight, optimistic line.

17
00:02:37,128 --> 00:02:48,216
Near the start they agree. Far out, the gap between the real curve and the approximation grows — and it grows faster than linearly. That gap is the collapse.

18
00:02:48,556 --> 00:03:01,708
There is a second, subtler problem. Once the model is no longer the original, removing a component changes damage that has already accumulated — something the starting-point map simply cannot see.

19
00:03:02,048 --> 00:03:09,416
Both problems are about where the map was drawn, not which components it ranked. So no importance score can escape them.

20
00:03:10,166 --> 00:03:14,630
AnchorPath's fix is to stop drawing one map for the whole journey.

21
00:03:14,970 --> 00:03:24,570
Instead, trace the path. Prune a little, then redraw the map at the new, partially-pruned model. Prune a little more, redraw again.

22
00:03:24,910 --> 00:03:30,190
The key is to separate two roles that one-shot secretly merges into one.

23
00:03:30,530 --> 00:03:40,994
The anchor — the original dense model — stays fixed. All damage is always measured against it, so we always know how far we've drifted.

24
00:03:41,334 --> 00:03:49,638
The expansion point — where we redraw the map — moves along the path with us. That is what keeps every step accurate.

25
00:03:49,978 --> 00:03:59,482
Separating them revives a signal one-shot throws away: it flags components that have become costly precisely because of the damage already done.

26
00:04:00,232 --> 00:04:13,096
Concretely, we split the budget into K steps. At each step we re-estimate the local curvature at the current model, greedily fill that slice of the budget, physically remove those components, and move on.

27
00:04:13,436 --> 00:04:22,820
Here is the elegant part. With K equal to one — a single step — AnchorPath is exactly one-shot pruning. Bit for bit.

28
00:04:23,160 --> 00:04:35,400
So every improvement comes purely from tracing the path more finely. With more steps, the errors add up gently instead of compounding — they add, rather than multiply.

29
00:04:35,740 --> 00:04:50,236
And it stays cheap. Instead of redrawing everything each step, we only re-check the components most affected by what we just removed — so the whole trace costs about four and a half times a single shot, offline, and nothing extra to deploy.

30
00:04:50,986 --> 00:04:58,570
So does it work? Take the headline model, LLaMA-3.1-8B, at fifty percent sparsity.

31
00:04:58,910 --> 00:05:11,750
One-shot's perplexity has exploded past four thousand. AnchorPath holds it at 245 — sixteen point eight times lower — and it beats every training-free baseline.

32
00:05:12,090 --> 00:05:21,378
On another model, the gap is nearly two orders of magnitude: a perplexity of sixteen thousand collapses to just sixty-two.

33
00:05:22,128 --> 00:05:34,416
This is not cherry-picked. Across seven models from five different families, AnchorPath is the lower envelope — the best of every training-free method — at high sparsity.

34
00:05:34,756 --> 00:05:50,932
And the effect is smooth and predictable. As we add steps — one, two, four, eight — perplexity falls monotonically: fifteen hundred, twelve hundred, eight hundred, down to about three hundred ninety. More tracing never hurts.

35
00:05:51,682 --> 00:05:59,170
Because only the mask changes — no weight is ever touched — we can see exactly why it wins.

36
00:05:59,510 --> 00:06:10,622
AnchorPath removes dozens of attention heads that one-shot never touches — heads that look cheap from the dense model, but whose true cost only appears once the model has moved.

37
00:06:10,962 --> 00:06:19,914
It is a compression that is fully inspectable, and it gives you the exact same fast, small model as one-shot at that budget — just a far better one.

38
00:06:20,664 --> 00:06:35,664
In deployment terms: a seven-billion-parameter model drops to under four billion, with a one-point-six times faster prefill and a third less memory — and since the pruned model is just a mask, there is no runtime overhead.

39
00:06:36,004 --> 00:06:53,524
The paper is careful about scope. No training-free pruner survives untouched far beyond moderate sparsity; there, AnchorPath's job is to hand the most recoverable model to a later recovery stage. Generation tasks need that stage too.

40
00:06:54,274 --> 00:07:00,898
So the lesson reaches beyond pruning. The collapse was never about picking the wrong components.

41
00:07:01,238 --> 00:07:10,190
It was about trusting a local map far past where it was drawn. The fix is to keep a fixed reference, and redraw the map as you go.

42
00:07:10,530 --> 00:07:19,146
One-shot is simply the single-step version of that idea. Trace the path, and the collapse dissolves — with no training at all.

43
00:07:19,486 --> 00:07:27,310
The project page, the tutorial, and the code and paper — coming soon — are linked on screen. Thanks for watching.