Preprint · Training-Free Structured Pruning

CoCurve

Cross-Module Co-Pruning Curvature: a calibration-only, fine-tuning-free method that prunes attention and FFN units jointly — by preserving the curvature of the edges between units, not just ranking the nodes.

Zhiren Gong^1,2, Zihao Zeng¹, Zijie Wang¹, Chau Yuen³, Wei Yang Bryan Lim¹

¹ College of Computing and Data Science, Nanyang Technological University ² Interdisciplinary Graduate Programme, Nanyang Technological University ³ School of Electrical and Electronic Engineering, Nanyang Technological University

On the instruction-tuned Llama-3.1-8B at 20%, CoCurve is the only training-free method strong across every capability class — and best preserves the fragile code ability others sacrifice. The catch everyone misses: ~98% of the curvature mass lives in the edges.

Paper · coming soon Code · coming soon ▶ Tutorial

Fraction of dense retained: CoCurve vs best baseline across capability classes — Figure. Fraction of dense capability retained on Llama-3.1-8B-Instruct at 20% — CoCurve leads every class, and most on code.

4 + 24B

Models (3B–8B) + a larger-scale point

14 tasks · 5 families

LM · commonsense · knowledge · math · code

~98%

Of curvature mass is off-diagonal (edges)

M ablations

Full edge matrix, no pairwise sweeps

0 fine-tuning

Calibration-only · one shot

Abstract

Structured pruning compresses LLMs by deleting whole units — attention heads and FFN channel groups. Almost every method scores each unit in isolation, implicitly assuming that the damage of pruning a set is additive. We show this is the wrong granularity for Transformers: because every sublayer reads from and writes to a shared residual stream, two individually weak units can be jointly indispensable, and two salient ones partly redundant.

CoCurve (Cross-Module Co-Pruning Curvature) prunes attention and FFN units jointly. From the token-level KL divergence between the frozen model and its masked copy, a second-order Taylor expansion yields a single Fisher matrix: its diagonal is classical node saliency, while its off-diagonal entries are co-pruning curvature edges — the extra damage from removing two units together.

We prove this entire edge matrix is a Gram product of single-unit ablations, recovering it from just M ablations with no pairwise sweeps, gradients, or labels — then prune in one shot under a shared budget. CoCurve reframes structured pruning from ranking nodes to preserving the curvature of the edges between them.

The Problem · Nodes Aren't Independent

Isolated scoring is the wrong granularity

The standard recipe gives every unit an importance score and assumes set damage is additive: ΔL(S) ≈ Σ ΔL(u). But a Transformer is a coupled system — every sublayer reads and writes one shared residual stream. Pruning a set is a joint operation, not a sum.

Bridge units get co-pruned

The consequence: a node-first ranker inevitably co-prunes a low-saliency bridge unit together with the very units that depend on it. Two weak-looking units can be jointly indispensable; two salient ones can be partly redundant. And activation correlation doesn't fix it — it is dominated by scale and disconnected from the output distribution.

Bridge units: low saliency, high connectivity — Figure. Bridge units (87 of 768 on Llama-3.1-8B) — low individual saliency but high edge connectivity — exactly what a node-only score wrongly deletes.

Method

One Fisher matrix, read two ways

Because the masked student equals the teacher at zero pruning, both the risk and its gradient vanish there, so the leading term of the token-level KL is purely second order: R(s) ≈ ½·sᵀH·s. The single matrix H carries everything.

Diagonal = node saliency

H_uu is exactly classical single-unit importance — a structured Optimal Brain Damage.
Non-negative by construction: a built-in correctness check on the estimate.
Yet on real models it is only a few percent of the total curvature mass.

Off-diagonal = co-pruning edges

H_uv = the extra damage from removing u and v together.
Positive → jointly harmful; negative → perturbations cancel; zero → independent.
These attention↔FFN cross-module edges carry the decisive mass.

Recovered from M ablations — the Gram theorem

In the Fisher metric, H = (1/P)·D̄ᵀD̄ is a Gram product of single-unit ablation features.
So the full M×M edge matrix comes from M single-unit ablations — no O(M²) pairwise sweeps, no gradients, no labels.
A one-shot O(MK) cost-normalized greedy solves a shared attention+FFN budget, so per-type ratios emerge from the objective.
A calibration-selected trust-region λ interpolates diagonal-only (λ=0, an OBD floor) to full edges (λ=1); an optional gated compensation is reported transparently and never degrades the result.

Step 1 · Ablate

Run M single-unit ablations on a small calibration set; extract Fisher-weighted delta-logit features.

Output: the features whose Gram matrix is H.

Step 2 · Assemble H

Form the single Fisher matrix — diagonal saliency and off-diagonal co-pruning edges — with no pairwise sweep.

Output: node saliency + attention↔FFN edges together.

Step 3 · Prune once

Cost-normalized greedy under a shared budget, trust-region λ from calibration risk; physically remove units.

Result: a jointly-pruned subnetwork, mask-equals-physical to <10⁻³.

Curvature correlation matrix with attention-FFN cross block — Figure. The estimated curvature correlation — the attention↔FFN cross-block (mean 0.086) is real and substantial, not noise.

Edge structure of the curvature matrix — Figure. Edge structure of H — the cross-module coupling that a node-only method is blind to.

Main Results

Llama-3.1-8B · 20%

12.91 Wiki · .610 Cmn · .389 MMLU · .091 Code

The only method strong across every capability class (dense 6.40 / .699 / .478 / .583).

Code is preserved

16% of dense code retained vs 11% for the best baseline

CoCurve best preserves the fragile code-generation ability other training-free methods sacrifice.

Perplexity everywhere

Leads WikiText perplexity on every model tested

Up to 14.5× lower than the best baseline at 40% on Mistral-Small-24B.

Main Table — 20% Structured Pruning (Wiki ↓ / Commonsense ↑ / MMLU ↑ / Code ↑)

Hint: scroll horizontally to view all columns on smaller screens.

Model	Method	Wiki ↓	Cmn ↑	MMLU ↑	Code ↑
Llama-3.1-8B-Instruct	Dense	6.40	.699	.478	.583
	Best baseline	13.23	.586	.379	.072
	CoCurve	12.91	.610	.389	.091
Mistral-7B	Dense	4.78	.675	.441	.322
	Best baseline	8.11	.607	.367	.095
	CoCurve	8.10	.605	.374	.079
Llama-3.2-3B	Dense	6.91	.628	.418	.323
	Best baseline	16.90	.518	.335	.024
	CoCurve	16.71	.512	.325	.046
Falcon3-7B	Dense	5.43	.670	.452	.532
	Best baseline	7.43	.591	.398	.157
	CoCurve (λ=.5)	7.03	.602	.390	.188
Mistral-Small-24B	Dense	4.70	.738	.523	.475
	Best baseline	10.42	.629	.446	.138
	CoCurve (λ=1)	8.31	.648	.462	.215

CoCurve vs dense vs best baseline across benchmarks — Figure. Per-benchmark scores on Llama-3.1-8B-Instruct at 20% — CoCurve matches or leads the best baseline everywhere, widening on code.

Accuracy across pruning ratios — Figure. CoCurve from 10–40% — graceful through the moderate regime; all training-free methods enter a collapse zone beyond ~30%.

Why It Works · The Edges Are Decisive

The cross-module edge is the ingredient

On Mistral-7B at 20%, a diagonal-only variant (node saliency alone) reaches 13.3 perplexity; adding the edges cuts the gap to dense roughly in half, to 8.1 (dense 4.8) — and lifts code (MBPP) from 0.6% to 12.2%, about 20×.

Tellingly, keeping only within-module edges (15.0) is worse than pure diagonal (13.1): it is specifically the attention↔FFN cross-block that is load-bearing.

An a-priori rule for when edges help

The edge bonus grows as FFN channels become less redundant — a quantity read from the calibration matrix alone, before any benchmark. Low-redundancy models (Falcon3 0.028, Llama-3.1-8B 0.074) take full edges (λ=1); high-redundancy ones (Qwen2.5-32B 0.242) damp toward the OBD diagonal.

So the trust-region λ is chosen without labels, and the method degrades gracefully to a strong node-saliency selector exactly where edges would not help.

Ablation — Mistral-7B, 20% (WikiText ↓ / MBPP ↑)

Variant	Wiki ↓	MBPP ↑
Dense	4.78	—
Diagonal only (λ=0)	13.26	.006
Head-only	1526	—
FFN-only	9.10	.076
Full CoCurve (λ=1)	8.10	.122

FFN redundancy vs edge benefit — Figure. The a-priori rule: edge benefit tracks FFN redundancy, measured from calibration alone across ten models.

Efficiency & Scope

Real removal — accuracy at a given speedup

At 20% CoCurve physically removes ~17% of non-embedding parameters: peak memory 0.83×, prefill ~1.15× faster.
Across nine training-free methods the prefill speedup band is a narrow 1.15–1.19× — governed by the removed parameter fraction, essentially method-independent.
So the real discriminator is not throughput but accuracy at a given speedup — where CoCurve wins.
Every reported speedup reflects real physical removal, validated mask-equals-physical to <10⁻³.

Honest scope

Without a recovery stage, all training-free methods degrade super-linearly beyond ~30% — CoCurve's sweet spot is moderate compression.
The edge bonus is architecture-dependent; on high-redundancy families it damps to a strong OBD diagonal (a graceful fallback, not a failure).
Fully-evaluated models span 3–10B; Mistral-Small-24B is a larger-scale point (dense & CoCurve final; some baselines completing). MoE is out of scope.

Resources

Paper

Under review. Coming soon.

Code

Reference implementation and reproduction. Coming soon.

Tutorial

A narrated, animated ~7-minute video tour — the coupling problem, the co-pruning curvature edges, and the results, built for a general audience.

Contact

For questions about this project, contact zhiren001@e.ntu.edu.sg.