Let's take a guided tour of LoTR — Logic-of-Thought Routing — a lightweight plug-in that improves how large language models reason, from the inside.
First, the problem. Today's reasoning methods work almost entirely from the outside.
Chain-of-thought, plan-and-solve, self-consistency, best-of-N, constrained beam search, tree search — each one prescribes a better external procedure for the model to follow.
But they all share one blind spot. None of them ask whether the model's own internal computation actually matches the logic the current step needs.
A transformer reasons through attention heads. At any step, some heads support the current move, some are irrelevant, and some actively interfere.
So a single fixed set of active heads is suboptimal. The pathway that is right for decomposing a problem is not the pathway that is right for verifying an answer.
LoTR's idea is simple. At each step, read the model's internal state, infer which logic regime it is in, and softly re-route its attention heads to match.
It does not replace chain-of-thought or search. It plugs in underneath them, adjusting the internal pathway while the outer scaffold stays exactly the same, and the backbone weights stay frozen.
The method has two stages. Offline, we compile a compact basis of logic states. Online, we simply read, and route.
Offline, we run frozen models across many paradigms and tasks, and read a sentence-level state vector at every reasoning step.
Clustering these states reveals a small set of recurring logic regimes — explore, consolidate, verify, exploit, and resolve — that show up again and again, whatever the task or the outer method.
For each regime we learn two things: a lightweight per-layer probe that recognises it, and a routing template that says how strongly each attention head should participate when that regime is active.
Online, at every sentence boundary, the probes read the current state and output soft mixing weights over the regimes. Because they are sigmoids, several regimes can be active at once.
Those weights blend the cached templates into a single gate matrix over all the heads, which is clipped to a valid range and applied to the next step.
The effect is gentle. Most heads — the logic-insensitive backbone — stay near identity. Only the few heads that clash with the active regime get turned down. No search, no planner, no extra weights.
So, does it help? Across three backbones, eight benchmarks, and eight reasoning paradigms, LoTR lifts average accuracy by nearly eight percent.
The gains are largest exactly where reasoning is hardest. On Llama, chain-of-thought, plan-and-solve, self-consistency and best-of-N each jump by around nine points.
And it is consistent across backbones — a sixteen-point paradigm-averaged improvement on Llama-3.1-8B, with steady gains on Qwen and Mixtral too.
Crucially, this is not brute force. LoTR adds only about three percent more reasoning tokens overall.
Because it routes rather than re-reasons, it can even cut latency — around eleven percent on Llama, and twelve percent on Mixtral.
Ablations confirm the mechanism. Collapse the logic basis, drop the soft targets, or fall back to one global template, and the gain erodes. What matters is matching the current logic state to the right template, not generic reweighting.
The takeaway: reasoning quality is not only about the scaffold you wrap around a model. It is about whether the model uses the right internal pathway at each step. LoTR gives you that — a small bank of probes and templates that route logic on the fly. The paper, the code, and the interactive project page are linked on screen.