LoTR: Logic-of-Thought Routing for Plug-and-Play Reasoning of LLMs

Reasoning paradigms prescribe external strategies — step-by-step, iterative revision, search-and-sampling — that guide how a model reasons. They rarely consider whether the model's internal computation pathway is well matched to the dynamic reasoning-logic state induced by the strategy, leaving part of each paradigm's potential unrealized.

We propose Logic-of-Thought Routing (LoTR), a plug-and-play module that strengthens existing paradigms through logic-conditioned internal pathway routing: it identifies dynamic logic states from the model's internal information transfer, couples them with attention-head pathway patterns, and uses a lightweight probe to infer the current logic-state mixture and softly route the corresponding pathway.

Across three backbones, ten benchmarks, and eight reasoning paradigms, LoTR improves model-level matched macros by 3.60% relative on average. On the primary eight-benchmark, eight-paradigm Llama study it gains 5.37% with only +0.15% total completion tokens, and it outperforms the mean of four plug-in intervention baselines on all three comparison panels — all with frozen backbone weights.

Online Plug-in Routing

As reasoning unfolds, LoTR evaluates the probes only at sentence boundaries. It composes the pathway for the next step by mixing the cached state templates according to the probe activations, applying the resulting soft gates directly to attention-head outputs. The external prompt or search procedure is left completely unchanged — only the internal execution pathway adapts, step by step, to the logic the step actually requires.

Method-Level Insight

• Logic states are recurring, probe-identifiable execution regimes inferred from sentence-level internal transfer — not static labels.
• State readout stays non-exclusive (sigmoid, not simplex), so several execution regimes can co-activate within one step.
• The online controller is lightweight: one probe evaluation per layer at each sentence boundary, then a weighted mix over cached templates.

Llama-3.1-8B Paradigm Macros (Accuracy %, benchmarks weighted equally)

Improves 44 of 64 cells and ties 7 more (51/64 helped-or-tied); every benchmark-level macro improves.

Llama-3.1-8B Full Benchmark Table (Identity vs LoTR)

Metrics are each benchmark's native score: accuracy (BoolQ, QASC, ScienceQA-text, MMLU, FOLIO), token-level F1 (DROP, NarrativeQA), and Pass@1 (HumanEval).

Click to Explore More Experimental Blocks

Token Efficiency on the Primary Llama Panel

Values are completion tokens, macro-averaged over eight datasets and eight paradigms.

Interpretation

• LoTR changes internal routing without adding sampling or search calls: accuracy rises +2.66 points while total completion tokens move only +0.15%.
• The gain therefore comes from logic-indexed pathway reuse, not from a larger decoding budget or brute-force expansion.
• Wall-clock timings are excluded because measurements come from heterogeneous shared experiment servers; tokens are the controlled decoding-work proxy.

Aligned Mechanism Ablation (Llama-3.1-8B)

Averaged over Vanilla, CoT, SC, and Best-of-N on matched items. Full LoTR improves the panel macro by +3.12 over Identity and +2.87 over a true single-state controller — the gain is not recovered by one globally routed template.

Ablation Insight

• State diversity is load-bearing: a distinguishable six-state basis keeps the pathway templates discriminative.
• Conditioning on the inferred state is what carries the gains — steering without reading the state fades to a generic tweak.
• LoTR is not knife-edge tuned: it stays positive across a broad probe-scale range and different state counts.

Under the same external wrapper, LoTR keeps step-level constraint verification and avoids premature answer commitment. The trajectory shows that improvements come from better internal pathway alignment, not from changing the external reasoning script itself.

Case Interpretation

LoTR does not replace the external wrapper; it reduces internal pathway mismatch. In practice this means fewer premature commitments and better late-step constraint verification under the same prompt scaffold.