Let's understand State of Thought, or SoT —
a way to let a language model steer its own reasoning from the inside,
built up from first principles.
First, how models reason today,
and why that reasoning is controlled from the outside.
A model solves a problem step by step.
Each step adds a new piece of thinking to a growing chain of evidence.
To make it reason better, we usually bolt on external control.
One line fixes the format.
Think step by step, then plan, then revise —
a fixed script the model is forced to follow.
Another line widens the search.
Sample many chains, branch into trees, explore more trajectories,
and then pick a winner.
Both work, but both steer from outside — a rigid script or a search policy —
blind to what the model is actually doing inside.
So the control is brittle across different kinds of problems, and expensive,
because every extra chain costs more tokens and more time.
State of Thought changes the control variable.
The steering now comes from inside the model itself.
At each step,
SoT reads a compact state from the model's own internal information transfer.
Four numbers: how concentrated the internal structure is,
how much progress the step made, whether the direction stays consistent,
and how uncertain the model is.
Together they form a dynamics-geometric state —
a compact fingerprint of where the reasoning is, right now.
Plotted across many trajectories,
scripted methods collapse into fixed regions of this state space,
while SoT spans a broader, more adaptive range.
The state then drives two decisions.
First: which past evidence to keep active.
Instead of carrying the whole history,
SoT keeps only the sparse subset that matches the current state.
Second: when to stop.
When the state signals that enough support has accumulated,
SoT ends reasoning and commits to an answer.
That closes the loop.
State selects evidence, evidence conditions the next step,
and the new step updates the state — with no external script imposed.
And the controller is lightweight.
The backbone stays frozen; only the tiny select-and-stop interface is trained.
Now the evidence.
Four backbones, twenty datasets, five families of reasoning.
On Llama-3.1-8B, SoT leads every domain average — quantitative,
general understanding, symbolic and code, and long-context reasoning.
The gains are largest exactly where fixed control struggles most:
long-context and symbolic reasoning.
And it reaches those numbers while spending less:
sixty-nine percent fewer generated tokens, and forty-nine percent lower latency.
On the accuracy-versus-cost frontier,
search-heavy methods drift into the expensive corner,
while SoT sits at high quality and low cost.
Why does trimming evidence help, instead of hurting?
Because state-matched context is exactly the context the next step actually needs.
Clustering the states makes it visible:
each reasoning regime maps to its own stable pattern of what to keep,
and when to stop.
In a single case you can watch it keep only the live sub-results,
drop the stale ones, and raise its stop-readiness as the answer locks in.
Does it depend on the full recipe?
Strip the training away and use fixed rules — still competitive.
Hide the internals and drive it from text embeddings alone — still competitive.
It transfers across architectures — dense, mixture-of-experts,
and a vision-language model —
and even to a black-box trajectory judge that agrees with correctness eighty-four percent of the time.
The advantage is the principle, not any single implementation.
So reasoning need not rely on ever more external scaffolding.
Adaptivity and efficiency can emerge from the model's own internal state.
Code, an interactive page, and the paper are linked on screen.