Let's take a guided tour of XDomainBench, a diagnostic benchmark for how large language models reason across scientific disciplines, in realistic, multi-turn conversations.
First, the problem. Modern science rarely stays inside a single field.
Designing a new material means combining analytical chemistry, solid-state physics, and a cost estimate from finance, all at the same time.
We would love language models to act as research assistants that fuse these disciplines on the fly.
But real scientific work is interactive. A question leads to an answer, which shapes the next question, over many turns.
Most benchmarks don't test this. They ask isolated, single-domain, one-shot questions.
So when a model fails, we can't tell why. Is it missing the knowledge, or can it simply not combine what it already knows?
XDomainBench is built to answer that question. It turns interdisciplinary reasoning into something we can control and measure, along two axes.
The first axis is composition order: how many distinct domains a session draws on, from one, to two, to three, and four.
The second is mixture structure: how strongly each turn leans on each domain, and how that balance shifts as the conversation evolves.
From twenty scientific domains, organized into six interdisciplinary families, we build meaningful combinations, not random ones.
Domains are paired by semantic similarity, so chemistry meets materials science, rather than something arbitrary.
The result is a large benchmark: eighty-five hundred interactive sessions, across twenty domains and four task types, multiple choice, factual recall, step-by-step reasoning, and code.
Each session is more than a pile of questions. It follows a designed trajectory.
Difficulty can stay stable, gradually rise, gradually fall, spike, or fluctuate across the turns of a conversation.
And in cross-domain sessions, the mixture of domains can hold steady, gradually shift, or wander from turn to turn.
Every session is generated under a target configuration, validated by three independent models, and checked by humans. Construction is kept separate from evaluation, so we measure reasoning, not imitation.
Now, the evidence. We evaluate twelve models, large, small, and mixture-of-experts, under one standardized, history-aware protocol.
The headline result is a systematic reasoning collapse. As composition order grows from one domain to four, average session success falls from thirty-nine percent to twenty-seven.
And the decline is non-linear. The drop grows steeper at higher orders, as combining disciplines compounds, rather than simply adds.
Model families differ. Small models degrade fastest, large models hold up better, and mixture-of-experts models stay strongest at every order, yet even they lose nearly half of their session success.
This is not a token-matching artifact. A semantic judge, scoring meaning rather than exact words, shows the very same downward trend.
So why does this happen? We trace the collapse to two mechanisms.
The direct mechanism appears immediately. Even at the very first turn, combining domains raises difficulty and lowers accuracy, before any conversation history has built up.
The indirect mechanism unfolds over the conversation, where certain trajectory patterns amplify small mistakes into failure.
We see three signatures: error accumulation, where performance decays after a stumble; reasoning breaks, sudden drops that contaminate later turns; and domain confusion, where the model leans on the wrong discipline.
Volatile trajectories, spikes and fluctuations, trigger these failures most often, and they cascade into a full session collapse.
The takeaway. Scaling alone does not fix compositional reasoning.
Models need to handle domain mixtures and hold their reasoning together across turns, and XDomainBench gives a controllable testbed to measure exactly that.
Explore the dataset, the code, and the full paper through the links below. Thanks for watching.