One Mechanism for Many Mental Spaces: A Shared Router over a Value Slot in Language Models

The setup

A model has to hold a base-world value and an alternative-space value for the same entity apart: a flower that's red in reality and purple in a painting, mentioned in the same sentence, queried later without the words “painting” or “reality” appearing again. Nine different builders open the alternative space: a painting, a belief, a dream, a movie, the past, yesterday, a hypothetical, a modal possibility, reported speech.¹

Both values stay decodable from the model's residual stream at a later mention, across every builder type, generalizing across which entity, which attribute, and which kind of entity is involved.² That's a necessary starting point, but decodability alone doesn't say whether one mechanism handles all nine builders or nine separate ones happen to leave a similar trace.

Coexisting values share one code

The binding is re-read from the source context each time it's queried, decaying like a retrieval effect, not a persistent register set once and forgotten.³ And it's low-rank: the per-builder-type directions live in a shared subspace (a handful of directions in the model's hidden state) of effective rank ~4–5 out of 10 possible dimensions, with one component carrying about 40% of the variance across builder types.

The space code also composes additively with entity identity (R² ≈ 0.92–0.95; the two factors nearly orthogonal, i.e. independent) and reaches the model's actual output: both the base and alternative values rise above chance through the late layers in all three families, written by a handful of late-layer heads. The heads that write the alternative-space value and the heads that write the base-world value don't overlap at all.⁴

Real: the writing layers (Qwen2.5-3B L31–33) and the 0-of-8 mover-set overlap are the paper's fig-dla direct-logit-attribution result. Schematic: which specific head index is a top mover is a drawing of “8 heads per layer, two disjoint sets,” not the exact per-head attribution.

Rank is what makes it real

Hand-built directions (a simple difference between alt-space and reality activations, or steering along it) sit right at, or inside, the band a matched random direction reaches by chance. A learned subspace is what actually controls the readout, and how much rank it needs is itself a measured, swept quantity.

Real: the flip rates (random / hand-built rank-1 stuck at 0.583; learned rank-1 0.883, rank-2 0.967, rank-4+ 1.000) and cross-type transfer indices (0.71–0.885, bar 0.70) are the paper's tab-das-rank and tab-transfer-index, at layer L34 on Qwen2.5-3B. Schematic: the circuit layout, the direction-arrows in the space index, and the traveling slice are a drawing of the interchange, not a readout of activations.

A random or hand-built rank-1 direction gets stuck at 0.583, the leakage floor. A learned rank-1 subspace already reaches 0.883; rank-2 reaches 0.967; rank-4 and up saturate at 1.000.⁵ Full control needs a subspace, not just a direction, and the subspace it needs is small.

That same rank result is also what makes the cross-type generality claim from the overview page meaningful: a rank-8 subspace trained on one builder type transfers to control an entirely different one at a transfer index of 0.71–0.89 across the three families, and rank-4 already gets within 0.06 of rank-8's transfer, so the shared mechanism itself is compact, not merely present somewhere in a big subspace.

What's said vs. what's used

If a model reports one color but derives a rule-based conclusion from it, are those the same internal value read out twice, or two things a single subspace happens to move together?

They're separable. In a single-space color-to-recipient sorting task, a subspace trained on the reported color fully flips the report (1.00) while leaving the rule-derived consequent almost untouched (0.00); a subspace trained on the consequent does the reverse.⁶ The two subspaces are nearly orthogonal. A model's stated answer and what it actually reasoned from can be moved independently, a mechanistic foothold for why a model's explanation of its own reasoning needn't match what drove the answer.

A taxonomy, but no special belief

Formal semantics predicts belief should be a distinct case: a different logic from counterfactual or fictional reference. The shared subspace does recover a coarse taxonomy: builder types that formally group together (attitude reports, intensional modals, temporal reference) transfer a little better within their group than across groups.⁷

But belief doesn't get its own boundary beyond that generic within-class effect. The mechanism does recover a semantic line; the line it uses simply groups belief with every other non-actual context rather than setting it apart. That reading is the paper's answer to the introduction's linguistic question: Fauconnier's unification, not the semanticist's separation, describes what the causal machinery is doing.

The belief companion

Belief earns its own paper because it carries independent philosophical and psychological weight (theory of mind, propositional attitudes) that this paper's cross-type sweep doesn't have room to develop, not because the mechanism needs a special case. The companion goes deep on exactly how belief fills its slot: directly when asserted, by a visibility-gated lookback when it has to be derived from what a character could see.

← Mental spaces overview