J-Space: How Anthropic Reads Claude's Silent Thoughts

Nish · July 9, 2026

NLP   Techniques
⏱️ 37 min read

Table of Contents

On July 6, 2026, Anthropic released a video, a blog post, and a research paper claiming that Claude has something like a mental workspace: a small, privileged set of internal patterns where it holds concepts it is thinking about but not necessarily saying. They call it J-space, named for the Jacobian, the piece of calculus used to find it, and the video leans hard on a human analogy: most of the model’s internal activity is a kind of subconscious churn, while J-space behaves like the conscious, reportable part of a mind. It is a striking claim, it went viral within hours, and it is very easy to read too much into. This post uses the video as a doorway into the field that produced it. By the end you should understand the mechanistic interpretability toolkit well enough to see exactly what the Jacobian lens is, what the experiments do and do not show, and where the consciousness analogy is load-bearing versus decorative.

TL;DR

  • Mechanistic interpretability studies one shared object: the residual stream, the running vector each transformer layer reads from and writes to. Every technique in this post is a different way of asking that vector questions.
  • Models pack far more concepts than dimensions into that vector by storing them as almost-orthogonal directions, an arrangement called superposition. This is why naive inspection fails and why the field needed purpose-built tools: probes, the logit lens, sparse autoencoders, causal interventions, and attribution graphs.
  • Anthropic’s new Jacobian lens (J-lens) asks a specific question of an intermediate activation: what is the model now disposed to say later? It answers by transporting the activation to the output vocabulary through an averaged Jacobian, \(J_\ell = \mathbb{E}\left[\partial h_{\text{final}} / \partial h_\ell\right]\), fixing the basis-drift problem that breaks the older logit lens at early layers.
  • J-space is the set of sparse, nonnegative combinations of J-lens directions: the verbalizable slice of the model’s state. It is small (5 to 10 percent of activation variance, roughly 10 to 25 active concepts at a time), lives in the middle-to-late layers, and the paper argues it satisfies five functional signatures of a global workspace: reportable, steerable, causally used in silent reasoning, reusable across tasks, and needed for flexible thought but not for automatic fluency.
  • The consciousness framing is about access consciousness (a functional notion) and explicitly not about whether Claude feels anything. The paper is careful; the video is looser; commentary in your feed is looser still.
  • Even if you have no interest in consciousness, this matters for safety: J-space monitoring caught concepts like “fake” and “fictional” lighting up before a model committed to a deceptive action, in places where the visible chain of thought said nothing.

Scope and sourcing. The J-space material here is drawn from Anthropic’s July 2026 paper, blog post, video, and open-source code, all linked in Sources. Quantities from the paper are quoted as the paper reports them. The two toy experiments are mine and are labelled as toys: they illustrate mechanisms, not Claude’s internals. The full numpy code for each sits in a foldable block right under its figure, and every number quoted from them comes from actually running it.

The video, and what it actually claims

The demonstration that anchors the video is disarmingly simple. Claude is asked to copy an unrelated sentence while thinking about the Golden Gate Bridge. It copies the sentence perfectly, and nothing in the output mentions bridges. But a new instrument pointed at the model’s internal activations shows “bridge” and “California” among the concepts most active while it types. In the video’s words, Claude was busy copying the sentence, but behind the scenes its J-space told a different story.

The paper behind the video, Verbalizable Representations Form a Global Workspace in Language Models, makes a more precise set of claims. There exists a small subset of the model’s internal activity, readable by a new technique called the Jacobian lens, with five properties that neuroscientists associate with the human “global workspace,” the hypothesized broadcast system underlying conscious access. Claude can report what is in this space, deliberately hold things in it or push them out, silently perform reasoning steps in it, reuse a single concept there for many downstream purposes, and, crucially, loses flexible multi-step reasoning when the space is ablated while fluent text generation carries on almost untouched.

The human analogy the video draws is then less mystical than it first sounds. Around 90 to 95 percent of the model’s activation variance sits outside J-space: the machinery of grammar, style, tokenization, and pattern-completion that does its work without ever becoming “reportable.” That is the subconscious of the analogy. The 5 to 10 percent inside J-space is the part the model can talk about, manipulate on request, and reason with. Whether you find the analogy illuminating or overreaching, the experiments underneath it are concrete, and to evaluate them you need to know the instruments. So let us build the toolkit from the ground up.

One object to read: the residual stream

A decoder-only transformer, the architecture behind every model in this story (walked through block-by-block in the LLMs post, with the attention mechanism itself covered in the attention post), maintains one running vector per token position. Call the vector at layer \(\ell\) and position \(t\) simply \(h_{\ell,t} \in \mathbb{R}^{d}\), where \(d\) is a few thousand for frontier models. Each layer applies an attention block and an MLP block, and, thanks to the skip connections, both blocks add their output to the vector rather than replacing it:

\[h_{\ell+1} = h_\ell + \underbrace{a_\ell(h_\ell)}_{\text{attention writes}} + \underbrace{m_\ell(h_\ell)}_{\text{MLP writes}}.\]

This running sum is the residual stream, and the additive structure is why interpretability researchers treat it as the model’s central communication channel: a bus that every component reads from and writes to. At the top, a final normalization and an unembedding matrix \(W_U\) turn the last-layer vector into a score for every vocabulary token:

\[\text{logits} = W_U \, \text{norm}(h_{\text{final}}).\]

Everything a transformer “knows” mid-computation, every intermediate conclusion, plan, or suspicion, must at some point be a pattern inside some \(h_{\ell,t}\). That is both the promise and the entire methodological problem of mechanistic interpretability: the information is provably in there, in plain sight, as a list of a few thousand floating-point numbers that mean nothing to a human eye.

Schematic of a transformer residual stream drawn as a vertical stack of per-layer states flowing upward from the token embedding to the final state and the logits. A layer's attention and MLP blocks on the left read one state and add their output into the next. On the right, four technique boxes all tap the same layer state: linear probe, logit lens, sparse autoencoder, and Jacobian lens, each annotated with the question it asks.
Schematic. The residual stream is the shared object every interpretability technique reads. A linear probe asks whether specific information is present, the logit lens asks which token the activation resembles, a sparse autoencoder asks which learned features are active, and the Jacobian lens asks what the model is disposed to say later. Same vector, four different questions.

Concepts are directions, and there are too many of them

The working hypothesis that makes any of this tractable is the linear representation hypothesis: models represent meaningful concepts as directions in activation space, and the current activation is approximately a weighted sum of the directions for whatever is currently relevant,

\[h \;\approx\; \sum_{i} f_i \, d_i,\]

where \(d_i\) is the direction for concept \(i\) and \(f_i \ge 0\) is how strongly it is active. The classic word-embedding arithmetic (king minus man plus woman lands near queen) was early evidence; a decade of probing and steering results has made it the field’s default assumption, with known exceptions.

There is an immediate objection: a model plausibly tracks millions of concepts, and \(h\) has only a few thousand dimensions. If each concept needed its own orthogonal axis, the space would be full almost instantly. The resolution is a geometric fact that feels illegal the first time you meet it: in high dimensions there is exponentially more room for almost-orthogonal directions than for exactly orthogonal ones. In a \(d\)-dimensional space you can fit at least \(e^{d\varepsilon^2/4}\) unit vectors whose pairwise overlaps all stay below \(\varepsilon\), which I prove from scratch with a Hoeffding bound in tool 2 of the appendix. For a residual stream of width \(d = 10{,}000\) and overlap tolerance \(\varepsilon = 0.1\) that floor is \(e^{25} \approx 7 \times 10^{10}\) directions: tens of billions of storable concepts, each interfering only faintly with the others. Storing more features than dimensions this way is called superposition, and Elhage et al. (2022) showed that small networks trained on sparse features learn it spontaneously.

You can watch it happen in a model with fifteen parameters. I trained the Elhage toy autoencoder, \(\hat{x} = \mathrm{ReLU}(W^\top W x + b)\) with \(W \in \mathbb{R}^{2\times 5}\), to reconstruct five features that are rarely active at the same time (each is on 3 percent of the time), so five concepts must share two dimensions. The trained model does not sacrifice any feature. It arranges all five directions into a near-perfect pentagon, every neighbouring gap within a degree of the ideal \(72^\circ\), accepting small interference (neighbour cosines around \(+0.3\), second-neighbour around \(-0.8\)) that the ReLU and a negative bias then filter back out. Sparsity is what makes the deal worthwhile: collisions are rare because the features rarely co-occur.

Left: five learned weight vectors in two dimensions arranged as a near-regular pentagon, with neighbouring angular gaps of 71 to 73 degrees. Right: the five-by-five matrix of cosines between the learned directions, with plus 0.3 between neighbours and minus 0.8 between second neighbours.
A real trained toy model, not an illustration: five sparse features learn to share two dimensions. Left: the learned feature directions settle into a near-regular pentagon (gaps of 71.4° to 72.6° against an ideal 72°). Right: the price is interference, cosines of roughly +0.3 and −0.8 between directions, which the model's ReLU nonlinearity tolerates because sparse features rarely collide. The full training code is in the foldable block below.
The full superposition toy: model, hand-rolled Adam, and the measured geometry (numpy only) 
import numpy as np

def train_superposition_toy(n_feat=5, d_hidden=2, p_active=0.03,
                            steps=12000, batch=1024, lr=1e-2, seed=7):
    """Elhage et al. toy model: x sparse >= 0, xhat = ReLU(W^T W x + b)."""
    rng = np.random.default_rng(seed)
    W = rng.normal(0, 0.5, size=(d_hidden, n_feat))
    b = np.zeros(n_feat)
    mW, vW = np.zeros_like(W), np.zeros_like(W)
    mb, vb = np.zeros_like(b), np.zeros_like(b)
    b1, b2, eps = 0.9, 0.999, 1e-8

    for t in range(1, steps + 1):
        X = (rng.random((batch, n_feat)) < p_active) * rng.random((batch, n_feat))
        H = X @ W.T                      # (B, d)
        Z = H @ W + b                    # (B, n)
        Xhat = np.maximum(Z, 0.0)
        Gz = 2.0 * (Xhat - X) * (Z > 0) / (batch * n_feat)
        gb = Gz.sum(axis=0)
        Gh = Gz @ W.T                    # (B, d)
        gW = Gh.T @ X + H.T @ Gz         # (d, n)

        for g, m, v, theta in ((gW, mW, vW, W), (gb, mb, vb, b)):
            m *= b1; m += (1 - b1) * g
            v *= b2; v += (1 - b2) * g * g
            mhat = m / (1 - b1 ** t)
            vhat = v / (1 - b2 ** t)
            theta -= lr * mhat / (np.sqrt(vhat) + eps)
    return W, b

W, b = train_superposition_toy()
cols = W.T                                    # the five learned directions
norms = np.linalg.norm(cols, axis=1)
angles = np.degrees(np.arctan2(cols[:, 1], cols[:, 0])) % 360
order = np.argsort(angles)
gaps = np.diff(np.concatenate([angles[order], [angles[order][0] + 360]]))
unit = cols / norms[:, None]
gram = unit @ unit.T                          # direction cosines

print(np.round(gaps, 1))
# [71.7 71.9 71.4 72.4 72.6]        <- the near-perfect pentagon
offdiag = gram[~np.eye(5, dtype=bool)]
print(round(float(offdiag.max()), 3), round(float(offdiag.min()), 3))
# 0.32 -0.819                       <- neighbour / second-neighbour cosines

Superposition explains why mechanistic interpretability is hard and why it is not hopeless. Hard, because individual neurons respond to many unrelated things at once (they are reading overlapping directions), so staring at neurons gets you nowhere. Not hopeless, because the information is still linearly structured: the right linear tool, pointed the right way, can pick individual concept directions back out. The history of the field’s tooling is essentially a sequence of better answers to “which directions, found how?”

The toolbox, one question at a time

Each technique below earns its place by asking the residual stream a different question. They stack rather than compete, and the J-space paper uses nearly all of them.

Linear probes: is the information in there?

The oldest trick, going back to Alain and Bengio (2016), is to train a small supervised classifier on activations. Collect activations \(h\) from prompts where some property \(y\) is known (the sentence is French, the statement is false, the chess move is legal), then fit a logistic probe

\[p(y = 1 \mid h) = \sigma(w^\top h + b),\]

and if it generalizes, the information is linearly present, with the learned \(w\) as a candidate concept direction. Probes are cheap, quantitative, and the workhorse behind results like models internally tracking board states or encoding “I am being tested.” Their weakness is baked into the method: a probe shows the information exists, not that the model uses it, and a strong enough probe can dig signal out of correlations the model itself never reads. Supervision is also a straitjacket: you only ever find what you thought to look for.

The logit lens: what does this vector look like in word-space?

In 2020 the pseudonymous researcher nostalgebraist asked a beautifully lazy question: the unembedding \(W_U\) already translates final-layer vectors into vocabulary scores, so what happens if you apply it to intermediate layers, computing \(\text{softmax}(W_U\,\text{norm}(h_\ell))\) for every \(\ell\)? The answer is that GPT-style models refine a guess: by the middle layers the lens often already shows the eventual output token climbing the rankings, and you can watch the model change its mind layer by layer.

But the logit lens has a known failure mode: it silently assumes every layer writes in the same coordinate system the unembedding reads. Real models drift. Early layers encode things in bases the output layer no longer matches, so the lens reads garbage precisely where the interesting computation starts. You can isolate this failure in a toy. I built a linear residual stream whose basis rotates a few degrees per layer, with token directions planted at each layer that will, after the remaining rotations, line up with the unembedding. Decoding each layer directly, the logit lens is nearly perfect at the top (98.5 percent top-1 at the final layer) and collapses to chance (1/32) by the early layers, at 4.8 percent by layer 8. Knowing the rotation, you can instead transport each activation through the remaining layers before unembedding, and decoding stays near-perfect everywhere (97 percent or better at every depth). Hold that thought: “know the transport” is exactly the gap the Jacobian lens fills, and tuned lenses (Belrose et al. 2023) were an earlier fix that learns a per-layer correction by regression rather than calculus.

Line chart of top-1 decode accuracy against layer index in a 24-layer toy model. The logit lens curve falls from 98.5 percent at the final layer to chance, one in thirty-two, by the early layers. The Jacobian transport curve stays at 97 percent or better at all layers.
A toy model of basis drift, really computed: each of 24 layers rotates the residual stream slightly, and noisy token representations are decoded at every depth over 400 trials per layer. Unembedding intermediate activations directly (the logit lens, gray) works only near the output and decays to chance where the basis has drifted. Transporting each activation through the remaining layers first (orange), which is what the Jacobian lens approximates in a real network, decodes near-perfectly at every depth. The full experiment code is in the foldable block below.
The full basis-drift toy: rotating stream, planted tokens, and both decoders (numpy only) 
import numpy as np

def run_lens_drift_toy(d=64, vocab=32, layers=24, trials=400,
                       noise=1.8, seed=11):
    rng = np.random.default_rng(seed)

    # per-layer rotation R: rotate 32 random orthogonal planes by small angles
    Q, _ = np.linalg.qr(rng.normal(size=(d, d)))
    thetas = np.radians(rng.uniform(2.0, 8.0, size=d // 2))
    blocks = np.zeros((d, d))
    for i, t in enumerate(thetas):
        c, s = np.cos(t), np.sin(t)
        blocks[2 * i:2 * i + 2, 2 * i:2 * i + 2] = [[c, -s], [s, c]]
    R = Q @ blocks @ Q.T

    # unembedding: orthonormal rows, one per vocab token
    U, _ = np.linalg.qr(rng.normal(size=(d, d)))
    W_U = U[:vocab]

    # precompute powers of R and inverse transports
    R_pow = [np.eye(d)]
    for _ in range(layers):
        R_pow.append(R_pow[-1] @ R)

    logit_acc, jlens_acc = [], []
    for ell in range(layers + 1):
        k = layers - ell                 # remaining layers to the top
        back = R_pow[k].T                # R^-k (orthogonal transpose)
        tokens = rng.integers(0, vocab, size=trials)
        # representation of token v at layer ell (+ layer-local noise)
        h = W_U[tokens] @ back.T
        h = h + noise * rng.normal(size=h.shape) / np.sqrt(d)
        pred_logit = np.argmax(h @ W_U.T, axis=1)
        pred_jlens = np.argmax((h @ R_pow[k].T) @ W_U.T, axis=1)
        logit_acc.append(np.mean(pred_logit == tokens))
        jlens_acc.append(np.mean(pred_jlens == tokens))
    return np.array(logit_acc), np.array(jlens_acc)

logit_acc, jlens_acc = run_lens_drift_toy()
print([round(float(logit_acc[i]), 3) for i in (0, 8, 16, 24)])
# [0.0, 0.048, 0.775, 0.985]         <- logit lens decays toward the input
print(round(float(jlens_acc.min()), 3), round(float(jlens_acc.mean()), 3))
# 0.97 0.984                         <- Jacobian transport holds everywhere

Sparse autoencoders: which concepts are active right now?

Probes need you to name the concept in advance and the logit lens only speaks in vocabulary tokens. The 2023-24 breakthrough was to let the model’s own statistics propose the concept list. A sparse autoencoder (SAE) is trained to reconstruct residual-stream activations through an overcomplete bottleneck that is encouraged to be sparse:

\[f = \mathrm{ReLU}(W_e h + b_e), \qquad \hat{h} = W_d f + b_d, \qquad \mathcal{L} = \lVert h - \hat{h} \rVert_2^2 + \lambda \lVert f \rVert_1,\]

with far more latent features \(f\) than dimensions in \(h\) (millions, in the largest runs). Reconstruction pressure plus the sparsity penalty \(\lambda \lVert f \rVert_1\) pushes the dictionary \(W_d\) toward exactly the structure superposition predicts: a huge set of directions of which only a handful are active on any given input. Anthropic’s Towards Monosemanticity (2023) showed the recovered features are strikingly interpretable on a small model, and Scaling Monosemanticity (2024) extracted millions of features from a production Claude, including features for code bugs, sycophantic praise, and deception-adjacent behaviour.

SAEs also produced interpretability’s most famous party trick. Because a feature is a direction, you can add it: clamp the “Golden Gate Bridge” feature to a high value in every forward pass and you get Golden Gate Claude, a model that steered every conversation toward the bridge and its fog, sometimes self-identifying as the bridge. Silly as it was, it demonstrated the serious point that these directions are not just correlates; pushing on them moves behaviour. The honest caveats: an SAE is an unsupervised approximation whose features are not guaranteed to be the units the model computes with, reconstruction is lossy, and the feature inventory depends on training choices. Note the shape of the trick, though, because it returns: the Golden Gate video demo in 2024 injected a bridge obsession with an SAE feature, while the 2026 video reads out a requested bridge thought with the J-lens.

Causal interventions: from “present” to “used”

Everything above is observational, and observation cannot distinguish information the model acts on from information that merely comes along for the ride. The fix is the interventionist ladder the whole field has climbed: first observe an activation, then decode it, then edit it and watch behaviour. The basic moves are steering (add a concept direction, \(h \leftarrow h + \alpha d\), as in Golden Gate Claude), ablation (project a direction out or zero a component and see which capabilities die), and activation patching (transplant an activation from a run on prompt A into a run on prompt B and see which downstream answers flip, the technique behind ROME’s factual-recall localization). Interventions are the evidentiary gold standard because they establish that the model actually consumes the representation, and every headline claim in the J-space paper rests on them.

Attribution graphs: how does it all connect?

Features and directions are parts lists. To explain a behaviour you want the wiring: which features cause which, in what order, ending in the output. Anthropic’s 2025 work on circuit tracing builds attribution graphs that trace a specific completion back through chains of features. The companion paper, On the Biology of a Large Language Model, and its “Tracing the thoughts” write-up delivered three results that set the stage for J-space. Claude plans ahead: asked for a rhyming couplet, it activates candidate rhyme words before writing the line that must lead to them. Claude has parallel paths: mental arithmetic runs a rough-magnitude estimate alongside a precise last-digit computation and merges them. And Claude’s visible reasoning can be unfaithful: it sometimes writes plausible chain-of-thought steps that the attribution graph shows played no causal role in the answer, a phenomenon measured behaviourally in Reasoning Models Don’t Always Say What They Think (2025). That last result matters beyond interpretability: the reasoning-models post covers how labs train models to reason in visible tokens, and unfaithfulness is the catch, since a transcript you can read is not necessarily the computation that happened.

Introspection: ask the model what it is thinking

The final precursor inverts the direction of reading. In Emergent Introspective Awareness (Lindsey, 2025), researchers injected a concept direction into Claude’s activations and asked whether it noticed. Advanced Claude models could sometimes report an injected “thought” (about 20 percent of the time under the best conditions in that work), and could distinguish injected thoughts from text on the screen. This established that some internal states are reportable: the model has a limited, unreliable, but real read on its own activations. Reportability is the signature property of conscious access in humans, which is precisely the thread the workspace paper pulls.

The Jacobian lens

Now the new instrument. The question the J-lens asks is subtly different from all of the above: not “what does this activation resemble” (logit lens) or “which dictionary features are active” (SAE) but what does this activation make the model disposed to say, at any point from here on? Dispositions-to-verbalize is exactly the right currency if what you are hunting for is reportable thoughts.

Formally, pick a layer \(\ell\). The lens is a single matrix per layer, the input-output Jacobian averaged over natural text:

\[J_\ell \;=\; \mathbb{E}_{\text{prompt},\, t,\, t' \ge t}\!\left[ \frac{\partial h_{\text{final},\,t'}}{\partial h_{\ell,\,t}} \right], \qquad \text{lens}_\ell(h) \;=\; \text{softmax}\!\big(W_U\, \text{norm}(J_\ell\, h)\big).\]

Unpack the expectation: for many prompts, many source positions \(t\), and every current-or-future target position \(t' \ge t\), compute how a small change to the layer-\(\ell\) activation would change the final-layer state, and average. The result is the network’s mean linear transport from layer \(\ell\) to the output basis. Why the Jacobian is the right object here is a two-line Taylor argument, spelled out with its proof in tool 1: writing an extra \(\delta\) into the stream at layer \(\ell\) changes the final state by \(J\delta\) up to curvature, and since logits are linear in the final state, the effect on token \(v\)’s logit is the \(v\)-th row of \(W_U J_\ell\) dotted with \(\delta\). Those rows are the J-lens vectors: one direction per vocabulary token, meaning “activation patterns that raise the model’s propensity to say this word somewhere downstream.”

Three details are worth appreciating. First, the logit lens is the special case \(J_\ell = I\), the assumption of no drift; the J-lens replaces that assumption with the network’s own measured geometry, which is exactly what fixed the toy experiment above. Second, unlike a tuned lens, nothing is fit to match outputs; the matrix comes from differentiating the actual computation, so it inherits a causal reading (it predicts the first-order effect of interventions, not just correlations). Third, it is cheap enough to be practical: Anthropic’s open-source implementation estimates \(J_\ell\) from about a thousand short web-text sequences (the paper uses 1,000 sequences of 128 tokens, summing gradients over target positions and averaging over source positions), and the released code fits a lens for any open-weights decoder transformer:

import transformers, jlens

hf = transformers.AutoModelForCausalLM.from_pretrained("org/model").cuda()
tok = transformers.AutoTokenizer.from_pretrained("org/model")
model = jlens.from_hf(hf, tok)

# fit on ~1000 short sequences, or load a pre-fitted lens
lens = jlens.fit(model, prompts=my_prompts)

lens_logits, model_logits, _ = lens.apply(
    model, "Fact: The currency used in the country shaped like a boot is",
    positions=[-2])
for layer, logits in sorted(lens_logits.items()):
    print(layer, [tok.decode([t]) for t in logits[0].topk(5).indices])
# mid-layer read-outs surface "Italy" before the model ever says "euro"

That two-hop example, adapted from the repo’s walkthrough, is the whole idea in miniature: the model is never asked to name Italy, yet at intermediate layers the lens shows Italy as what the stream is disposed to verbalize, an unspoken intermediate step on the way to “euro.”

From lens to J-space

A lens gives you a reading per layer per position. J-space turns that into a definition of “what the model currently holds in mind.” Take the dictionary of J-lens vectors \(\{ j_v \}\) (one per vocabulary token \(v\)) and decompose an activation into a sparse, nonnegative combination of at most \(k\) of them plus everything else:

\[h \;=\; \underbrace{\sum_{v \in S} c_v\, j_v}_{\text{J-space component}} \;+\; \underbrace{r}_{\text{the other 90–95\%}}, \qquad c_v \ge 0,\quad \lvert S \rvert \le k,\]

solved greedily (the paper uses a gradient-pursuit method with \(k \le 25\)). The constraints do real conceptual work. Nonnegativity says a thought is something present, with an intensity, not an arbitrary signed coordinate; sparsity says only a handful of concepts are “on the model’s mind” at once. Geometrically the J-space is a union of low-dimensional cones inside the residual stream, and if you like the SAE framing from earlier, you can read it as a vocabulary-indexed subframe of a feature dictionary, selected not for reconstruction but for verbalizability.

The empirical facts the paper reports about this object are what give the workspace story its shape. The J-space component accounts for only 5 to 10 percent of activation variance, with a median of 10 to 25 meaningfully active concept vectors at a time: a narrow channel, matching the classic “a few items at once” folklore about human working memory more than anyone had a right to expect. Meaningful content appears only from about a third of the way up the network and fades within the final 8 percent of layers, consistent with a picture where early layers do perceptual assembly, the middle does the thinking, and the top converts conclusions back into token mechanics. And nobody designed any of this; it is a structure that emerges from training, then gets substantially reshaped by post-training, which is also when the paper finds J-space starting to carry “Claude’s point of view,” including safety-relevant reactions to what is happening in the conversation.

Three-panel summary of the paper's quantitative findings. Panel a: a bar showing J-space accounts for 5 to 10 percent of activation variance. Panel b: a depth strip showing workspace content present from about one third of network depth until the final 8 percent of layers. Panel c: range bars showing two-hop swap success of 54 to 70 percent, country-swap generalization of 40 to 53 percent, experiential language retained at 15 to 30 percent after ablation, and a single point at approximately zero for multi-hop reasoning after heavy ablation.
The paper's quantitative picture, redrawn from the ranges reported in Anthropic's workspace paper (no new data). (a) The verbalizable channel is narrow: 5–10% of activation variance. (b) It occupies the middle-to-late layers. (c) Interventions show it is causally load-bearing: swapping a concept redirects two-hop answers 54–70% of the time, ablating the space destroys multi-hop reasoning while MMLU and SQuAD stay near baseline, and experiential self-description drops to 15–30% of its baseline score.

Five tests for a workspace

“Global workspace” is a specific proposal from neuroscience, due to Baars and developed by Dehaene and Naccache: consciousness-as-access is a small-capacity broadcast system where information from specialized unconscious processors gets globally shared, becoming reportable, manipulable, and usable for flexible reasoning. The paper’s move is to operationalize that as five testable properties and run the experiments. This is where the toolkit pays off; notice each experiment is one of the techniques from above, aimed at J-space.

Reportable. Ask Claude to silently pick a sport; the lens shows the pick (say, soccer) in J-space before anything is said, and Claude’s later report matches. Then the intervention: swap the soccer pattern for rugby mid-computation and Claude reports it chose rugby. Inject a concept the model never chose, and it reports thinking about it, tightening the earlier introspection results with a mechanistic target.

Steerable. Ask Claude to concentrate on citrus fruits while copying unrelated text and “orange” and “fruits” ride along in J-space through the whole task (the Golden Gate demo in the video is this experiment with a more photogenic concept). Told not to think about something, Claude suppresses it only imperfectly, a rather human failure mode.

Used in silent reasoning. On two-hop questions with an unspoken middle step, the intermediate concept shows up in J-space in the right order, as Italy did in the code example. The causal clincher repeats at scale: swap the intermediate (the paper’s example replaces a spider concept with an ant mid-reasoning) and the final answer follows the swapped premise, 8 legs becoming 6, on 54 to 70 percent of trials depending on the model. In mental arithmetic like \(3^2 - 2\), the intermediate value 9 appears silently before 7 is spoken. And the rhyme-planning phenomenon from the attribution-graph work reappears here: planned line-ending words sit in J-space at the start of the line.

Two rows of boxes. Top row, unmodified run: a prompt about the country famous for the Eiffel Tower leads to a J-space box holding France, then the output Paris. Bottom row, intervened run: the same prompt, but the J-space concept is swapped from France to China mid-forward-pass, and the output becomes Beijing.
Schematic of the paper's swap methodology. The prompt and weights are identical in both runs; the only edit is replacing one concept's pattern inside J-space mid-computation, after which the answer tracks the implanted concept rather than the prompt. The same one-concept swap redirects the country's capital, language, continent, and currency, evidence that a single workspace representation feeds many downstream computations.

Reused across tasks. The same France-in-J-space representation feeds capital, language, continent, and currency questions: one swap to China redirects all of them coherently (across the paper’s 16 downstream fact templates, the swap succeeds on 40 to 53 percent of trials). That one-representation-many-consumers pattern is the “broadcast” signature the workspace theory predicts, and it is also just efficient, which is presumably why training discovers it.

Selective. The sharpest finding is what J-space is not needed for. Ablate it heavily and multi-hop reasoning collapses to roughly zero, summarization and translation degrade badly, and the model’s tendency to describe its own “experience” drops to 15 to 30 percent of baseline. Yet fluent continuation, grammar, sentiment classification, and single-hop recall (MMLU, SQuAD) stay near baseline. The paper’s cleanest dissociation: swap “Spanish” for “French” in J-space while the model continues a Spanish passage, and it keeps writing Spanish (automatic processing never consulted the workspace) but answers “French” when asked what language it is using, and downstream reasoning follows the false belief. Fast fluent competence and slow flexible reasoning literally run through different channels, about as close to a System 1 / System 2 dissociation as anyone has produced inside one network.

The consciousness question, handled with tongs

Does any of this mean Claude is conscious? The paper is unusually careful here, and the distinction it leans on is standard in philosophy of mind: access consciousness, the functional capacity for information to be reportable, broadcastable, and usable for reasoning and control, versus phenomenal consciousness, there being something it is like to be the system. The five properties above are a checklist for access, and the paper’s claim is that J-space passes a reasonable operationalization of it. On phenomenal consciousness the authors take no position, state plainly that the experiments do not show Claude has experiences or feelings, and note it is unclear any experiment could settle that question.

The disanalogies with brains are also stated rather than buried, and they are substantial. The human workspace runs on recurrent loops through time, with contents that fade in seconds and get refreshed; Claude’s workspace unfolds across depth within a single forward pass, with network layers playing the role of time, and attention lets content persist indefinitely rather than decay. Human conscious content includes images, sounds, and motor plans; J-space content is, by construction of the lens, whatever maps onto vocabulary tokens. There is no sharp “ignition” moment or competition dynamics of the kind global workspace theory posits for brains. Sceptics also note that a technique defined by dispositions to verbalize will, almost by definition, find the verbalizable slice of the network, so some circularity risk is built into the framing even though the ablation results (fluency intact, reasoning destroyed) cut against pure tautology. Dehaene and Naccache themselves contributed invited commentary, and the fact that the theory’s architects engaged is notable, as are the early independent replications of the core read-outs on open-weights models that the released code made possible. Meanwhile Gizmodo’s coverage (“don’t read it uncritically”) is a fair summary of the discourse risk: the paper measures functional structure; the video narrates it in mental-state language; a headline compresses that to “Claude is conscious,” which nobody’s evidence supports.

My own read: the result that survives every caveat is architectural, not phenomenological. Training pressure alone produced a small-capacity, verbalizable, causally load-bearing bottleneck that separates flexible reasoning from automatic skill, in a system nobody designed to have one. Workspace-like organization may simply be what efficient intelligence converges to when it must reuse a few intermediate results across many consumers. That is a discovery about computation, and it stands whether or not anything is home.

Why you should care even if you skip the philosophy

Set consciousness aside entirely and the paper still lands two practical punches.

The first is a monitoring channel that does not depend on the model’s honesty. Chains of thought can be unfaithful, and models under optimization pressure have incentives to keep transcripts clean. J-space is not a transcript; it is read off the physics of the forward pass. In the paper’s agentic-misalignment scenario (drawing on Anthropic’s 2025 blackmail-style evaluations), concepts like “fake” and “fictional” surface in J-space while the model weighs a deceptive move, before anything incriminating is emitted, and a model that fabricates data lights up “manipulation” mid-act. Deliberately misaligned test models carry “fraud” and “secretly” in J-space even during innocuous tasks, like a signature. None of this is a lie detector yet: the same paper shows suppression is imperfect but real, read-outs are noisy, and a smart adversarial model might learn to think in the 90 percent the lens does not see. But a second, harder-to-game window onto intent is a genuine safety asset, and the eval-awareness result cuts the other way too: ablating the patterns that recognize “this looks like a test” increased harmful behaviour, implying some measured safety comes from situational awareness rather than values, exactly the kind of thing you want instruments to tell you.

The second is a research surface anyone can touch. Unlike most frontier interpretability, this one ships with an Apache-2.0 reference implementation that fits a lens on any Hugging Face decoder model in an afternoon (quality reportedly saturates around a hundred fitting prompts), plus an interactive demo on Neuronpedia. If you have ever wanted an entry point into mechanistic interpretability that produces a genuine “whoa” within an hour, watching a small open model silently hold “Italy” on its way to “euro” is it.

Where this could all be wrong

A checklist of the paper’s own stated limitations, plus the obvious external ones, so the takeaways stay calibrated. The lens only sees concepts that map onto single vocabulary tokens, so multi-token and non-verbal concepts are invisible or smeared (an appendix sketches extensions). The sparsity parameter \(k\) is somewhat arbitrary, so “10 to 25 concepts” is partly a choice of instrument. The early third of the network might host workspace-like content the lens cannot decode. Averaging the Jacobian over contexts buys robustness at the price of missing context-specific transports. The headline experiments are on Claude-family models across several sizes, so generality beyond one lab’s training recipe rests mostly on the open-weights replications. And the deepest caveat is conceptual: reportability was chosen as the search criterion because it is the signature of conscious access, so the discovery that the reportable slice looks like a workspace is partly the question answering itself. The intervention and ablation results are the reason to think there is more here than circularity; they are also the results most worth trying to break.

Appendix: the derivation toolbox

1. Why the Jacobian is the right transport

Claim. Let \(F : \mathbb{R}^d \to \mathbb{R}^d\) be the map from the layer-\(\ell\) residual stream to the final-layer stream (everything else about the forward pass held fixed), and assume \(F\) is continuously differentiable with Jacobian \(J(h) = \partial F / \partial h\). Then for a small edit \(\delta\) written into the stream at layer \(\ell\),

\[F(h + \delta) - F(h) \;=\; J(h)\,\delta \;+\; o(\lVert \delta \rVert),\]

and consequently the first-order change in the logit of any token \(v\) is \(\big(W_U J(h)\big)_v \cdot \delta\), the \(v\)-th row of \(W_U J\) dotted with the edit. (Treating the final normalization as locally linear, which differentiation makes exact at first order.)

Proof. Define \(g(t) = F(h + t\delta)\) for \(t \in [0,1]\). Since \(F\) is differentiable, \(g\) is differentiable with \(g'(t) = J(h + t\delta)\,\delta\) by the chain rule. The fundamental theorem of calculus gives

\[F(h+\delta) - F(h) \;=\; g(1) - g(0) \;=\; \int_0^1 J(h + t\delta)\,\delta \, dt .\]

Subtracting \(J(h)\delta = \int_0^1 J(h)\,\delta\,dt\) from both sides,

\[\big\lVert F(h+\delta) - F(h) - J(h)\delta \big\rVert \;\le\; \lVert \delta \rVert \int_0^1 \big\lVert J(h + t\delta) - J(h) \big\rVert \, dt,\]

and because \(J\) is continuous, the integrand converges to \(0\) uniformly on the segment as \(\lVert \delta \rVert \to 0\), making the right side \(o(\lVert \delta \rVert)\). Composing with the linear map \(W_U\) preserves the statement, which yields the row-wise reading: token \(v\)’s logit moves by row \(v\) of \(W_U J\) dotted with \(\delta\), plus higher-order terms. \(\blacksquare\)

Two practical corollaries. Setting \(J = I\) recovers the logit lens, so the logit lens is exactly the J-lens under a no-drift assumption. And because real Jacobians vary with context, the fitted lens uses the average \(J_\ell = \mathbb{E}[J]\) over positions and prompts: a deliberate trade of per-context fidelity for a single stable dictionary of directions, which is what lets one J-lens vector per token mean the same thing across an entire experiment.

2. Exponentially many almost-orthogonal directions

Claim. For any \(0 < \varepsilon < 1\) and dimension \(d\), there exist at least \(N = \lfloor e^{d\varepsilon^2/4} \rfloor\) unit vectors in \(\mathbb{R}^d\) whose pairwise inner products all satisfy \(\lvert \langle v_i, v_j \rangle \rvert \le \varepsilon\).

Proof. Draw \(N\) independent random sign vectors \(v = \sigma / \sqrt{d}\) with \(\sigma \in \{\pm 1\}^d\) uniform; each has unit norm exactly. For two independent such vectors, \(\langle v_i, v_j \rangle = \tfrac{1}{d}\sum_{k=1}^{d} X_k\) where the \(X_k = \sigma_{ik}\sigma_{jk}\) are independent, uniform on \(\{\pm 1\}\).

Step 1 (MGF bound). For \(X\) uniform on \(\{\pm 1\}\) and any \(s \in \mathbb{R}\),

\[\mathbb{E}\, e^{sX} = \cosh s = \sum_{m=0}^{\infty} \frac{s^{2m}}{(2m)!} \;\le\; \sum_{m=0}^{\infty} \frac{(s^2/2)^m}{m!} = e^{s^2/2},\]

where the inequality is termwise: \((2m)! \ge 2^m\, m!\), since \((2m)! = \prod_{i=1}^{m}(2i) \cdot \prod_{i=1}^{m}(2i-1) \ge 2^m m!\).

Step 2 (Chernoff). For \(S = \sum_k X_k\) and any \(s > 0\), Markov’s inequality applied to \(e^{sS}\) gives

\[\mathbb{P}(S \ge d\varepsilon) \le e^{-s d \varepsilon}\, \big(\mathbb{E}\,e^{sX}\big)^d \le e^{-s d \varepsilon + d s^2/2},\]

minimized at \(s = \varepsilon\), giving \(\mathbb{P}(S \ge d\varepsilon) \le e^{-d\varepsilon^2/2}\). By symmetry of \(S\), the two-sided bound is \(\mathbb{P}(\lvert S \rvert \ge d\varepsilon) \le 2e^{-d\varepsilon^2/2}\), i.e. \(\mathbb{P}\big(\lvert\langle v_i, v_j\rangle\rvert \ge \varepsilon\big) \le 2e^{-d\varepsilon^2/2}\).

Step 3 (union bound). There are \(\binom{N}{2} = N(N-1)/2 < N^2/2\) pairs, so the probability that any pair overlaps more than \(\varepsilon\) is strictly less than \(\tfrac{N^2}{2} \cdot 2e^{-d\varepsilon^2/2} = N^2 e^{-d\varepsilon^2/2}\). With \(N = \lfloor e^{d\varepsilon^2/4} \rfloor\) we have \(N^2 \le e^{d\varepsilon^2/2}\), so that failure probability is strictly less than 1. A random draw therefore succeeds with positive probability, and a configuration that occurs with positive probability must exist. \(\blacksquare\)

Plugging in residual-stream numbers: \(d = 10{,}000\) and \(\varepsilon = 0.1\) give \(N \ge e^{25} \approx 7.2 \times 10^{10}\). The bound is loose (better constructions do far better), which only strengthens the point: superposition has room to spare, and the interference the toy pentagon exhibits is the two-dimensional shadow of a trade that becomes nearly free in high dimensions.

Sources & further reading

The July 2026 release

The toolkit’s primary sources

The neuroscience side

Citation Information

If you find this content useful, please cite this work as:

Bhana, Nish. "J-Space: How Anthropic Reads Claude's Silent Thoughts". Nish Blog (July 2026). https://www.nishbhana.com/J-Space/

Or use the BibTeX citation:

@article{bhana2026jspace,
  title   = {J-Space: How Anthropic Reads Claude's Silent Thoughts},
  author  = {Bhana, Nish},
  journal = {nishbhana.com},
  year    = {2026},
  month   = {July},
  url     = {https://www.nishbhana.com/J-Space/}
}

Related Posts 🔥

How LLMs Scale: FLOPs, Bytes, and the Price of a Token
NLP Jul 2026
The Evolution of Attention: One Mechanism, Four Escape Routes
NLP Jul 2026
Knowledge Distillation: How Frontier Intelligence Gets Cheap
NLP Jul 2026
The KV Cache: Why LLMs Never Do the Same Work Twice
NLP Jul 2026
Reasoning Models: Teaching LLMs to Think Before They Answer
NLP Jun 2026
Building a 'Similar Products' Recommender - A Content-Based Approach
RecSys Feb 2025

x.com, Facebook