Watching a language model get the joke

The question

A pun works only if both readings of a word are in the room at once - Koestler's "two strings tied by an acoustic knot." Human data say both meanings of an ambiguous word are accessed within a few hundred milliseconds and the irrelevant one is then pruned; a pun is a sentence that will not let the pruning finish. Does a language model do the same thing? Until recently that was answered from the outside, by whether the model explained the joke. The J++ Lens (Ayonrinde & Lindsey, 2026) makes it answerable from the inside: at any layer, at any token, it prints a ranked list of the words the model is disposed to produce later. Three questions, each with a control:

• Reading. At the pun word, are both senses in the readout? Compared with the same word in a one-sense sentence, and with the pun's setup minus the pun word.
• Using. If one sense's directions are projected out of the residual stream, does the model's judgement of the primary sense move? Compared with projecting out random directions.
• Timing. When does a riddle's answer appear - while reading, at the answer slot, or during a chain of thought? Compared with pre-registered wrong candidates and an unrelated word.

Setup

• Lens. J++ lenses released by the paper's authors (koayon/jpp-lenses): at layer l the readout is unembed(Jl · hl), with Jl a corpus-averaged, noise-filtered Jacobian from the layer-l residual to the final layer. Non-semantic tokens (punctuation, whitespace, control tokens) are masked from every ranking. Rank 1 = top of the vocabulary (~150k–260k tokens); the paper's hit criterion is top-10, we also report top-20.
• Models. Qwen3.5-9B (32 layers; hybrid GatedDeltaNet + attention; one A6000; bare-text prompts); Qwen3.6-27B (64 layers; same family, 2× depth; two A6000s; bare-text prompts); Gemma-4-31B-it (60 layers; different family; lens fitted on the base checkpoint; sentences inside a user turn). Same item sets and scripts for all three; layer windows scaled to depth (e.g. "late" = the last quarter).
• Puns. 20 puns ending on the ambiguous word, each with two five-word proxy clusters per sense written before any results (e.g. interest: money/bank/rate/payments/loan vs bored/curiosity/attention/motivation/enthusiasm), plus three controls: the word in a one-sense sentence for each sense, and a swap sentence (setup with the pun word replaced). Measure: best rank of each cluster at the final token over the late layers.
• Clamp. Eight puns. One sense's five proxy lens-vectors projected out of the residual over the middle ~40% of depth, then a forced choice ("which meaning does the word carry in the primary reading - A or B? One word") read as the log-odds of the two labels at the first token. Controls: five random tokens' vectors; and two stronger variants that also remove the pun word's own vector.
• Riddles. Five riddles each paired with a direct question sharing the answer (rank at the question mark, before generation); sixteen riddles with thinking on, tracking the answer, two pre-registered wrong candidates and one unrelated word (another riddle's answer) through the chain. Hits = top-10 at content positions (whitespace and indent tokens are geometric outliers and are excluded).
• Hardware. Two RTX A6000s. Models over 40 GB are pipeline-split across both cards; see the asymmetries callout for a hardware bug this surfaced.

1. At the pun word, both senses - but one arrives late

Qwen3.6-27B. Best lens rank of each sense cluster at the ambiguous word (layers 48–60), 20 puns. Left: the pun; middle: the same word in a one-sense sentence (the off-sense never rises); right: the setup with the pun word replaced.

In every model the second reading is roughly an order of magnitude more present at a pun word than at the same word in a plain sentence, and the setup alone does not produce it. The strict both-in-top-20 count is highest on the 27B (8/20) and lowest on Gemma (3/20) - the latter for the proxy-readability reason above; Gemma's pun/control gap (82 vs 1154) is the widest of the three.

Qwen3.6-27B. Rank of each sense at the pun word across all 64 layers for four puns (solid), with one-sense controls (dotted) and swap (dashed). The setup-primed sense is in range by layer 5–20; the second sense arrives at layers 36–55, and in two of four the first recedes as it does.

The depth profile is where the human analogy breaks. Human priming data describe both-then-one: exhaustive access, then selection. The model's profile is one-then-the-other: the sense the setup primed is top-10 early, the second is absent for most of the network and is lifted in the last third - and sometimes the first drops out as the second comes in (wedding rank 1 at layers 22–33, gone by 38 as radio takes over). In a plain sentence the other sense never rises at any layer, so there is nothing to prune. The model does not disambiguate a pun; it re-ambiguates a sentence, at the end.

Where the late sense arrives does not scale monotonically with depth (four traced puns; early-layer hits below 20% of depth are ignored because the lenses are noisy there). Depth does not simply buy earliness.

Watching it happen, layer by layer

Each frame is one layer deeper into the network, at the pun word. Left: the top-8 words in the lens readout, with hits for each sense tagged. Right: the rank of both sense clusters for the pun and its controls.

The same walk on the 9B: reception, charged.

2. Is the readout used? A clamp, and its trap

Qwen3.6-27B. Change in forced-choice log-odds (sense A primary) when one sense's lens vectors are projected out. Red = clamping A (should fall), blue = clamping B (should rise). Right: five random tokens' vectors.

On both Qwen models the proxies-only clamp moves the forced choice by about 1.2–1.5 nats in the predicted direction while random vectors do ≈0: modest, consistent, causal. A free-text "explain the joke" is unaffected - the readout subspace is used, not necessary. The larger numbers from also removing the pun word's own vector are a trap: the model can no longer read the word and explains a different joke ("the double meaning of lost", "twelve mints", "get a"). That is lexical deletion, not sense suppression.

On Gemma the clamp cannot be interpreted: random vectors move the answer by up to 27 nats, as much as the real clamp, and explanations degrade under either. The projection is destructive on this model, most likely because base-lens directions are not clean directions in the instruct model's residual stream. The causal claim is Qwen-only until a lens is fitted on the model being clamped.

3. Riddles: when the answer appears

Best rank of the answer over all layers at the question mark, before any generation. The 9B does not have the riddle answer while reading - it is rank 18 at best and over a thousand at worst, while the direct question's answer is top-3 by layer 8–12. The 27B has piano and clock at rank 1 and candle at rank 4 at the question mark, in its last quarter (layers 48–56); its direct questions resolve ~20 layers earlier. The two-hop problem that needs generation in the small model is folded into the forward pass of the larger one. (Gemma's pilot was not run; its chain results below.)

Qwen3.6-27B. Rank of 'towel' at each generated token for 'What gets wetter the more it dries?'. Top: no chain-of-thought - the answer jumps to the top at ' is', before the article. Bottom: thinking on - 'towel' is top-10 tens of tokens early, and so are its neighbours 'sponge' and 'cloth'; the unrelated word never is.

Qwen3.6-27B. Share of content positions where each kind of word is top-10, by stage of the thinking chain (16 riddles).

If a chain of thought were a search, rival candidates would be live early and eliminated. They are never live: wrong candidates stay under 10% of positions in every window on every model, the unrelated word is at zero, and the answer ramps - from 1% to 16% on the 9B, 9% to 23% on the 27B, 12% to 31% on Gemma - then anchors near rank 1 once written. More capable models carry the answer as a live candidate earlier; none of them prune. What co-rises with the answer are its lexical neighbours (keyboard with piano, hand with glove, sponge/cloth with towel) - a limit of a token-level readout, not evidence of rival hypotheses.

Qwen3.6-27B, all 16 riddles: log rank of each candidate across the thinking chain (red = answer, grey = unrelated, dashed = answer first written).

The same chain on the 9B: towel.

4. Three models side by side

(a) Rank of the weaker sense at the pun word, puns vs one-sense controls. (b) Riddle answer / wrong / unrelated top-10 rate through the chain. (c) Clamp shift under sense proxies vs random vectors; Gemma's control is as large as its clamp.

What transfers across families: the pun reading (both senses, the second arriving late) and the riddle ramp. What transfers within the family but not across: the clamp. What changes with capability: whether a riddle is solved during the read - the 9B cannot, the 27B can - and how early the answer is carried in a chain of thought.

5. A lens of Gemma's own, and a direction instead of five words

Section 5 left two loose ends: the Gemma clamp was a method failure, and the pun readout rested on hand-written proxy words. Both were followed up (Nov 1–2, 2026).

Fitting a J++ lens on Gemma-4-31B-it itself. With the authors' pipeline and the viewer's two-GPU split, a lens was fitted on the instruct checkpoint: 64 WikiText prompts through the LRP backward pass at 8 Jacobian rows per pass (578 s per prompt, 617 min on two A6000s), 22 source layers, expert weights fitted on the fit split of the paper's readout evals, every prompt presented inside a user turn as in the experiments. On the held-out evals it is a far better readout for this model than the borrowed base lens: recall@10 61.0% vs 48.4% (recall@1 45.7% vs 20.6%; the pooled R-lens of the same fit: 46.8% / 30.3%).

The clamp still fails - so the intervention, not the lens, was the problem. Rerun with the fitted lens, the proxies-only clamp moves Gemma's forced choice in the predicted direction 8/16 times (chance), by a median 8.3 nats, while removing five random tokens' vectors moves it by 12.4 (base lens: 13/16, 9.1 vs 11.4; Qwen 27B: 15/16, 1.2 vs 0.5). Projecting five directions out of fourteen layers is destructive on Gemma's residual stream whatever the directions. The fitted lens is also worse for the single-word sense readouts at the pun word (both senses ≤ 20 in 1/20 puns vs 3/20; one-sense sanity 14/40 vs 26/40): its expert weights are tuned to the paper's readout tasks. The base lens stays the readout for the pun experiments, with its stated confound.

Sense probes. For each of the 20 pun words, 40 sentences per sense that end on the word (the sense fixed by the context) and 15 per sense that carry the sense without the word. One direction per (word, layer) = the normalised difference of class means of the residual at the word token. Every word yields a probe on the 27B (5-fold CV 0.99–1.00, shuffled labels 0.44; transfer to the without-word sentences 0.73–1.00 median) and 18/20 on Gemma (CV ≥ 0.85, transfer median 0.90). "Financial" vs "curiosity" is now a direction, not five words.

(a) Sense score at the pun word - (h·u − midpoint)/half-gap, so the two one-sense controls sit near ±1 - for every pun on both models. (b) The pun word's mean score across depth. (c) Steering the 27B's forced choice along the probe direction, by which tokens are steered.

The pun word is a mixture that never resolves. Scored on the probe (+1 = sense A, the setup-primed sense; −1 = sense B, the sense the final phrase completes), the pun word sits at -0.43 on the 27B where the one-sense controls sit at +0.77 and -0.98; 17/20 puns lean B, 12/20 are in the interior with margin from both controls. Gemma: -0.30 between +0.80 and -0.87, 12/18 interior. The mixture is there from the first layers (L8: -0.48) and still there at the last (-0.45); the model does not pick a sense. Before the word, the pun setup scores -0.10 against +0.52 / -0.53 for the two controls' setups: the setup primes A, and the word then lands half-way back toward its literal reading.

Where the mixture is read, and where it is made. Steering along the probe direction (layers 24–52) and asking the forced-choice question: steer only the pun word's token and almost nothing happens (Δ log-odds toward A / B at 0.25 half-gaps: +0.08 (12/20) / -0.24 (16/20); at 1.0: -0.29 (8/20) / -0.29 (15/20)). Steer only the setup tokens and there is a small, consistent effect at the lowest dose - +1.07 (19/20) / -1.52 (17/20) - that shrinks as the dose grows (+0.62 (14/20) / -1.18 (15/20) at 1.0): larger pushes leave the manifold. Steer every token of the question and the numbers are large (+6.41 (18/20) / -8.47 (20/20)), but those positions include the two label words and two more mentions of the pun word, so that is answer-biasing, not evidence about the pun. Gemma shows the same ordering (word +2.25 (13/20) / -2.04 (15/20); setup +6.96 (15/20) / -5.43 (17/20); all +7.89 (15/20) / -16.97 (19/20)). Attribution of the mixture on the 27B: the pun − control shift at the word (+0.55 half-gaps at L48) is built +0.39 by MLPs and +0.16 by attention heads, no single head above 0.04; a few heads recur across words (L28H6 in 12/20, L36H14 in 12/20, L40H5 in 11/20, L44H5 in 10/20), attending to the setup. There is no pun head; the mixture is a distributed, MLP-built representation.

What this changes. The readout finding is now stated with a direction rather than proxies, on two models, with shuffled-label and transfer controls, and it is layer-stable: both senses are live in the word's representation to the end. The causal side is weaker than a first pass suggested: the model's sense judgment is sensitive to the setup's sense content (small, consistent), and inert to the word-token representation at these layers; whether anything downstream reads the word's mixture in a sense-specific way is open. The dose curve also explains the clamp saga - interventions above about a quarter of the natural sense gap distort the stream, violently on Gemma.

6. Reading the instruments against each other, and against the human literature

Sections 1 to 5 report what each instrument showed. This section is interpretation: what the two instruments actually measure, why they appear to disagree, how the result sits against the psycholinguistics of ambiguity and insight, and what the intervention failures say about method. It is separated from the results on purpose; the numbers above stand whether or not this reading of them survives.

Two instruments, two quantities. The lens (section 1) and the sense probes (section 5) give opposite-looking pictures of the same twenty words. The lens shows the second sense arriving in the last third of the stack, at layers 36 to 55, after forty layers in which only the setup-primed sense is readable. The probe shows the pun word sitting between its two senses at layer 8 and still sitting there at layer 64, with no trend. Both are correct, because they measure different things. The lens maps a residual vector to the vocabulary: it answers "which tokens is the model disposed to emit from here?" and a sense is visible only through the tokens that name its neighbourhood. The probe is a direction fitted on the model's own activations for the word in each sense: it answers "where does this vector sit relative to the model's own representations of the two readings?" What arrives late under the lens is therefore sayability, the point at which the model becomes disposed to produce radio or financial as output. What is present throughout under the probe is the blend of the two readings in the vector that carries the word. The earlier description of the pun as "one reading, then the other, sometimes with a hand-off" (section 1) is a description of the readout and should be read as such. The representation does not hand off; it is mixed from the start and stays mixed.

Why the lens lags. This is consistent with what the J++ paper reports for its own benchmark: late layers read far better than early ones because the map from residual to vocabulary is only well-conditioned where the model has begun to commit to tokens. A sense that is represented but not yet "spoken for" is invisible to a vocabulary readout and visible to a probe. The practical lesson for this kind of work is that lens-rank trajectories are evidence about output disposition over depth and should not be used, on their own, to date when a representation forms.

Against the human data. The prediction brought to the model came from Swinney's cross-modal priming work: both meanings of an ambiguous word are accessed within a few hundred milliseconds regardless of context, then the irrelevant one is selected against. Coulson and Severens' ERP study of puns adds that pun contexts keep both meanings available through the end of the sentence rather than letting selection finish. The lens trajectory looked like a failure of the first half of that prediction (the second sense did not appear early); the probe restores it. Measured by representation rather than readout, the model matches the human profile for puns, both senses live to the end, and for plain sentences, one sense only. Where it departs is the selection step. Nothing in any trajectory looks like a prune: there is no window in which both senses are strongly present and one is then suppressed. The controls sit at one sense at every late layer; the puns sit between from the first layer. A human reader holds both and then drops one; this model holds a weighted blend and never revises it within the pass. Giora's graded-salience account, in which the more salient meaning is accessed first whatever the context, maps onto the setup-primed sense being the one the lens can read early, but the probe gives no evidence of a temporal order inside the forward pass, so that mapping is suggestive only.

Insight, riddles, and the missing plateau. For riddles the relevant human literature is on insight. Metcalfe and Wiebe's feeling-of-warmth ratings stay flat on insight problems and then jump; Kounios and Beeman's EEG work puts a right anterior temporal burst shortly before the solver can report the answer. Neither signature has an analogue here. In the 27B the riddle answer is in the top ten at the question mark, sixteen layers from the top, with no search visible at any earlier layer; the 9B does not get there in the pass and needs the reply. With chain-of-thought the answer rises as a ramp from the first third of the chain and is accompanied by its lexical neighbours, not by rival hypotheses, and this holds on all three models (section 3). The block that produces a plateau and a release in a person is, for the larger models, a few more layers of the same pass. Whether that is because the model has nothing like a competing-hypothesis stage or because a token-level lens cannot see one is open; the glove/hand and piano/keyboard cases show the lens collapsing neighbourhood and hypothesis into one measure.

What the interventions teach about method. Three interventions were tried, and their ordering by effect size is the inverse of their ordering by validity. The all-position probe steering (section 5, panel (c) of the probe figure) moved the forced choice by four to eight nats and was an artifact: the prompt contains the two answer words and repeats the pun word, so steering those tokens along a sense direction biases the answer directly. The five-vector lens clamp (section 2) moved it by about one nat, reliably, but removing five random vectors on Gemma did as much, with either lens (section 5), which means the projection itself is the perturbation. Setup-restricted probe steering gave about one nat at the gentlest dose and less at higher doses. The consistent reading is that these models encode a word's sense redundantly, so one-dimensional removal barely registers, and that any push larger than roughly a quarter of the natural gap between the senses takes the activations off-distribution rather than editing them. Lederman and Mahowald's replication of the thought-injection paradigm on Qwen3-235B and Llama 405B reached the same two conclusions independently: coherence decays with steering strength, so results must be filtered to doses the model survives, and a model's ability to name an injected concept depends on the injection continuing while it writes (stop steering at the end of the prompt and naming collapses while detection does not). Their naming/detection dissociation is the template for the one experiment this project has not run (below).

Where is the knot read? The strongest residual finding is negative. The pun word carries both senses, but the model's judgment of which sense is primary is nearly indifferent to the representation at that token at layers 24 to 52, and sensitive instead to the setup. Attribution confirms that nothing in the word's own position is special: the pull toward the primed sense is built by feed-forward blocks across many layers, with no single head contributing more than a few percent of it. So whatever downstream computation answers "which sense?" is reading it somewhere other than the pun word's late residual, most plausibly through attention from the answer position onto the setup, or from the word's representation at layers earlier than the clamp window. A probe on the answer token's attention to the pun word, or steering the word only at the layers where the answer position attends to it, would settle this. Until then the honest statement is that the mixture is present and causally inert at the position and layers tested.

First-order and second-order. The mixture is a property of the word's representation: in the language of the introspection literature, a first-order state. Nothing here bears on whether the model has any second-order access to it, any representation of "this word is doing two things." That is the question the thought-injection work is now trying to operationalize, and its current answer for other models is that they can detect that something is unusual in their own activations without reliable access to what. The experiment that would ask it for puns is describable with the tools already built: inject the probe direction at the pun word at a dose the model survives (0.25 half-gaps), ask whether the final word reads as ambiguous, and separate detection from identification with the controls Lederman and Mahowald used, including a zero false-positive requirement on uninjected trials and a random-direction injection of matched norm. It has not been run, and nothing in this write-up should be read as evidence either way.

Human-written puns. A separate check (wiki page jspace-retroactive-fit, 2026-10-23) applied the lens to 66 human-written puns from the HaHackathon corpus with crowd funniness ratings. The second sense is already in the top 20 at the end of the setup in about half of them (median rank 17), so professional puns pre-load the reading the model has to reach late in the twenty constructed ones. Neither the setup-end rank of the second sense nor its rank at the pun word predicted crowd funniness (correlations near zero); the only correlate was offensiveness, negatively. The probe mixture score has not yet been computed on that set, and the author's own ratings of the 66 are the planned criterion.

What would change this reading. If a probe on the answer position's attention found that it reads the pun word's blend sense-specifically, "causally inert" above would narrow to "inert at the residual, read through attention." If the probe mixture score on the 66 human puns tracked funniness where the lens ranks did not, the sayability/representation distinction would acquire a behavioural correlate. If a Gemma intervention that preserves norm and stays under a quarter-gap still produced nothing, the Gemma clamp result would stop being a method failure and become a model result.

Per-model figures

Qwen3.5-9B

Qwen3.5-9B: 20 puns, late-layer ranks.

Qwen3.5-9B: sense ranks across depth, four puns.

Qwen3.5-9B: clamp forced-choice shifts.

Qwen3.5-9B: towel riddle, no-think and think.

Qwen3.5-9B: candidate hit rates by chain window.

Gemma-4-31B-it

Gemma-4-31B-it: 20 puns, late-layer ranks.

Gemma-4-31B-it: sense ranks across depth, four puns.

Gemma-4-31B-it: clamp forced-choice shifts.

Gemma-4-31B-it: towel riddle, no-think and think.

Gemma-4-31B-it: candidate hit rates by chain window.

What this does and doesn't show

• The lens reads tokens. Two senses of one word are visible only through proxy words, and a riddle "candidate set" is a neighbourhood in readout space. Stronger claims about rival hypotheses need a representation that separates "considering X" from "X is nearby."
• A clamp removes five directions of several thousand; read it as "this readout subspace," not "this concept." The causal effect is real but modest. Section 5 shows why: the sense is redundantly encoded, so one-dimensional removal is nearly inert, and interventions above about a quarter of the natural sense gap distort the stream rather than edit it - violently on Gemma, with either lens.
• The pun proxy clusters in sections 1–4 are hand-written and are the main degree of freedom; they were fixed before results and are the same for all models, which is why they under-read Gemma. Section 5 replaces them with per-word sense probes, with shuffled-label and transfer controls.
• Four traced puns for the depth-arrival table; three models, one lens each. The timing claim ("the second sense arrives late") is robust across items and models; the scale claim about where it arrives is not.

What I'd change

• Find where the word's mixture is read: a probe on the last token's attention onto the pun word, or steering the word only at the layers where the answer position reads it, to settle whether anything downstream uses the mixture sense-specifically.
• A norm-preserving, magnitude-capped intervention for Gemma, whose residual stream cannot take a projection; the dose curve says the edit must stay under a quarter of the natural sense gap.
• Probe the mixture on the rated corpora (the 66 human-written puns, the 120 one-liners) to test whether how evenly the two senses are held predicts funniness - the earlier likelihood and lens-rank measures did not.
• Run the riddle-at-'?' pilot on Gemma, and on a model between 9B and 27B, to find where in-read riddle solving switches on.
• Human comparison with matched materials: the same twenty puns in a cross-modal priming or ERP design, so "both-then-one" vs "one-then-the-other" is a measured contrast rather than a literature one.

Changelog

• November 5, 2026 - Sections 5 and 6 added. Section 5: J++ lens fitted on Gemma-4-31B-it (held-out recall@10 61% vs 48% for the base lens; clamp still at chance, so the intervention was the failure), sense probes on 20 words for both models (the pun word sits between the senses from the first layer to the last), position-restricted steering (setup-sensitive, word-token inert; the all-position effect was answer-biasing) and attribution (diffuse, MLP-built, no pun head). Section 6: interpretation, kept separate from results: sayability versus representation, the human ambiguity and insight literatures, what the three interventions teach about dose, the open question of where the mixture is read, and the first-order/second-order distinction with the introspection work (Lederman and Mahowald).
• October 13, 2026 - First publication. Qwen3.5-9B (Oct 10–12), Qwen3.6-27B (Oct 12–13) and Gemma-4-31B-it (Oct 13). A Gemma base-checkpoint run was made and discarded (6/16 riddles, unstable forced choice); the instruct model with the base-fitted lens is the comparable run. The 27B riddle pilot (answer at the question mark) was added on Oct 13.