2000 character limit reached
The Truth Lies Somewhere in the Middle (of the Generated Tokens)
Published 11 May 2026 in cs.LG and cs.CL | (2605.09969v1)
Abstract: How should hidden states generated autoregressively be collapsed into a representation that reflects a LLM's internal state? Despite tokens being generated under causal masking, we find that mean pooling across their hidden states yields more semantic representations than any individual token alone. We quantify this through kernel alignment to reference spaces in language, vision, and protein domains. The improvement through mean pooling is consistent with information being distributed across generated tokens rather than localized to a single position. Furthermore, representations derived from generated tokens outperform those from prompt tokens, and alignment across generation reveals interpretable dynamics in model behavior.
Paper Prompts
Sign up for free to create and run prompts on this paper.
Top Community Prompts
Explain it Like I'm 14
What this paper is about (in simple terms)
This paper asks a surprisingly tricky question: if a LLM (an AI that writes text) has lots of “thoughts” while it’s writing, what’s the best way to capture what it’s thinking about as one useful summary vector (an “embedding”)? The authors discover that the best summary isn’t any single moment in the writing. Instead, averaging the model’s internal signals across many of the words it generates gives a clearer, more meaningful picture of what the text is about.
What the researchers wanted to find out
Here are the main questions they explored:
- Can we get better text embeddings by averaging the model’s internal states over the words it generates, instead of using just the last word or parts of the prompt?
- Is helpful information spread out across many generated words, or is it mostly in one place?
- Do these ideas hold up across very different tasks, like describing images, solving math/science questions, and talking about proteins?
- Do the model’s internal states change in understandable “phases” as it generates text?
How they studied it (with simple analogies)
To test their ideas, they needed a fair way to judge whether one embedding is “better” than another. They used a method called kernel alignment. Here’s the idea, without jargon:
- Imagine you have 1,000 image–caption pairs. You put the images on a big board, arranging them so that similar images are close together (this arrangement comes from a strong image model). Then you ask the LLM to produce embeddings for the corresponding captions and arrange the captions on another board so similar captions are close.
- Kernel alignment is a score that says: do the two arrangements match? If captions that are near each other also come from images that are near each other, the alignment score is high. That means the text embeddings “got the meaning right.”
What they compared:
- Last-token vs mean-token pooling: Do you use the internal state at just the final generated word, or do you average the internal states across many generated words?
- Prompt tokens vs generated tokens: Is it better to summarize what the user typed (the prompt) or what the model wrote (the generated continuation)?
- Mixing parts: They also tried mixing summaries from different parts of the generated text (early, middle, late) and even mixing different generations (different random runs or different viewpoints) to see if combining them helps.
Where they tested it:
- Vision–language: captions paired with images, using a reference image model (like DINOv2).
- Reasoning: math/science questions paired with correct solutions.
- Proteins: protein names paired with 3D protein structure embeddings (from a protein model like ESM-3).
What they did in practice
- They used a popular LLM (Qwen3-14B) to generate 128 tokens (words/parts of words) given a prompt like “Imagine what it would look like to see: {caption}.”
- At each generated token, the model has a hidden state (think of it like a snapshot of what it’s currently “thinking”). They combined these hidden states in different ways:
- Last-token embedding: use the final snapshot only.
- Mean-token embedding: average the snapshots across many generated tokens.
- Mixes: combine averages from different segments (first third, middle third, last third) or from different generations.
- They then calculated how well these text embeddings line up with the “reference” arrangement (images, solutions, or protein structures) using the alignment score described above.
The main findings and why they matter
The authors present several key results:
- Averaging across generated tokens beats any single token:
- Why it matters: The model’s understanding isn’t concentrated at one word; it’s spread across the whole generation. Averaging pulls these pieces together to make a stronger, clearer summary.
- Mixing different parts works better than using any one part:
- Whether they mix early/middle/late segments, finer slices, different random generations of the same prompt, or different region-based descriptions of the same image, combinations in between the extremes usually score best.
- Why it matters: Different parts of the generated text capture different, complementary details. Putting them together gives a truer picture of the meaning.
- Generated-token embeddings are better than prompt-token embeddings:
- Even though the prompt contains the key information, the model’s hidden states during generation line up better with the real-world reference. Also, averaging prompt tokens doesn’t help much.
- Why it matters: As the model writes, it reorganizes and clarifies the meaning internally. So if you want a good embedding, don’t just read the prompt—summarize the model’s hidden states while it’s generating.
- The model shows “phases” as it writes:
- The alignment score changes in stages: generic preamble → repeating the prompt → recalling relevant facts → specific description. Even inserting a generic phrase like “Let me recall what I know” can trigger a noticeable shift in the model’s internal state.
- Why it matters: The model’s output words sometimes act like switches that move its internal state into a more meaningful mode, even before it says the detailed content.
- Model-specific dynamics:
- If one model generates text and a different model builds the embeddings, the neat “phase” pattern fades or breaks. Also, if later generated text disagrees with a model’s knowledge, averaging more tokens can actually hurt alignment in that model’s space.
- Why it matters: These effects are tied to how a specific model thinks about its own text, not just the text itself.
- Averaging across layers (depth) doesn’t help like averaging across tokens:
- Averaging across different layers of the network is about as good as picking the best layer, but not better. In contrast, averaging across generated tokens clearly helps.
- Why it matters: The “spread-out information” effect lives across time (tokens), not across depth (layers).
- Not just statistical smoothing:
- They tested whether “averaging helps” only because it reduces noise. When they broke the true image–caption pairings (shuffled them), averaging didn’t bring a benefit. So the gain comes from combining meaningful complementary information, not from blind smoothing.
Why this is important
- Better embeddings with no extra training: You can get higher-quality text embeddings simply by letting the model generate a bit and averaging its internal states—no retraining needed.
- Works across different domains: The benefits show up for image–caption matching, math/science reasoning, and protein structure—very different types of meaning.
- A new way to inspect model “thinking”: Watching the alignment score over time reveals how the model’s internal state evolves, and how certain phrases can shift its “mode” of processing.
What this could lead to
- Improved search and retrieval: If you want to find images, answers, or proteins similar to your query, embeddings from averaged generated tokens may help you match things more accurately.
- Better tools for understanding AI behavior: The phase changes and the impact of simple phrases like “let me recall” give researchers a window into how models organize their thoughts.
- Practical trade-offs: This method costs more compute than a standard one-shot embedding because the model must generate tokens first. Also, these averaged vectors are best treated as analysis tools, not as inputs to continue generation.
Bottom line
The “truth” of what a LLM understands about a prompt isn’t packed into a single word or moment. It’s spread across the words it generates. By averaging the model’s hidden states over those generated tokens, you get a clearer, more faithful summary of meaning—one that matches the real world better across images, reasoning tasks, and even protein structures.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
Below is a single list of concrete gaps and questions that remain unresolved and could guide future research:
- Generalization across LLMs: Do the findings hold across a broader set of architectures, sizes, and training regimes (e.g., instruction-tuned vs. base, RLHF vs. SFT, multilingual vs. monolingual), beyond Qwen3-14B and OLMo3-7B-Think?
- Decoding hyperparameters: How do temperature, top-p, nucleus/beam search, and repetition penalties affect the efficacy of mean pooling over generated tokens and the observed phase dynamics?
- Generation length and stopping criteria: What is the optimal number of tokens to average (T) for different tasks/domains, and can adaptive early stopping (e.g., alignment saturation detection) reduce compute while preserving representation quality?
- Adaptive pooling vs. uniform mean: Can learned or heuristic token-weighting schemes (e.g., attention-weighted means, gradient/logit contribution weights, recency weights) outperform uniform averaging across generated tokens?
- Joint pooling of prompt and generated tokens: Does combining prompt-side and generation-side hidden states (with learned or rule-based weights) further improve alignment, or does prompt-side information consistently dilute the signal in autoregressive decoders?
- Mechanistic explanation of distributed semantics: What circuits (attention heads, MLPs, residual stream dynamics) distribute complementary information across generated tokens, and how do these mechanisms differ from prompt-side representations?
- Phase dynamics causality: Are alignment “spikes” from generic phrases (e.g., recall tokens) causally mediating representational transitions, or merely correlated? Can controlled interventions that hold subsequent text constant isolate causal effects across models?
- Cross-model embedding mismatch: Can a learned mapping (linear or nonlinear) between models’ embedding spaces restore the phase structure when embedding text generated by a different model, indicating a canonical trajectory in latent space?
- Reference space dependence: How robust are conclusions to the choice of reference kernels—e.g., different vision encoders (CLIP/EVA/VOX), protein encoders/structure aggregation schemes, and reasoning targets (symbolic vs. natural language solutions)?
- Metric sensitivity: Do results persist under alternative similarity metrics (cosine vs. dot product), different kernel alignment variants (e.g., HSIC, PWCCA), and normalization choices (feature clipping, L2 normalization), or are gains partly metric artifacts?
- Dataset scope and scale: Are effects stable on larger and more diverse corpora (beyond 1,024 samples), under distribution shift, and in multilingual settings (WIT is multilingual, but multilingual behavior was not explicitly analyzed)?
- Domain coverage: Do the mixing and mean-pooling gains extend to code, conversational dialogue, multi-turn interactions, audio/speech, and cross-modal tasks beyond image captions and protein names?
- Handling incorrect/inconsistent generations: Can alignment dynamics be used as an online detector for hallucination/inconsistency (e.g., when later tokens degrade alignment), and does filtering or reweighting such tokens improve representations?
- Segmenting by semantics rather than contiguity: Does slicing generations by semantic units (e.g., clauses, discourse markers, NER entities) yield stronger complementary mixing than the contiguous-token slices studied here?
- Layer-wise discrepancy: Why do generated-token representations peak at late layers while prompt-token representations often peak at intermediate layers? What mechanisms explain this divergence, and can it be exploited for better readouts?
- Efficiency and deployment: What are the compute/latency trade-offs for using generative embeddings in retrieval/ranking pipelines, and can partial decoding, speculative decoding, or distillation yield similar gains at lower cost?
- Downstream task breadth: Beyond the reported retrieval/ranking/clustering improvements (referenced but not fully detailed in the main text), how do generative mean-pooled embeddings affect classification, entailment, QA, and long-context retrieval?
- Training-time interventions: If inference-time mean pooling already improves embeddings, what additional gains are achievable by training readout heads or lightweight adapters for token weighting, or by joint generation–embedding objectives?
- Robustness to normalization/clipping: The 95th-percentile feature clipping and L2 normalization may influence kernel geometry; how sensitive are conclusions to these choices and to alternative preprocessing (whitening, per-layer normalization)?
- Protein/biomedical prompting: Using protein names as prompts may be an indirect proxy for structure/function; does prompting with sequences, motifs, or curated functional descriptions change representational alignment to structural targets?
- Reasoning reference construction: Gold solution embeddings may capture stylistic differences as much as correctness; do symbolic references (e.g., normalized proof graphs or answer keys) yield cleaner alignment signals than language-based solutions?
- Nonlinear mixing and compositionality: Do nonlinear combinations (e.g., kernel regression, attention over pooled segments) capture complementary information beyond convex linear mixing, and can they be learned without labeled targets?
- Stability across seeds and text variability: While seed variability is low for token-wise alignment, how do content-preserving paraphrases, stylistic changes, and different prompt templates affect the mixing gains and phase patterns?
- Safety/adversarial robustness: Do generative embeddings resist adversarial phrasing or jailbreak prompts better than prompt-side embeddings, and can alignment monitoring serve as a defense mechanism?
- Formal theory: Can a principled model (e.g., of autoregressive inference under causal masking) predict why semantic information is distributed across generated tokens and when averaging should provably improve alignment?
Practical Applications
Overview
Below are actionable, real-world applications grounded in the paper’s main findings: mean-pooling hidden states across generated tokens produces more semantic and useful representations than any single token (and better than prompt-only embeddings), mixing segments/seeds integrates complementary information, and tokenwise alignment reveals interpretable—and sometimes controllable—representational phases. Applications are grouped by deployment horizon and annotated with sectors, possible tools/workflows, and key assumptions or dependencies.
Immediate Applications
- Embedding upgrade for retrieval and RAG (software, enterprise search)
- What: Replace prompt-side embeddings with mean-pooled embeddings across generated tokens for queries and documents to improve semantic search, reranking, clustering, and retrieval-augmented generation (RAG).
- Why it helps: Generated-token mean pooling consistently outperforms both last-token and prompt-token pooling; mixing token segments captures complementary information.
- Workflow/tool:
- Use a sensory/task-priming prompt (e.g., “Imagine what it would look like to see: {text}”) to elicit the right internal state.
- Autoregressively generate T tokens (e.g., 64–128), capture final-layer hidden states at each generated token, mean-pool across generated tokens, l2-normalize; index/store the vector.
- Optionally mix embeddings from a few short, independent decoding seeds (convex combination) to boost robustness.
- Assumptions/dependencies: Requires access to hidden states during generation; added compute (linear in T); prompt choice matters; validate per model/domain.
- Higher-fidelity cross-modal retrieval from text (media, e-commerce, vision)
- What: Use generative embeddings for text-to-image and text-to-video search to better align with vision encoders (e.g., DINOv2).
- Why it helps: Kernel alignment to vision reference spaces increases when averaging generated tokens; combining region-level prompts or seed mixtures further improves alignment.
- Workflow/tool:
- For each text query, generate tokens under a visualizing prompt and mean-pool hidden states.
- Optionally, create multiple “view”-conditioned generations (e.g., region/object-focused prompts) and mix their embeddings.
- Assumptions/dependencies: Reference vision encoders (CLIP/DINOv2) define the target space; benefits shown on Qwen3-14B—retest with your stack.
- Robustness-aware reranker for LLM pipelines (software, developer tools)
- What: Add a generative-embedding reranker stage to RAG or QA pipelines.
- Why it helps: Generated embeddings better capture the prompt’s semantics than prompt-only states.
- Workflow/tool:
- Obtain top-k candidates with a fast encoder (e.g., sentence-BERT).
- Re-embed the query (and optionally the candidates) with generative mean-pooling; rerank by cosine similarity.
- Assumptions/dependencies: Additional latency; consider caching generative embeddings for documents and using generative embeddings only on the query side.
- Early confidence and routing in reasoning systems (education, software)
- What: Use tokenwise alignment trajectories as a proxy for solution quality/confidence and to trigger tool use or fallback models.
- Why it helps: Mixing tokens improves alignment to solution spaces; phases (preamble → recall → task-specific) are interpretable and predictive of progress.
- Workflow/tool:
- During chain-of-thought, compute a rolling mean-pooled embedding; if alignment to a reference (e.g., solution embeddings, rubric vectors) stalls or drops, trigger retrieval, calculator/solver tools, or escalate to a stronger model.
- Assumptions/dependencies: Requires in-house reference representations or handcrafted rubric vectors; access to hidden states; do not leak sensitive internal states.
- Inconsistency/hallucination sentinel via cross-model alignment (safety, compliance)
- What: Monitor degradations in alignment across tokens (or when embedding text from another model) to flag potential hallucinations or off-distribution reasoning.
- Why it helps: When embedding OLMo3-generated text in Qwen3’s space, alignment later degrades as the text diverges from Qwen3’s internal knowledge—useful for detection.
- Workflow/tool:
- Compute tokenwise alignment proxy signals (e.g., similarity to a curated knowledge-graph embedding, or a stronger model’s reference embeddings).
- Flag sequences where alignment decreases after an initial rise, prompting grounding or fact-checking.
- Assumptions/dependencies: No universal reference; must define a domain-appropriate reference space or proxy; false positives possible.
- Practical prompt templates for embedding tasks (software, content platforms)
- What: Adopt task-cued prompts (e.g., sensory prompts for visual domains, biologically relevant prompts for proteins) before generating and pooling embeddings.
- Why it helps: Generation under domain-appropriate prompts moves internal states closer to the desired reference space.
- Assumptions/dependencies: Domain-specific prompt engineering is required; measure impact per use case.
- “Strong-generator, cheap-embedder” pattern (cost optimization)
- What: Generate continuations with a stronger model and compute mean-pooled embeddings with a smaller/cheaper model when suitable.
- Why it helps: In the paper, embeddings in a weaker model’s space improved when fed text generated by a stronger model.
- Assumptions/dependencies: Gains are model- and task-dependent; ensure text still induces useful states in the embedder; watch privacy constraints when crossing model boundaries.
- Protein and scientific text retrieval aids (biotech, materials)
- What: Use generative embeddings from protein/chemical prompts to improve text-side retrieval against structural/physics encoders (e.g., ESM-3).
- Why it helps: Mixing tokens improves alignment to protein structural similarity references.
- Workflow/tool:
- Generate domain-cued descriptions for protein/gene names; mean-pool across tokens; use for literature/asset retrieval against structure-aware indexes.
- Assumptions/dependencies: Alignment ≠ correctness; do not treat as predictive of function; for retrieval/support only.
- Academic tooling: Representational phase probes (ML research, interpretability)
- What: Release/extend a “PhaseScope” tool to visualize tokenwise alignment and detect representational phases during generation.
- Why it helps: Phases (preamble, prompt repetition, recall, specific response) are consistent and interpretable; generic “recall” phrases causally shift internal state.
- Assumptions/dependencies: Requires access to hidden states; relies on chosen reference spaces (vision, solution sets).
- Best-practice guidance for embeddings (engineering)
- What:
- Prefer mean-pooling across generated tokens over last-token or prompt-only pooling.
- Do not average across layers (no further gain observed); use final-layer states.
- Mixing segments or seeds often helps; mixing prompt tokens generally doesn’t (for decoder-only).
- Assumptions/dependencies: Validate with your model; different vendors may expose different internals.
- Daily-life productivity search (consumer apps, PKM)
- What: Improve personal search/clustering (notes, journals, photo captions) by embedding with short generative continuations and mean pooling.
- Why it helps: Better captures semantics of user-authored captions/descriptions.
- Assumptions/dependencies: On-device/open models needed for privacy; manage compute by limiting T and caching.
Long-Term Applications
- Generative Embedding Mode as a first-class API (software platforms, foundation models)
- What: Vendor-supported API that returns mean-pooled generated-token embeddings (with options for T, seeds, prompts) without requiring manual decoding or hidden-state plumbing.
- Why it helps: Operationalizes the paper’s “generation-then-pool” result; broad adoption in search, RAG, and recommendation.
- Dependencies: Model providers must expose hidden states or an equivalent embedding endpoint; standardized configuration and telemetry.
- Real-time state-aware decoding and control (safety, reasoning, robotics)
- What: Use alignment signals as feedback during decoding to steer generation (e.g., trigger “recall” state, stop when alignment plateaus, or branch when misalignment rises).
- Why it helps: Representational phases are causal and manipulable (e.g., “Let me recall…” spikes alignment); enables safer, more interpretable generation.
- Dependencies: Fast alignment proxies (no heavy reference needed), latency budget, research into robust control policies.
- Hallucination prevention via external-reference alignment (healthcare, finance, policy)
- What: Continuous alignment to domain ontologies/knowledge graphs during generation; throttle, slow-think, or enforce retrieval when alignment dips.
- Why it helps: Misalignment correlates with inconsistency; valuable in high-stakes domains.
- Dependencies: Trusted, up-to-date reference embeddings; rigorous evaluation; governance and data access controls.
- Curriculum and assessment analytics from phase dynamics (education technology)
- What: Detect students’ reasoning phases and provide targeted feedback or hints based on alignment/phase profiles of their generated explanations.
- Why it helps: Phase trajectories correlate with recall and solution specificity; supports formative assessment and tutoring.
- Dependencies: Ethical data use; validated pedagogical mappings between alignment signals and learning outcomes.
- Cross-domain semantic unification for scientific discovery (biotech, materials, energy)
- What: Unified embedding spaces where language queries align with structural/physical representations (proteins, molecules, materials), enabling “text-to-structure” retrieval and hypothesis generation.
- Why it helps: Generative token mixing improves alignment to structure encoders; paves the way for cross-modal exploration.
- Dependencies: High-quality reference encoders per domain (ESM-3, scientific foundation models), domain prompts, careful evaluation; does not replace lab validation.
- Training objectives for representation-quality during generation (ML research, productization)
- What: Jointly optimize generation and representational alignment (e.g., via contrastive or alignment-aware losses) to produce models whose generated-token embeddings are natively strong.
- Why it helps: Turn an inference-time trick into a trained capability; reduce reliance on long continuations or multi-seed mixing.
- Dependencies: Access to multi-domain references; compute; careful balancing with generative quality.
- Lightweight approximations to generative pooling (edge, on-device)
- What: Distill or approximate mean-pooled generative embeddings with compact encoders or small “simulation” heads to avoid full decoding at inference.
- Why it helps: Brings benefits to resource-constrained settings.
- Dependencies: Distillation data, teacher-student pipelines; may retain only part of the gains.
- Compliance/audit standards based on representational alignment (policy, governance)
- What: Define audit metrics that track model phase behavior and alignment to approved references across scenarios (e.g., safety benchmarks, domain corpora).
- Why it helps: Provides measurable, model-internal indicators beyond surface text; supports certification and monitoring.
- Dependencies: Standardized references and protocols; community consensus on metrics; avoid overreliance on alignment as a proxy for truth.
- Reasoning-phase token design and libraries (developer ecosystem)
- What: Curated libraries of “state-shifting” phrases or soft prompts that reliably induce helpful representational phases (recall, plan, verify) across models.
- Why it helps: The paper shows generic phrases can causally shift internal states; systematizing this could improve tool-use and reliability.
- Dependencies: Model-specific behavior; risk of overfitting to phrases; need for broad validation.
- Domain-specific cross-model pipelines (enterprise MLOps)
- What: Architectures where a strong generator produces continuations optimized for embedding, followed by cost-effective embedders powering search/retrieval across large corpora.
- Why it helps: Observed benefits when mixing generation and embedding models; balances cost and quality.
- Dependencies: Latency budgets, IP/privacy constraints, careful routing and caching.
General Assumptions and Dependencies Across Applications
- Access to hidden states: Many benefits require capturing per-token hidden states; open-source or vendor support needed.
- Compute/latency: Generative embeddings require decoding T tokens; employ caching, early stopping (e.g., when alignment proxy saturates), and selective use (query-only).
- Prompting: Domain-cued prompts materially affect results; invest in prompt validation.
- Model specificity: Results demonstrated on Qwen3-14B and select references (DINOv2, ESM-3); reproduce with your model/reference stack.
- Alignment ≠ correctness: Kernel alignment measures relational structure, not absolute truth; pair with factuality checks in safety-critical settings.
- Privacy/governance: Generated continuations used for embedding should be handled in-memory; avoid logging sensitive content; comply with data policies.
Glossary
- 2-simplex: The set of all convex weight vectors over three components (nonnegative entries summing to 1). Example: "where A2 = {w E R3 : w; ≥ 0, 2; w; = 1} denotes the 2-simplex."
- Activation tensor: The multi-dimensional array of hidden activations across layers, tokens, and features. Example: "Let H E RLXTxD denote the activation tensor,"
- Autoregressive decoding: Generating tokens sequentially where each token conditions on previous outputs. Example: "Generative representations require autoregressive decoding,"
- Autoregressive LLM: A model that generates text token by token, each conditioned on prior context. Example: "Given an input prompt p, an autoregressive LLM generates a continuation of T tokens;"
- Barycentric grid: A discretization of a simplex using barycentric coordinates for mixing weights. Example: "Alignment is evaluated over a uniform barycen- tric grid on the simplex."
- Causal masking: A constraint preventing a token from attending to future tokens during training/inference. Example: "Despite tokens being generated under causal masking,"
- Causal self-attention: Self-attention restricted by causal masking so positions attend only to past tokens. Example: "through causal self-attention (Vaswani et al., 2017)."
- Centered Kernel Alignment (CKA): A similarity measure between representation spaces based on centered kernel matrices. Example: "We quantify this using the debiased Centered Kernel Alignment (CKA) (Kornblith et al., 2019)."
- Convex combinations: Weighted sums of vectors with nonnegative weights that sum to one. Example: "We then form convex combinations"
- Decoder-only Transformer: A Transformer architecture that uses only the decoder stack for autoregressive generation. Example: "Qwen3 (Yang et al., 2025) is a decoder-only Transformer"
- DINOv2: A self-supervised vision model used to obtain image embeddings as a reference space. Example: "For vision, we use DINOv2 (Oquab et al., 2024),"
- ESM-3: A protein modeling architecture used to embed protein structures as references. Example: "and for protein structures, ESM-3 (Hayes et al., 2025)."
- Final-layer representation: The embedding taken from the last layer of a model after any pooling over non-feature axes. Example: "and use the final-layer representation."
- GPQA Diamond: A challenging benchmark split for graduate-level question answering. Example: "and the GPQA Diamond split (Rein et al., 2024)."
- Kernel (similarity matrix): A matrix of pairwise similarities between sample embeddings. Example: "we form two similarity matrices, or kernels, K, L E Rnxn, where"
- Kernel alignment: Evaluating how similarly two kernels (e.g., model vs. reference) organize the same data. Example: "Kernel alignment asks whether these two similarity struc- tures agree."
- l2-normalized: Scaling a vector to unit Euclidean norm. Example: "and then l2-normalized before computing kernels."
- Last-token embedding: A representation formed by taking the hidden state at the final generated token. Example: "Last-token embedding"
- Mean pooling: Averaging hidden states across tokens (or other axes) to form a single embedding. Example: "we find that mean pooling across generated tokens yields more semantic representations"
- Mutual information: A measure of dependence between random variables; here, between thinking tokens and correctness. Example: "such as "Hmm" or "Wait" generated during reasoning can exhibit a significant increase in mutual information to the correct answer"
- OLMo3-7B-Think: A specific LLM variant used for generation and embedding comparisons. Example: "a weaker model (OLMo3- 7B-Think),"
- Prompt-token representations: Embeddings derived from hidden states at prompt token positions. Example: "Prompt-token representations do not benefit from mean pooling."
- Qwen3-14B: A 14B-parameter LLM used as the primary generator/encoder in experiments. Example: "We use Qwen3-14B with thinking mode in the main text"
- Reference embeddings: Fixed embeddings from another model/modality that define the target semantic structure. Example: "Reference embeddings are held fixed across experiments,"
- Reference space: The semantic space defined by reference embeddings against which alignment is measured. Example: "the reference space is derived from gold solutions,"
- Representational phases: Distinct stages in the evolution of model representations during generation. Example: "Generated tokens induce representational phases."
- Self-supervised Vision Transformer: A ViT trained without labeled data to learn visual representations. Example: "a self-supervised Vi- sion Transformer,"
- Thinking mode: A decoding configuration where the model produces explicit reasoning traces or thoughts. Example: "We use Qwen3-14B with thinking mode in the main text"
- Token-mean embedding: An embedding formed by averaging hidden states across generated tokens. Example: "Token-mean embedding"
- Token-slice ensembles: Groups of contiguous token segments whose pooled embeddings are mixed for analysis. Example: "Vision-language alignment under convex combinations of token-slice ensembles at increasing levels of granularity."
- UniProt: A comprehensive protein database used to source entries for protein-language evaluation. Example: "we sample 1024 entries from the UniProt database (Consortium, 2024),"
- Vision-language alignment: Agreement between text representations and visual reference embeddings. Example: "Vision-language alignment under convex combinations of token-slice ensembles at increasing levels of granularity."