Context-Parameter Copying Capturing Algorithm

• Context-Parameter Copying Capturing algorithm is a diagnostic tool that distinguishes token usage from external context versus internal parametric knowledge during Chain-of-Thought generation.
• It conducts dual model executions—with and without context—to capture top-K token candidates, logits, and hidden states for precise attribution of context versus parameter sourcing.
• The method facilitates detailed analysis of copying behavior, supporting improvements in contextual faithfulness and reducing hallucinations in language model outputs.

The Context-Parameter Copying Capturing algorithm is a token-level probing algorithm for LLMs that separates and quantifies two distinct sources of generation during Chain-of-Thought output: contextual copying, defined as use of the external context $C$, and parametric copying, defined as use of internal parametric knowledge, operationalized as what the model would produce without seeing $C$. Introduced as an interpretability and diagnostic tool in “Copy-Paste to Mitigate LLM Hallucinations” (Long et al., 1 Oct 2025), the method runs the same model both with and without context, examines the top-$K$ candidates at each generation step, and captures separate sequences of probabilities and hidden states associated with contextual versus parametric knowledge usage over time. In the authors’ framework, the algorithm is not a training component; rather, it is used to analyze why CopyPasteLLM improves contextual faithfulness and to support the claim that the model “recalibrates reliance on internal parametric knowledge rather than external knowledge during generation” (Long et al., 1 Oct 2025).

1. Definition, objective, and conceptual scope

The algorithm defines context copying as reuse of knowledge whose source is the external context $C$, operationalized as high-probability tokens in the contextual run that appear in the context and are not attributable to overlap with the context-free answer. Parameter copying, or parametric knowledge usage, is operationalized as high-probability tokens in the contextual run that appear in the context-free answer $A_{\text{para}}$ but not in the context $C$. Tokens common to both are excluded to avoid ambiguous attribution (Long et al., 1 Oct 2025).

Its immediate purpose is mechanistic analysis rather than inference-time control or model optimization. The stated research question is: “What are the underlying mechanisms of CopyPasteLLM's contextual belief? We will interpret this by analyzing logits and hidden states.” The algorithm is therefore designed to measure, over Chain-of-Thought time, how much a model leans on contextual versus parametric knowledge; to compare this behavior before and after CopyPasteLLM training; and to test whether improved faithfulness is associated with changes in contextual representations or with changes in reliance on internal knowledge (Long et al., 1 Oct 2025).

This situates the method within a broader family of copying analyses, but with a distinct emphasis. Earlier sequence-to-sequence copying models such as CopyNet and SeqCopyNet were generation architectures that learned when and what to copy from the input, including tokens or contiguous spans (Gu et al., 2016, Zhou et al., 2018). By contrast, the Context-Parameter Copying Capturing algorithm is a post hoc probe over decoder trajectories. This suggests a useful distinction between copying as an architectural mechanism and copying as an object of interpretability analysis.

2. Formal specification and mathematical objects

The algorithm takes as input a context $C$, a query, a contextual answer $A_{\text{ctx}}$ of length $n$, per-step logits $\mathcal{P}_i$, per-step hidden states $C$0, a context-free answer $C$1, and a top-$C$2 scope for knowledge capture. It returns four outputs: $C$3, $C$4, $C$5, and $C$6, namely lists of probabilities and hidden states corresponding to captured contextual and parametric tokens (Long et al., 1 Oct 2025).

A central companion notion is the paper’s copying degree between answer $C$7 and context $C$8, defined in terms of exact copied fragments $C$9: $K$0 Here, $K$1 is Copy Coverage, the fraction of answer tokens covered by copied fragments, and $K$2 is Copy Density, a length-sensitive measure emphasizing long verbatim spans (Long et al., 1 Oct 2025). These quantities are conceptually separate from the capturing algorithm itself, but they belong to the same analytical framework and are used elsewhere in the paper for high-copying preference construction.

Within CP-Refine, the paper defines a copy-based score

$K$3

with refinement terminating when $K$4 (Long et al., 1 Oct 2025). This score is not part of Context-Parameter Copying Capturing, but it anchors the paper’s broader operationalization of copying and helps connect surface copying behavior to internal token-level diagnostics.

The core aggregation statistic derived from captured tokens is logits power: $K$5 where $K$6 is the logit value of the $K$7-th token in the captured set and $K$8 is the number of samples in the dataset that have contextual or parametric knowledge at that position. This statistic is used to construct per-position curves for contextual and parametric reliance across response trajectories (Long et al., 1 Oct 2025).

3. Algorithmic procedure and token attribution

The procedure begins with two model executions. First, the model is run with context, typically in a Chain-of-Thought setting, to obtain $K$9 together with logits $C$0 and hidden states $C$1 at every generation step. Second, the same model is run without context to obtain $C$2, which serves as an approximation to the model’s parametric prior (Long et al., 1 Oct 2025).

Before token attribution, the method computes

$C$3

which identifies overlapping substrings between the external context and the context-free answer. Tokens in this set are ignored during capture because their origin is ambiguous: they may reflect either contextual reuse or parametric reproduction (Long et al., 1 Oct 2025).

At each position $C$4 of the contextual run, the algorithm softmaxes logits,

$C$5

sorts the vocabulary by descending probability, and scans only the top-$C$6 candidates. The highest-probability non-meaningless candidate is then classified according to membership tests. If a token appears in the context $C$7 and not in the ambiguous overlap set, and has not already been captured as a contextual token, it is added to $C$8 and its corresponding hidden state $C$9 is added to $A_{\text{para}}$0. If instead the token appears in $A_{\text{para}}$1 but not in $A_{\text{para}}$2, and has not already been captured as parametric, it is added to $A_{\text{para}}$3 and $A_{\text{para}}$4 is added to $A_{\text{para}}$5. Tokens judged meaningless, such as function words or punctuation, are skipped (Long et al., 1 Oct 2025).

Two properties follow directly from this design. First, attribution is sequential and position-aware: it tracks knowledge-source usage over the entire Chain-of-Thought trajectory rather than only at the final answer. Second, the outputs are sparse proxies rather than exhaustive decompositions: for each time step, at most one contextual token or one parametric token is captured (Long et al., 1 Oct 2025). This suggests that the method is best understood as a structured probe over salient token candidates, not as a full causal partition of every generated token.

4. Outputs, derived analyses, and empirical interpretation

The captured probabilities and hidden states support two forms of downstream analysis. The first is a per-position comparison of CTX logits power and Para logits power. In the paper’s visualization scheme, contextual logits power is plotted above the x-axis and parametric logits power below the x-axis, negated for visualization. This yields curves over response length that summarize how strongly each knowledge source is expressed across the trajectory (Long et al., 1 Oct 2025).

The second analysis concerns hidden-state geometry. The sets $A_{\text{para}}$6 and $A_{\text{para}}$7 are treated as two distributions and visualized with UMAP. The paper compares Base-CTX against Base-Para, CopyPasteLLM-CTX against CopyPasteLLM-Para, Base-CTX against CopyPasteLLM-CTX, and Base-Para against CopyPasteLLM-Para. The observed pattern is that base models show minimal separation between contextual and parametric hidden-state distributions, whereas CopyPasteLLM shows clearer separation; at the same time, contextual hidden states are almost co-distributed between the finetuned and base models, while parametric hidden states differ substantially (Long et al., 1 Oct 2025).

These analyses are used to support a specific interpretation. According to the paper, in contextual Chain-of-Thought both base models and CopyPasteLLM lean more on contextual than parametric tokens overall, but CopyPasteLLM shows stronger contextual utilization, reduced parametric utilization, and earlier peaks of contextual reliance in the response. Combined with the hidden-state plots, this is interpreted as evidence that CopyPasteLLM does not fundamentally change how context is encoded; instead, it recalibrates confidence in parametric knowledge and sharpens separation between contextual and parametric regimes (Long et al., 1 Oct 2025).

The diagnostic role of the algorithm is especially important because the broader CopyPasteLLM results are framed around faithfulness rather than mere copying. The paper reports that high-copying responses achieve superior contextual faithfulness and hallucination control, and that CopyPasteLLM attains best performance in both counterfactual and original contexts on FaithEval, ConFiQA, and PubMedQA, with 12.2% to 24.5% accuracy improvements on FaithEval over the best baseline while requiring only 365 training samples (Long et al., 1 Oct 2025). The Context-Parameter Copying Capturing algorithm is the mechanism by which these output-level improvements are tied to internal dynamics.

5. Relation to copying mechanisms and adjacent mechanistic work

The algorithm is explicitly said to be inspired by Knowledge Token Capturing (KTC), but differs in scope: KTC primarily analyzes short final answers, whereas Context-Parameter Copying Capturing extends analysis to the entire Chain-of-Thought response trajectory, enabling sequential, position-aware assessment of contextual versus parametric reliance (Long et al., 1 Oct 2025). In that sense, its novelty lies less in inventing copying as a concept than in making knowledge-source attribution longitudinal.

A broader comparison emerges when it is placed alongside copying mechanisms in sequence modeling. CopyNet incorporates copying into neural sequence-to-sequence learning by combining a generator over a fixed vocabulary with a copy mode over source tokens (Gu et al., 2016). SeqCopyNet extends this from single-token copying to span-level copying, using a copy switch gate, start and end pointers, and a copy state transducer to select contiguous subsequences from the source (Zhou et al., 2018). Those models solve the operational problem of producing copied content. The Context-Parameter Copying Capturing algorithm instead probes whether already-generated content reflects contextual or parametric sourcing. This suggests that the notion of “copying” has bifurcated in the literature into at least two technical senses: a decoding mechanism and an attribution framework.

Recent mechanistic studies deepen that second sense. “Context Copying Modulation: The Role of Entropy Neurons in Managing Parametric and Contextual Knowledge Conflicts” argues that entropy neurons in the last FFN layer are responsible for suppressing context copying across a range of LLMs, and that ablating them leads to significant changes in the generation process (Tighidet et al., 12 Sep 2025). “To Copy or Not to Copy: Copying Is Easier to Induce Than Recall” extracts an arbitration vector from residual activations to steer behavior in two directions, Copy$A_{\text{para}}$8Recall and Recall$A_{\text{para}}$9Copy, under misleading or relevant context (Farahani et al., 17 Jan 2026). These works address the same parametric-versus-contextual conflict, but do so by causal intervention rather than by token-level capture. A plausible implication is that Context-Parameter Copying Capturing can be read as a descriptive diagnostic that complements intervention-based analyses.

Architectural work on copying provides a further point of contrast. “Repeat After Me: Transformers are Better than State Space Models at Copying” formalizes copying from context as exact string copying, n-gram lookup, and phone-book lookup, and proves that a two-layer transformer can copy strings of exponential length while generalized state space models are limited by fixed-size latent state (Jelassi et al., 2024). “An All-MLP Sequence Modeling Architecture That Excels at Copying” shows that Causal Relation Networks can match Transformers on the copying task under exponential activation and pre-activation normalization (Cui et al., 2024). “Resona: Improving Context Copying in Linear Recurrence Models with Retrieval” augments linear recurrent models with chunk-and-search retrieval and masked cross-attention to improve context copying and in-context learning (Wang et al., 28 Mar 2025). These papers concern the model’s ability to recover and reproduce context; the Context-Parameter Copying Capturing algorithm concerns the model’s observable choice between context and parameters during generation.

6. Limitations, caveats, and significance

The paper does not present the method as a definitive attribution scheme, and several limitations are evident from its own description. Attribution is token-based and heuristic: commonSubstringMatching filters ambiguous overlap between $C$0 and $C$1, but ambiguous tokens can still exist. The isMeaningless filter depends on a heuristic classification of function words. The algorithm captures only one token per step per category, which makes it a sparse probe rather than a complete decomposition of knowledge usage. It also does not directly label hallucinations; instead, it provides mechanistic evidence that can be correlated with faithfulness and hallucination metrics (Long et al., 1 Oct 2025).

Its implementation constraints are equally specific. The method requires access to token-level logits $C$2 and hidden states $C$3, as well as the ability to run the same model with and without context. It is therefore practical for decoder transformers whose decoding loop is exposed, including Mistral-7B-Instruct-v0.2, Llama-3-8B, and Llama-3.1-8B, but cannot be applied to black-box APIs that conceal internals (Long et al., 1 Oct 2025).

Within those limits, the algorithm occupies a distinctive place in the analysis of contextual faithfulness. It turns the abstract contrast between “context copying” and “parameter copying” into concrete token-level statistics over the full reasoning trajectory, and it links these statistics to both logit-level confidence and hidden-state geometry (Long et al., 1 Oct 2025). This suggests a broader methodological pattern: when contextual faithfulness is the target, internal analysis need not be framed exclusively in terms of attention maps or gradient attribution. Output-token capture, paired with hidden-state collection and longitudinal aggregation, can serve as a technically lightweight but still position-sensitive probe of how a model arbitrates between external context and internal knowledge.