Contextual Citation Alignment Mechanism

• Contextual Citation Alignment Mechanism is a method that anchors citation text to the surrounding local discourse and evidence, ensuring precise and topic-sensitive outputs.
• It redefines citation generation by shifting the target from a single citation span to the entire local context, thereby preventing generic summaries.
• This approach underpins various systems from citation generation to verification and recommendation, demonstrating measurable improvements in citation relevance and coherence.

Searching arXiv for papers relevant to contextual citation alignment, citation generation, attribution, and verification. A contextual citation alignment mechanism is a class of methods for ensuring that a citation, citation sentence, or citation judgment is anchored to the specific local discourse and source evidence that justify it. In its most explicit formulation, introduced for abstractive citation text generation, the mechanism redefines the target of generation from the citation span alone to the entire local context window, so that the model must reconstruct the surrounding sentences while generating the citation itself. This reframing is intended to prevent generic summaries of the cited paper and to induce topic- and stance-sensitive citation text that matches the citing context (Mandal et al., 2024). In later work, closely related mechanisms appear in editorial auditing, sentence-level attribution, citation recommendation, citation verification, and quotation detection, where “alignment” refers to linking local claims, citation markers, retrieved evidence, and model-internal signals in a more explicit and inspectable manner (Mahesh et al., 15 May 2026).

1. Formalization as full-window citation reconstruction

In the formulation of “Contextualizing Generated Citation Texts,” a single citation instance is described by four components: $Cbefore$, the sequence of up to three sentences immediately before the citation slot; $Cafter$, the sequence of up to three sentences immediately after the citation slot; $Citemark$, the bare citation marker in the text; and $A$, the reference paper’s title and abstract, with some variants also using the introduction. Standard infilling uses encoder input

$$X = [INTRO] \oplus Cbefore \oplus “[MASK]” \oplus Cafter \oplus Citemark \oplus A$$

and trains only on the masked citation span. The contextualized mechanism retains the same encoder input but changes the decoder target to the entire local window: $$Y = Cbefore \oplus “[SEP]” \oplus citation\_text \oplus “[/SEP]” \oplus Cafter.$$ At inference time, the generated citation is obtained by extracting the tokens between $[SEP]$ and $[/SEP]$ (Mandal et al., 2024).

The training objective is ordinary negative log-likelihood over the full target sequence: $$L(X,Y) = - \sum_{t=1}^{T} \log p(y_t \mid y_{<t}, X),$$ and over a dataset of $N$ examples,

$Cafter$0

Because $Cafter$1 contains $Cafter$2, the citation text, and $Cafter$3, the model is trained to reproduce the local discourse frame rather than only emit a plausible standalone summary of the cited work. The paper states that this prevents the generator from “cheat[ing]” by ignoring context, since recovering the context sentences requires attention to the contextual clues that determine what topic to discuss and what stance to take (Mandal et al., 2024).

A central implication is methodological rather than architectural. The mechanism does not begin from a new encoder-decoder family; it begins from a different target sequence. This is important because it makes contextual alignment an objective-design intervention rather than a wholesale model replacement.

2. LED-based implementation and empirical behavior

The implementation in the same work uses a Longformer Encoder–Decoder backbone and leaves its self-attention and cross-attention components unchanged. The two explicit modifications are confined to the decoder regime: the hard mask that previously prevented generation of the context sentences is removed, and the special tokens $Cafter$4 and $Cafter$5 are added to the vocabulary and treated as ordinary generation tokens. Encoder-side processing remains the baseline concatenation of $Cafter$6, the masked paragraph, the citation marker, and the reference-paper text (Mandal et al., 2024).

The training loop is correspondingly simple. For each example, the encoder input is the masked local paragraph plus reference information, while the decoder target is the concatenation of left context, opening delimiter, gold citation, closing delimiter, and right context. Inference generates the full sequence token by token until $Cafter$7 or end-of-sequence, after which only the delimited citation span is returned. The mechanism therefore enforces contextual conditioning without requiring any special decoding rule beyond post hoc span extraction (Mandal et al., 2024).

The experimental protocol uses the CORWA corpus, merging the human-annotated train split of approximately 2,243 papers and 11,564 citations with the distant-supervision split for approximately 32,512 training examples, and evaluates on a held-out test set of 1,322 citation instances. The baseline is the original LED citation generator of Li et al. (2022), retrained from scratch with identical hyperparameters except for the contextualized target formulation. The authors explicitly do not rely on ROUGE because it is “known to correlate poorly with citation coherence,” and instead use human evaluation by six computer-science graduate students over randomly sampled citation instances, with preferences rated on Fluency, Relevance, Coherence, and Overall (Mandal et al., 2024).

The reported outcome is that human judges prefer the contextualized model over the baseline in Relevance, Coherence, and Overall in a statistically significant majority of cases, approximately $Cafter$8–$Cafter$9 versus approximately $Citemark$0 for the baseline, while the baseline retains a small edge in raw Fluency. Qualitative analysis attributes the gain to better uptake of fine-grained local cues such as “dependency-level information,” “only work that applied saliency,” “criticizes their convergence focus,” and “further extends the earlier RNN,” whereas the baseline often “regurgitates” generic statements of the reference paper’s main contribution (Mandal et al., 2024).

A common misconception is that contextual alignment here depends on modifying LED’s attention mechanism. The paper states the opposite: the encoder-decoder architecture is effectively preserved, and the key intervention is the decoder target definition.

3. Evidence-grounded alignment in auditing, verification, and attribution

In editorial auditing, the term has been extended from generation to manuscript-level decision support. CitePrism defines a five-stage pipeline consisting of manuscript parsing and citation-context extraction, LLM-assisted contextual reasoning, embedding-based semantic similarity, metadata verification and integrity flagging, and fused relevance scoring with threshold-based triage. For each citation, it extracts a “citation neighborhood” comprising the target sentence plus one sentence before and after, stores this neighborhood in SQLite, scores contextual fit with Llama-3.1-8B-Instruct, computes an embedding similarity score with all-MiniLM-L6-v2, and fuses them by

$Citemark$1

In a single-manuscript pilot on 104 references from pavement engineering, operating at $Citemark$2, agreement with human binary relevance labels reached Cohen’s $Citemark$3, recall on human-labeled irrelevant citations was $Citemark$4, and precision on flagged citations was $Citemark$5, reflecting a conservative screening mode that catches all human-labeled irrelevant citations at the cost of substantial false positives (Mahesh et al., 15 May 2026).

SemanticCite addresses citation verification through full-text source analysis rather than citation generation. Its retrieval pipeline combines BM25 over 512-character overlapping chunks, dense retrieval with sentence embeddings, and a FlashRank cross-encoder reranker, after which the top $Citemark$6 passages are passed to fine-tuned Qwen3 models for four-way classification into SUPPORTED, PARTIALLY SUPPORTED, UNSUPPORTED, or UNCERTAIN. The system also extracts evidence spans, merges adjacent chunks when overlap and attention patterns indicate continuity, and produces reasoning and confidence outputs. On a test set of 112 examples from a dataset of 1,111 citations across eight disciplines, the best Qwen3-4B model achieved standard accuracy $Citemark$7, weighted accuracy $Citemark$8, macro-$Citemark$9 $A$0, and weighted-$A$1 $A$2 (Haan, 20 Nov 2025).

SelfCite shifts alignment to sentence-level attribution in long-context question answering. It uses context ablation to test whether a cited set of context sentences is necessary and sufficient for the model’s own generated statement. For a statement $A$3 and citation set $A$4, it defines

$A$5

$A$6

and combines them as

$A$7

This reward is used both for best-of-$A$8 citation sampling and for SimPO fine-tuning. On LongBench-Cite, baseline LongCite-8B averaged citation $A$9, best-of-10 sampling increased this to $$X = [INTRO] \oplus Cbefore \oplus “[MASK]” \oplus Cafter \oplus Citemark \oplus A$$0, SimPO fine-tuning yielded $$X = [INTRO] \oplus Cbefore \oplus “[MASK]” \oplus Cafter \oplus Citemark \oplus A$$1, and SimPO plus best-of-$$X = [INTRO] \oplus Cbefore \oplus “[MASK]” \oplus Cafter \oplus Citemark \oplus A$$2 reached $$X = [INTRO] \oplus Cbefore \oplus “[MASK]” \oplus Cafter \oplus Citemark \oplus A$$3, a gain of $$X = [INTRO] \oplus Cbefore \oplus “[MASK]” \oplus Cafter \oplus Citemark \oplus A$$4 points over baseline (Chuang et al., 13 Feb 2025).

FACTUM makes alignment mechanistic and model-internal. It decomposes citation generation in long-form RAG into attention and FFN pathway contributions and defines four scores: Contextual Alignment Score, Beginning-of-Sentence Attention Score, Parametric Force Score, and Pathway Alignment Score. The reported signature of a correct citation is scale-dependent: on Llama-3.2-3B, correct citations exhibit significantly higher CAS, BAS, PFS, and PAS than hallucinated ones, whereas on pruned Llama-3.1-8B, correct citations retain higher BAS and PFS but show lower PAS, implying more orthogonal contributions from attention and FFN pathways. FACTUM reaches approximately $$X = [INTRO] \oplus Cbefore \oplus “[MASK]” \oplus Cafter \oplus Citemark \oplus A$$5 AUC on Llama-3.2-3B and approximately $$X = [INTRO] \oplus Cbefore \oplus “[MASK]” \oplus Cafter \oplus Citemark \oplus A$$6 on Llama-3.1-8B, exceeding the cited ECS+PKS baseline in both settings (Dassen et al., 9 Jan 2026).

A related extension in trustworthy citation generation is RAEL with INTRALIGN. There, contextual alignment means allowing segment-level references to come either from retrieved documents or from internal model knowledge with an associated confidence score, and training with higher loss weights on reference spans, citation markers, and confidence tokens. In the GT, PK scenario for Llama-3.1-8B-Instruct, INTRALIGN reports the highest Accuracy, $$X = [INTRO] \oplus Cbefore \oplus “[MASK]” \oplus Cafter \oplus Citemark \oplus A$$7, $$X = [INTRO] \oplus Cbefore \oplus “[MASK]” \oplus Cafter \oplus Citemark \oplus A$$8, $$X = [INTRO] \oplus Cbefore \oplus “[MASK]” \oplus Cafter \oplus Citemark \oplus A$$9, Convincingness, and Conciseness, while achieving the lowest ECE among the compared methods (Shen et al., 21 Apr 2025).

4. Alignment in recommendation and multi-level document matching

In citation recommendation, contextual alignment often means aligning a local citation context to candidate papers while also modeling broader structural correspondences. “Multilevel Text Alignment with Cross-Document Attention” augments a Hierarchical Attention Network with cross-document attention, enabling document vectors and, in the deep variant, sentence vectors to attend across the paired documents. The shallow variant updates a document representation by attending over the other document’s sentence and document vectors, while the deep variant additionally updates sentence representations before document aggregation. The model is weakly supervised by document-level labels only, yet yields emergent sentence-to-document alignments. On AAN, GRU-HAN improves from $$Y = Cbefore \oplus “[SEP]” \oplus citation\_text \oplus “[/SEP]” \oplus Cafter.$$0 to $$Y = Cbefore \oplus “[SEP]” \oplus citation\_text \oplus “[/SEP]” \oplus Cafter.$$1 accuracy with Deep CDA, fine-tuned BERT-HAN rises to $$Y = Cbefore \oplus “[SEP]” \oplus citation\_text \oplus “[/SEP]” \oplus Cafter.$$2 with Shallow CDA, and on S2ORC the Deep CDA + GRU-HAN variant attains $$Y = Cbefore \oplus “[SEP]” \oplus citation\_text \oplus “[/SEP]” \oplus Cafter.$$3 versus $$Y = Cbefore \oplus “[SEP]” \oplus citation\_text \oplus “[/SEP]” \oplus Cafter.$$4 for plain HAN (Zhou et al., 2020).

DocCit2Vec formalizes another notion of alignment by introducing “structural context,” defined as the set of all other citations already present in the manuscript. Given a citation tuple $$Y = Cbefore \oplus “[SEP]” \oplus citation\_text \oplus “[/SEP]” \oplus Cafter.$$5, it jointly embeds the citing document, local context words, and co-cited papers, then predicts the target citation. In the averaging variant,

$$Y = Cbefore \oplus “[SEP]” \oplus citation\_text \oplus “[/SEP]” \oplus Cafter.$$6

followed by either softmax or negative-sampling training. On DBLP, DocCit2Vec-avg in the full structural-context setting achieves Recall@10 $$Y = Cbefore \oplus “[SEP]” \oplus citation\_text \oplus “[/SEP]” \oplus Cafter.$$7, MAP $$Y = Cbefore \oplus “[SEP]” \oplus citation\_text \oplus “[/SEP]” \oplus Cafter.$$8, and nDCG $$Y = Cbefore \oplus “[SEP]” \oplus citation\_text \oplus “[/SEP]” \oplus Cafter.$$9, outperforming HyperDoc2Vec in the reported comparison and degrading monotonically as structural context is reduced or removed (Zhang et al., 2020).

HybridCite combines local citation recommendation components rather than learning a single aligned representation. It treats a citation context $[SEP]$0 as a short query, produces ranked lists with BM25, LDA, and embedding models such as HyperDoc2Vec, and fuses them with a semi-genetic sampler whose fitness is based on reciprocal rank and whose sampling probability is

$[SEP]$1

The paper reports that a hybrid model containing embedding and information-retrieval components outperforms its individual components and other algorithms by a large margin in offline evaluation and in a user study (Färber et al., 2020).

SciRGC extends the alignment problem to a two-module pipeline that first recommends citation articles and then generates citation sentences. The recommendation stage fuses a local transformer encoder with citation-network collaborative filtering, while the reranker incorporates a citation intent embedding into a SciBERT-based scorer. The generation stage conditions on the citing abstract, local context, recognized intent, and recommended abstracts, and combines reasoning-augmented supervised fine-tuning with Direct Preference Optimization. The framework also introduces CITEVAL,

$[SEP]$2

where the sub-scores correspond to Purpose-driven Articulation, Semantic Accuracy, Contextual Fit, and Information Density. Reported ablations indicate that removing ScCF or CsCF drops MRR by $[SEP]$3 on ACL-200, while generation without DPO lowers CITEVAL from $[SEP]$4 to approximately $[SEP]$5 (Li et al., 26 May 2025).

5. Contextual signals for citation placement, marker semantics, and quotation structure

Contextual alignment is also used before citation generation proper, at the stage of deciding whether a sentence needs citation. In “Modeling citation worthiness by using attention-based bidirectional long short-term memory networks and interpretable models,” the mechanism is a contextual fusion architecture over a section label, the previous sentence, the current sentence, and the next sentence. Each span is encoded by character embeddings, GloVe word embeddings, a BiLSTM, and self-attention, producing vectors $[SEP]$6, which are concatenated with sentence-length features and binary citation-placeholder flags before MLP classification. On ACL-ARC, the model achieves $[SEP]$7, and on PMOA-CITE the contextualized variant reaches $[SEP]$8, an absolute improvement of $[SEP]$9 over the no-context attention model. The paper also reports that section labels and surrounding sentences are crucial for improved predictions (Zeng et al., 2024).

In attributed LLMs, the same label has been used for citation-marker semantics. C$[/SEP]$0-Cite proposes a contextual-aware citation generation framework in which a contextual citation alignment mechanism “first encodes the retrieved document contexts into the symbol representation of citations, then aligns the marker numbers by decoding information from a citation router function.” According to the abstract, this turns citation markers from “generic placeholders” into “active knowledge pointers” and improves the ALCE benchmark by an average of $[/SEP]$1 in citation quality and $[/SEP]$2 in response correctness (Yu et al., 19 Nov 2025).

Outside contemporary scientific writing, ACT applies a related idea to quotation-rich Aggadic literature. Its three-stage pipeline performs preprocessing and indexing, morphology-aware alignment via a Smith–Waterman-style dynamic program, and context-sensitive enrichment with pattern inference for Simple, Wave, Echo, and Compound quotation structures. The enrichment stage combines a rarity-based score, contextual reinforcement across same-verse candidates, and a calibrated threshold $[/SEP]$3. On the reported benchmarks, the full ACT-QE pipeline achieves $[/SEP]$4, Recall $[/SEP]$5, and Precision $[/SEP]$6, outperforming Dicta, Passim, Text-Matcher, and the compared ACT ablations (Miller et al., 29 Dec 2025).

Taken together, these cases show that the mechanism need not be confined to one granularity. It may align a sentence with a citation need, a citation marker with retrieved documents, or a quotation fragment with a source verse.

6. Evaluation logic, misconceptions, and open technical issues

Across this literature, one recurring point is that contextual citation alignment is not reducible to lexical overlap. The LED-based citation-generation work explicitly avoids ROUGE because it correlates poorly with citation coherence, and SciRGC introduces CITEVAL precisely to score argumentative purpose, semantic accuracy, contextual fit, and information density rather than $[/SEP]$7-gram reuse (Mandal et al., 2024, Li et al., 26 May 2025).

Another misconception is that alignment alone yields autonomous editorial or verification decisions. CitePrism states that it is “pilot-stage decision support, not as an autonomous misconduct detector or automated editorial decision system,” and its pilot exhibits perfect recall on human-labeled irrelevant citations only at the cost of many false positives. SemanticCite likewise presents evidence-grounded classification with explicit confidence and “please review” handling for low-confidence cases rather than fully automatic adjudication (Mahesh et al., 15 May 2026, Haan, 20 Nov 2025).

Several limitations recur. CitePrism is validated on a single manuscript in one domain, with fusion weights and threshold not systematically optimized. The citation-worthiness model reports poor transfer when trained on one corpus and tested on the other, with PMOA$[/SEP]$8ACL $[/SEP]$9 and ACL$$L(X,Y) = - \sum_{t=1}^{T} \log p(y_t \mid y_{<t}, X),$$0PMOA $$L(X,Y) = - \sum_{t=1}^{T} \log p(y_t \mid y_{<t}, X),$$1, indicating strong dataset dependence. SelfCite notes reward exploitation toward overly long citations and uses length caps and “length balancing” to counteract it. FACTUM shows that the signature of a correct citation changes with model scale, so a fixed alignment heuristic does not transfer straightforwardly from 3B to 8B settings (Mahesh et al., 15 May 2026, Zeng et al., 2024, Chuang et al., 13 Feb 2025, Dassen et al., 9 Jan 2026).

A broader technical implication is that contextual citation alignment is better understood as an objective shared by multiple mechanisms than as a single canonical algorithm. In some systems it is achieved by reconstructing the local discourse window; in others by cross-document attention, context ablation, mechanistic scoring, citation-network fusion, or evidence retrieval and reranking. What unifies these designs is the attempt to constrain citation behavior so that references are not merely plausible or stylistically fluent, but locally appropriate, evidentially grounded, and inspectably tied to the text they are supposed to support.