TrackStar Retrieval Attribution

• TrackStar Retrieval Attribution is an attribution-centric retrieval framework that links model outputs to explicit sources like query tokens, documents, and training examples.
• It employs gradient compression and influence-function approximations to retrieve and rank influential training data, guiding query rewriting and evidence attribution.
• Empirical studies demonstrate enhanced retrieval accuracy, explainability, and system auditability across applications from fact tracing to regulatory compliance.

Searching arXiv for TrackStar and closely related attribution papers to ground the article in current literature. arxiv_search(query="TrackStar scalable influence fact tracing LLM pretraining", max_results=5) arxiv_search(query="Improving Neural Retrieval with Attribution-Guided Query Rewriting (Garouani et al., 12 Feb 2026)", max_results=3) arxiv_search(query="Tracing Target Answers in Poisoned Retrieval Corpora via Token Influence Attribution (Chen et al., 24 Jun 2026)", max_results=3) arxiv_search(query="Bergson open source library data attribution TrackStar (Quirke et al., 10 Jun 2026)", max_results=3) arxiv_search(query="TRAK Attributing Model Behavior at Scale (Park et al., 2023) Imperfect Influence Preserved Rankings (Tong et al., 1 Feb 2026)", max_results=5) “TrackStar Retrieval Attribution” denotes an attribution-centered view of retrieval systems in which retrieval behavior is traced to explicit sources—query tokens, retrieved documents, answer sentences, citations, rules, or training examples—and those attributions are then reused for diagnosis, control, and auditing. In the literature, this view appears in several closely related forms: token-level attribution for retriever-guided query rewriting, token influence attribution over retrieved documents in retrieval-augmented generation, sentence-level attribution of long-form answers to evidence documents, per-rule attribution in regulatory question answering, and scalable retrieval of influential training examples through TrackStar’s compressed influence machinery (Garouani et al., 12 Feb 2026, Chen et al., 24 Jun 2026, Chen et al., 2023, Ju et al., 28 May 2026, Quirke et al., 10 Jun 2026). This suggests that “TrackStar Retrieval Attribution” is best understood not as a single algorithm but as an attribution-centric retrieval layer in which explanation and retrieval control are explicitly coupled.

1. Scope and conceptual structure

In Bergson’s terminology, data attribution aims to quantify the causal effect of each training item on a model behavior of interest, typically represented as a differentiable loss on some query example(s), and Bergson introduces the first open-source implementations of three leading data attribution methods: MAGIC, SOURCE, and TrackStar (Quirke et al., 10 Jun 2026). Within that landscape, TrackStar is a scalable influence-function approximation that operates on a single checkpoint and uses heavy gradient compression plus a simple Hessian proxy to enable search or retrieval over massive pretraining corpora (Quirke et al., 10 Jun 2026). In parallel, several retrieval and RAG papers describe “TrackStar-like” systems in which attribution is attached not to training items but to query tokens, retrieved passages, answer sentences, or citations (Garouani et al., 12 Feb 2026, Chen et al., 24 Jun 2026, Li et al., 22 May 2025).

This cross-paper usage yields a stable decomposition of attribution targets. One line asks which query tokens drive retrieval scores under a fixed retriever. A second asks which retrieved tokens or documents steer a generated answer. A third asks whether answer sentences are attributable to the evidence set actually supplied to the model. A fourth asks whether citations or rules are aligned with the claims they are meant to support. A fifth asks which training examples most influenced the model behavior that later appears at retrieval time. These are different objects, but they share a common design principle: attribution is treated as a retrieval-time or audit-time signal rather than as post-hoc decoration.

Attribution target	Mechanism	Representative paper
Query tokens	Integrated Gradients over the retriever’s relevance score	(Garouani et al., 12 Feb 2026)
Retrieved document tokens	Gradient norm of $\log P_\theta(y \mid x_{1:n})$	(Chen et al., 24 Jun 2026)
Answer sentences	Supported / Partially Supported / Not Supported relative to evidence documents	(Chen et al., 2023)
Rule-level claims	Per-rule attribution with rule identifiers mapped to arrays of claims	(Ju et al., 28 May 2026)
Training examples	Projected gradients with an empirical Fisher approximation in compressed space	(Quirke et al., 10 Jun 2026)

In this broader sense, retrieval attribution is a layered provenance problem. The same retrieval system may require token-level explanations for query reformulation, document-level influence tracing for answer debugging, and training-data retrieval for fact tracing or dataset auditing. The “TrackStar” label is most explicit in the training-data literature, but the same attribution logic recurs across inference-time retrieval systems.

2. Query-side retrieval attribution

“Improving Neural Retrieval with Attribution-Guided Query Rewriting” formalizes one of the clearest inference-time forms of retrieval attribution (Garouani et al., 12 Feb 2026). The problem is that neural retrievers are described as “brittle” because small changes or underspecifications in the query often lead to large, unpredictable changes in the retrieved set, even when relevant documents exist. The paper focuses on SPLADE and TCT-ColBERT, where the latter scores relevance by token-level late interaction,

$$s(q,d) = \sum_i \max_j \,\mathrm{sim}(\mathbf q_i,\mathbf d_j).$$

For each query–document pair, the attributed score is the retriever’s relevance score $s(q,d)$, and token-level attributions are computed with Integrated Gradients over query token embeddings. These document-level token attributions are then aggregated across the top-$k$ retrieved documents,

$$\alpha_i = \frac{1}{k}\sum_{d\in D_q^{top}} \mathrm{IG}(t_i, s(q,d)),$$

with $k=5$, followed by per-query normalization (Garouani et al., 12 Feb 2026).

The attribution scores are passed to Mistral-7B-Instruct through a structured prompt that includes the original query, the list of $(\text{token}, \alpha_i)$ pairs, and explicit rewriting rules: preserve the original user intent, do not remove important concepts, preserve or emphasize high-attribution tokens, clarify low or negative attribution tokens, avoid adding new concepts, and produce a single rewritten query. Attribution is used as soft guidance rather than as a hard threshold: all tokens are shown, no masking step is applied, and no iterative loop is used (Garouani et al., 12 Feb 2026).

The empirical pattern is sharply diagnostic. Across all three datasets and both retrievers, GLLM consistently outperforms Org, LLM, and Tkn. The paper summarizes the gains as up to ~9% nDCG@10 improvement over original queries and up to ~22% over LLM-only rewriting for SPLADE, and up to ~10% and ~16% respectively for TCT-ColBERT. The ablation is equally important: Tkn (“Top-Attribution Tokens Only”) is generally worst, showing that attribution should not be used as a hard mask on tokens, and LLM-only rewriting is inconsistent, confirming that retriever-agnostic rewriting is unreliable (Garouani et al., 12 Feb 2026).

The canonical example is “What is actually in chicken nuggets?” with token attributions what (0.008), is (0.012), actually (0.013), in (0.018), chicken (0.217), nuggets (0.648). Tkn collapses this to “chicken nuggets,” LLM-only rewriting gives “What is in chicken nuggets?”, and attribution-guided rewriting yields “What are the ingredients in chicken nuggets?” The mechanism is that “chicken” and “nuggets” are preserved because they are already driving retrieval, while the weakly attributed portion is clarified into an explicit information need. In this formulation, retrieval attribution is simultaneously an explanation of current retriever behavior and a control signal for improving the next retrieval pass.

3. Retrieved-context and answer attribution in RAG

Attribution over retrieved context is developed most directly in TRACE and ARC-JSD. TRACE studies a standard RAG pipeline in which a dense retriever, Contriever, retrieves the top-5 documents and the target LLM conditions on the concatenated documents plus the question. It defines token influence attribution over retrieved-document tokens as

$$\mathrm{Influence}(x_i)=\left\|\nabla_{x_i}\log P_\theta(y\mid x_{1:n})\right\|_2,$$

where $y$ is either the fixed affirmation string "Yes, the answer is:" or a candidate keyword or phrase from a second-stage verification procedure (Chen et al., 24 Jun 2026). The method extracts top-$k$ influential tokens per document, filters punctuation, frequent function words, and tokens already in the question, merges contiguous indices into phrases, and then checks whether high-influence phrases recur across documents. A poisoning alarm is raised when verified trigger tokens persist across documents after this two-phase process.

The detection results are strong. In the high-TPR configuration, TRACE achieves >90% TPR across nearly all models and datasets, with Phi-4-mini averaging 98.87% TPR across datasets. In the FPR-minimizing configuration, FPR is <10% across models. Detection Accuracy for recovering the attacker’s target answer is approximately 90% on NQ and MS-MARCO and still >70% on HotpotQA across models. Runtime is ~1–9 seconds per query, versus 5–17 seconds for RAGForensics, while matching or outperforming it in TPR (Chen et al., 24 Jun 2026). Here retrieval attribution is used as a model-internal provenance mechanism: it identifies which retrieved tokens are actually steering the answer and which repeated phrases act as semantic anchors in poisoned corpora.

ARC-JSD addresses a related but distinct question: which retrieved context sentences are causally responsible for a generated response. Given query $$s(q,d) = \sum_i \max_j \,\mathrm{sim}(\mathbf q_i,\mathbf d_j).$$0, retrieved context sentences $$s(q,d) = \sum_i \max_j \,\mathrm{sim}(\mathbf q_i,\mathbf d_j).$$1, and generated response $$s(q,d) = \sum_i \max_j \,\mathrm{sim}(\mathbf q_i,\mathbf d_j).$$2, the method removes one context sentence at a time and compares the token distributions with and without that sentence using Jensen–Shannon divergence. The sentence score is

$$s(q,d) = \sum_i \max_j \,\mathrm{sim}(\mathbf q_i,\mathbf d_j).$$3

This requires only $$s(q,d) = \sum_i \max_j \,\mathrm{sim}(\mathbf q_i,\mathbf d_j).$$4 forward passes per example, versus hundreds for surrogate-based ContextCite, and improves attribution accuracy by about 10.7 percentage points on average on TyDi QA, HotpotQA, and MuSiQue while reducing compute time by up to a factor of three (Li et al., 22 May 2025). The mechanistic extension localizes attribution-relevant computation in intermediate and higher layers, especially MLP layers, and masking the top-10 JSD-identified attention heads causes significantly larger JSD shifts than masking random heads (Li et al., 22 May 2025).

A third perspective comes from long-form question answering. “Understanding Retrieval Augmentation for Long-Form Question Answering” defines sentence-level attribution as whether a generated sentence is Supported, Partially Supported, or Not Supported by the evidence documents, with supporting document sentences and unsupported spans annotated by humans (Chen et al., 2023). The paper reports that WebGPT with WebGPT documents yields 95% Supported sentences, GPT-3.5 with WebGPT documents 85%, and Alpaca with WebGPT documents 61%, while GPT-3.5 without documents drops to 22% Supported. It also shows that answer sentences tend to follow the order of the in-context evidence documents, that later parts of the evidence are under-used, and that the last sentence is almost twice as likely to be unsupported as other sentences (Chen et al., 2023). The error taxonomy—retrieval failure, hallucinated facts, and incorrect synthesis—provides a direct failure model for retrieval attribution systems that aim to verify grounding rather than merely ranking.

Taken together, these works define retrieved-context attribution at three levels: token influence on answer generation, sentence-level causal contribution of context, and sentence-level human support relative to a provided evidence set. A plausible implication is that a TrackStar-style system should treat these as complementary rather than interchangeable: influence tracing, ablation-based causal attribution, and entailment-style support labeling answer different provenance questions.

4. Citation-, rule-, and telemetry-based attribution

Citation attribution turns retrieval provenance into an explicit alignment problem. CiteGuard reframes citation evaluation as citation attribution alignment: whether LLM-generated citations match those a human author would include for the same text. Its retrieval-aware agent retrieves relevant passages from candidate citation documents and from the wider corpus, scores evidence, and then asks an LLM to validate whether a claim span is actually supported by the retrieved snippets. CiteGuard improves the prior baseline by 12.3%, achieves up to 65.4% accuracy on the CiteME benchmark, is on par with human-level performance at 69.7%, and can identify alternative but valid citations (Choi et al., 15 Oct 2025). The distinctive feature is not only citation checking but citation augmentation: the system may discover additional valid sources beyond those proposed by the generator.

RefWalk moves from citation attribution to per-rule attribution in a regulatory setting where complete evidence-set closure matters more than single-passage support. Regulatory Compliance QA is defined as answering a practitioner’s query by retrieving the exhaustive set of governing articles and detailing the precise claims derived from each rule. RefWalk uses a shared topic anchor, multi-view retrieval queries, 1-hop expansion over an Operational Knowledge Graph along REFERENCES, DELEGATES_TO, and SPECIFIES edges, Reciprocal Rank MAX fusion,

$$s(q,d) = \sum_i \max_j \,\mathrm{sim}(\mathbf q_i,\mathbf d_j).$$5

with $$s(q,d) = \sum_i \max_j \,\mathrm{sim}(\mathbf q_i,\mathbf d_j).$$6, and schema-constrained generation in which JSON keys must be exact node_ids from the retrieved context and values are arrays of claims derived from that rule (Ju et al., 28 May 2026). On RegOps-Bench, RefWalk reaches Recall@10 63.8%, FullCov@10 44.4%, and nDCG@10 57.4% in retrieval, and in end-to-end RAG with Qwen3.6-35B it reaches Citation F1 54.2 and Claim F1 40.4, compared with NativeRAG’s 46.1 and 37.0 (Ju et al., 28 May 2026). The contrastive HIPAA evaluation shows saturation on flat-structure rules, which the paper uses to argue that hierarchical, cross-referenced regulatory corpora are necessary to stress-test attribution systems (Ju et al., 28 May 2026).

A different but complementary dimension is attribution telemetry. “The Attribution Crisis in LLM Search Results” defines the attribution gap as

$$s(q,d) = \sum_i \max_j \,\mathrm{sim}(\mathbf q_i,\mathbf d_j).$$7

where $$s(q,d) = \sum_i \max_j \,\mathrm{sim}(\mathbf q_i,\mathbf d_j).$$8 is the number of unique relevant sites visited and $$s(q,d) = \sum_i \max_j \,\mathrm{sim}(\mathbf q_i,\mathbf d_j).$$9 is the number of unique grounded citations in the answer (Strauss et al., 27 Jun 2025). Using approximately 14,000 LMArena conversation logs, it identifies three exploitation patterns: No Search, No citation, and High-volume, low-credit. Gemini has zero citations in 92% of answers, Sonar visits a median of 10 sites and cites a median of 5, and citation efficiency varies from 0.19 to 0.45 additional citations per additional URL opened under head-to-head regression. The paper recommends a transparent LLM search architecture based on standardized telemetry and full disclosure of search traces and citation logs, including OpenTelemetry-style fields such as LLM.retrieval.ids and LLM.retrieval.scores (Strauss et al., 27 Jun 2025). In a TrackStar-like system, this turns provenance from an internal diagnostic into an auditable systems interface.

These citation and rule-level frameworks share a strong common design choice: attribution is enforced structurally at generation time, either through retrieval-augmented citation validation or through schema-constrained claim-to-rule mapping. This sharply contrasts with footer-style post-hoc citation, which the same literature repeatedly treats as fragile.

5. TrackStar as training-data retrieval attribution

TrackStar in the strict sense belongs to training-data attribution. Bergson places it alongside MAGIC and SOURCE, but characterizes it differently: TrackStar is a single-checkpoint influence approximation that projects per-example gradients into a low-dimensional space, builds an empirical Fisher or autocorrelation Hessian approximation on the compressed gradients, uses unit normalization and optionally optimizer-based gradient normalization, and then retrieves top-$s(q,d)$0 influential training points by a preconditioned inner product in the projected space (Quirke et al., 10 Jun 2026). In Bergson’s summary, TrackStar is explicitly retrieval-oriented: it is designed to index training gradients and retrieve influential points, and Bergson provides the first open-source implementation of TrackStar (Quirke et al., 10 Jun 2026).

The mathematical lineage runs through TRAK. TRAK defines attribution as tracing a model prediction back to specific training data by turning the trained model into a kernel machine over projected gradient features, then applying leave-one-out style approximations in that feature space (Park et al., 2023). The paper’s projected feature map is

$s(q,d)$1

and the resulting single-model estimator is

$s(q,d)$2

with $s(q,d)$3 (Park et al., 2023). TRAK is effective and computationally tractable for large-scale differentiable models, and the retrieval interpretation is direct: positive scores identify “proponents,” negative scores “opponents,” and the projected gradients themselves can be indexed for nearest-neighbor retrieval (Park et al., 2023).

Bergson’s TrackStar implementation adopts this retrieval logic at pretraining scale. It uses per-module Kronecker random projections, empirical Fisher in compressed space, optional Adam-based normalization, and FAISS-backed top-$s(q,d)$4 lookup over on-disk gradient stores (Quirke et al., 10 Jun 2026). In Bergson’s fact-tracing-across-styles case study, TrackStar improves Top-1 from 0.00% to 7.70%, Top-5 from 0.80% to 15.26%, and reduces Style Leak@1 from 73.01% to 33.80% relative to a raw gradient inner product baseline (Quirke et al., 10 Jun 2026). That result is particularly relevant for retrieval attribution because it shows that compressed influence retrieval can separate factual content from superficial stylistic similarity.

The theory paper “Imperfect Influence, Preserved Rankings” supplies the main caution. It shows that TRAK’s approximations incur significant errors, but the estimated influence remains highly correlated with the original influence and therefore largely preserves the relative ranking of data points (Tong et al., 1 Feb 2026). In the paper’s decomposition, the dominant error source is linearization, ALO contributes only lower-order error, and projection quality depends on the ratio $s(q,d)$5 and the signal scale. The theoretical message is that absolute influence values are poorly calibrated, yet rankings—especially at the top—remain useful (Tong et al., 1 Feb 2026). For retrieval systems, that is a decisive distinction: TrackStar is better read as a ranking tool than as a precisely calibrated causal estimator.

An emerging alternative preserves the same first-order counterfactual target while removing per-query backward passes. “Scalable Data Attribution via Forward-Only Test-Time Inference” simulates each training example’s parameter influence through short-horizon gradient propagation during training and later reads out attributions for any query using only forward evaluations, yielding orders-of-magnitude lower inference cost on standard MLP benchmarks while matching or surpassing baselines such as TRAK on LOO and LDS (Ma et al., 25 Nov 2025). This suggests a future TrackStar-like direction in which attribution remains influence-based but shifts even more computation from query time to offline preprocessing.

6. Evaluation instability, failure modes, and design consequences

The sharpest evaluation warning comes from “Do LLM Attribution Metrics Transfer?” The paper audits eight automatic scorers across provenance/topicality, generated-answer attribution, and fact-check entailment, and defines transfer as staying within the 95% confidence interval of the best audited scorer on every dataset in a multi-dataset construct. In generated-answer attribution, none does (Ding et al., 22 Jun 2026). Metric rankings invert across datasets: Kendall tau is $s(q,d)$6 on AttributedQA versus LFQA, an off-the-shelf NLI scorer that reaches AUROC 0.90 on AttributedQA collapses to AUROC 0.53 on LFQA, where BERTScore reaches 0.91, and a naive “best-on-average” rule incurs mean held-out regret 0.172 AUROC. A prompt-based LLM judge avoids the chance-level collapse on LFQA but is ~100x costlier and non-deterministic (Ding et al., 22 Jun 2026). For TrackStar Retrieval Attribution, this means that attribution metrics themselves require target-dataset validation; provenance scoring is not portable by default.

Across the broader literature, several recurrent failure modes appear. In query-side retrieval attribution, low attribution does not necessarily mean semantic unimportance to the user, Integrated Gradients can suffer from gradient saturation or baseline sensitivity, and the overall pipeline is computationally expensive because it requires retrieval, IG over top-$s(q,d)$7 documents, and an LLM rewrite (Garouani et al., 12 Feb 2026). In TRACE, the method assumes white-box gradient access and relies on recurrence across documents; if an attacker injects only a few poisoned documents, recurrence thresholds may not be met (Chen et al., 24 Jun 2026). In long-form answer attribution, unsupported content is frequently driven by retrieval failure, hallucinated facts, or incorrect synthesis, and the later parts of long answers are especially vulnerable (Chen et al., 2023). In regulatory attribution, even RefWalk leaves FullCov@10 below 50% on hard queries, and the paper explicitly identifies a precision–recall trade-off in citation behavior (Ju et al., 28 May 2026). In web-search attribution, the apparent attribution gap depends on telemetry completeness, and GPT-4o’s very low measured gap is argued to be best explained by selective log disclosures rather than better attribution (Strauss et al., 27 Jun 2025).

The cumulative design consequence is clear. Retrieval attribution systems should separate structural leakage from genuine signal, measure completeness as well as local support, distinguish ranking quality from calibration quality, and prefer integrated attribution mechanisms over post-hoc citation lists. They should also expose provenance through machine-readable telemetry or schema-constrained outputs, because attribution that cannot be externally audited remains operationally fragile. In this sense, the literature treats attribution not merely as explanation but as an organizing principle for retrieval design, evaluation, and control.