DeepTRACE: Traceability for AI Systems

• DeepTRACE is a set of advanced traceability methodologies linking artifacts in deep learning systems to ensure transparent audit trails in safety-critical, forensic, epidemic, and research domains.
• It employs diverse techniques such as artifact mapping in DNNs, convolutional trace extraction for DeepFake detection, and GNN-regressed likelihood estimation for epidemic source identification.
• DeepTRACE also provides an audit framework for generative AI systems, focusing on evidence attribution, citation accuracy, and the mitigation of unsupported or overconfident outputs.

DeepTRACE refers to a set of distinct audit, analysis, and tracking frameworks at the intersection of deep learning, AI system traceability, and generative model forensics. The term has been used in peer-reviewed research to denote (1) an artifact-oriented traceability protocol for deep neural networks in safety-critical environments, (2) a forensics-driven method for detecting GAN-generated DeepFake images via convolutional trace extraction, (3) a GNN-based strategy for epidemic network source tracing, and (4) a contemporary audit methodology for reliability evaluation of generative AI systems with respect to citation and factual support. Each instantiation operationalizes the concept of “trace”—a provable, bidirectional link between artifacts or events—towards transparent, robust, and explainable outcomes in its respective subdomain.

1. Traceability of Deep Neural Networks in Safety-Critical Systems

DeepTRACE, introduced by Aravantinos and Diehl, generalizes classical software requirements traceability to the context of deep neural networks (DNNs), addressing the collapse of Low-Level Requirements (LLR) and software architecture layers that traditionally underpin safety standards like DO-178C and ISO 26262. In DNN pipelines, explicit code-to-requirement mappings are replaced by a series of artifact streams and experiment chains, due to the black-box nature of trained weights and the trial-and-error modality of model production (Aravantinos et al., 2018).

The protocol introduces new artifact classes:

Artifact Type	Symbol/Set	Example Instance
High-Level Requirements	$\mathcal{H}$	“Detect pedestrians”
Domain Coverage Model	$\mathcal{D}$	“Roundabout at dusk”
Raw Training Dataset / Labels	$\mathcal{S}$	Subset of annotated images
DNN Architecture	$\mathcal{A}$	Network topology
Learning Configuration (loss, optimizer, HPs)	$\mathcal{C}$	SGD, cross-entropy, LR=0.01
Trained Weight Values	$\mathcal{W}$	Model checkpoint .pth file
Inference Architecture	$\mathcal{A}'$	Deployment serialization
Test/Validation Metrics	$\mathcal{M}$	Accuracy, safety-weighted score

Trace links $t_{X\leftrightarrow Y}\subseteq X\times Y$ formalize the justification or refinement between any two artifacts. The workflow encompasses (1) decomposition of HLRs via a domain coverage model to ensure dataset representativity, (2) mapping each datum through domain fragments back to requirements, and (3) an experiment-to-experiment chain, where each step is justified by measurable improvement in a safety-critical metric. The process restores “rationale” in the absence of source code to LLR chains.

Key challenges remain unresolved: specific traceability of neurons or weights to requirements is unattainable, version misdirection is only partially mitigated by artifact locking and controlled evolution, and coverage modeling must adapt to incremental, real-world data settings. The approach allows for the inclusion of expert rules and hybrid systems as traceable artifacts (Aravantinos et al., 2018).

2. DeepTRACE in DeepFake Forensics via Convolutional Traces

In the forensics domain, DeepTRACE designates a framework to detect GAN-generated DeepFake images by extracting and leveraging explicit “convolutional traces”—fingerprints left by transpose-convolution layers in generator architectures (Guarnera et al., 2020). Unlike end-to-end deep networks, this approach relies on reverse engineering and explicit modeling.

The core pipeline involves:

1. For each image channel (R, G, B), the Expectation-Maximization (EM) algorithm estimates a local convolution kernel $\mathbf{k}\in\mathbb{R}^{N\times N}$ that best explains observed pixel correlations.
2. These kernels, concatenated to form a feature vector, become the basis for lightweight classifiers (K-NN, SVM, LDA) tasked with discriminating between genuine camera-captured images and DeepFakes produced by specific GANs.
3. Typical kernel sizes ($N\in\{3,4,5,7\}$) correspond to the known last-layer kernel shapes in popular GAN architectures.

The formal model assumes a two-component mixture: a convolutional model $M_1$ explaining most pixels and an outlier “noise” model $M_2$. The EM maximizes the complete log-likelihood given the respective assignments. Features excluding the kernel center, for all three channels ($3(N^2 - 1)$), yield high discriminative power: accuracy exceeds 99% for several GANs, outperforming standard deep features (VGG-16 obtains only ~53% on binary tasks).

The framework facilitates both real/fake classification and GAN attribution, with effectiveness relying on accurate kernel size priors. Limitations include a focus on static, aligned face crops and unknown robustness to photometric or geometric transforms, suggesting future work on more generalized, adversarially-robust EM-based feature extraction (Guarnera et al., 2020).

3. Graph Neural Network–Driven DeepTrace for Epidemic Source Identification

DeepTrace, as applied to digital contact tracing, models the problem of tracking epidemic source nodes as an ML estimation problem defined over temporal subgraphs of social contact networks (Tan et al., 2022). The protocol reframes forward and backward contact tracing as online graph exploration equipped with a GNN-accelerated maximum-likelihood estimator.

The formal problem centers on estimating, at each stage $n$ with observed subgraph $G_n$, the source node $$\hat{v}_n = \arg\max_{v\in V(G_n)} \mathbb{P}(G_n | v)$$, where the likelihood is obtained by summing over all “permitted permutations” (infection orders) starting at node $v$. Direct enumeration is infeasible due to combinatorial explosion; thus, DeepTrace uses a GraphSAGE-style GNN regressing approximations of these log-likelihoods:

• Node features include degree ratio, infected-neighbor proportion, and normalized boundary distance.
• Updates are performed via LSTM aggregators and ReLU activations, with regression targets sourced from Monte Carlo permutation sampling in pretraining and exact calculation in small-graph fine-tuning.
• The framework integrates BFS or DFS for forward exploration, as dictated by the practical tracing scenario.

Empirical validation on synthetic and real COVID-19 networks demonstrates top-1 accuracy rates of 71% on clusters up to 2,500 nodes post-fine-tuning, with per-step computational cost linear in graph edges and sub-100 ms latency per tracing step. DeepTrace thus provides scalable, interpretable estimation for epidemic source tracing beyond classical rumor centrality or brute-force likelihood enumeration (Tan et al., 2022).

4. DeepTRACE: Auditing Evidence Attribution in Research AI Systems

DeepTRACE, in its most recent form, operationalizes a comprehensive audit methodology to evaluate generative search engines (GSEs) and deep research LLM agents for reliability in statement grounding, source attribution, and citation practices (Venkit et al., 2 Sep 2025). Prompted by real-world failure cases (overconfident responses, unsupported claims, misleading citations), DeepTRACE implements eight measurable dimensions across answer text quality, source usage, and citation correctness.

The framework details:

• Statement-Level Decomposition and Labelling: Each answer is segmented into atomic statements, annotated for relevance and stance (pro/con/neutral on debate queries) using LLMs.
• Confidence Scoring: LLM-judged Likert scale (1–5) quantifies answer assertiveness; overconfidence is flagged when strongly confident one-sided answers are produced.
• Citation and Factual-Support Matrices: Two binary matrices—Citation ($C$), built from in-text markers, and Factual Support ($F$), generated via LLM-judged source–statement alignment—allow rigorous metric computation.

Eight quantitative metrics are tracked, including one-sidedness, overconfidence, relevant statement proportion, fraction of uncited sources, unsupported statements, minimal source necessity, citation accuracy (fraction of citation links truly supported), and citation thoroughness (fraction of all supports that are cited). These enable high-fidelity evaluation of systems' end-to-end reasoning and evidence attribution.

A summary of selected empirical metrics illustrates the current state of leading GSE and DR models:

Metric	GSE (You.com)	GSE (BingChat)	DR (GPT5 DR)	DR (YouChat DR)
One-Sided Answer (%)	51.6	48.7	54.7	63.1
Overconfident Answer (%)	19.4	29.5	15.2	19.6
Relevant Statements (%)	75.5	79.3	87.5	45.5
Unsupported Statements (%)	30.8	23.1	12.5	74.6
Citation Accuracy (%)	68.3	65.8	79.1	72.3
Citation Thoroughness (%)	24.4	20.5	87.5	83.5

Findings indicate large fractions of unsupported statements (up to 47%), moderate citation accuracy (40–80%), and pervasive one-sidedness. “Deep-research” configurations attain higher citation thoroughness but at times suffer from reduced relevance and persistent one-sidedness. Automated LLM-judging shows moderate to substantial agreement with human annotators ($r=0.72$ for confidence scoring, $r=0.62$ for fact alignment).

DeepTRACE is implemented via an automated extraction pipeline comprising web scraping, statement decomposition and annotation (GPT-5), confidence assignment, source retrieval, and matrix construction. The approach enables continuous, scalable sociotechnical auditing, highlights latent systemic weaknesses, and suggests design interventions (e.g., highlighting source necessity, surfacing uncertainty on unbalanced answers) for more trustworthy generative search and research agents (Venkit et al., 2 Sep 2025).

5. Comparative Synthesis and Implications

All instantiations of DeepTRACE operationalize rigorous, artifact-level traceability for their respective domains, substituting brute-force transparency or explainability with measurable, artifact-driven or matrix-driven rationale. Whether through experiment-to-experiment evolution graphs in neural system engineering, explicit kernel estimation in forensics, log-likelihood–regressing GNNs in network epidemiology, or matrix-based citation audits in LLM-based research frameworks, DeepTRACE protocols systematically address the problem of unobservable or opaque process steps by reconstructing or empirically quantifying the linkages between critical artifacts, inputs, or claims.

A plausible implication is that continuous, tightly-coupled artifact tracking—whether for safety, forensic discrimination, public health, or source reliability—constitutes a necessary condition for deploying deep learning and generative AI systems in settings demanding auditability, accountability, and end-to-end justification.

6. Limitations and Future Directions

Known limitations of DeepTRACE variants include persistent interpretability challenges (e.g., mapping neural weights to semantic requirements is still infeasible), susceptibility to adversarial or incremental data shifts (dataset evolution), constrained generalization in forensics protocols (fixed alignment, kernel priors), and reliance on automated LLM judgment pipelines for reliability audits. Contemporary audit frameworks do not consider multimodal outputs or precise user interface placement of citations.

Prospective developments include integrating explainability tools into DNN trace graphs, extending forensic trace extraction to video and adversarially-robust regimes, leveraging vision-based UI analysis in audit pipelines, and operationalizing human-in-the-loop review for high-stakes research support environments. Such efforts can further reinforce the auditability and trustworthiness of AI systems in critical domains.