dedupT: Transformer Crash Deduplication

• dedupT is a transformer-based crash deduplication method that treats stack traces as unified documents instead of isolated frames.
• It preprocesses stack traces, adapts a pretrained language model through contrastive learning, and employs pairwise ranking to detect duplicates.
• Empirical results show dedupT enhances Mean Reciprocal Rank by over 15% and maintains high ROC-AUC across various datasets.

Searching arXiv for the primary and related papers on dedupT and neighboring deduplication methods. dedupT is a transformer-based approach to stack trace-based crash deduplication that groups automated crash reports by treating each crash’s stack trace or stack traces as a single “document,” rather than a bag of isolated frames (Mamun et al., 26 Aug 2025). In its most specific usage, the term denotes the pipeline introduced in “Stack Trace-Based Crash Deduplication with Transformer Adaptation” (Mamun et al., 26 Aug 2025), where a pretrained LLM is adapted to stack traces and paired with a lightweight ranking network for duplicate ranking and unique crash detection. Related literature uses closely allied deduplication ideas in adjacent settings, including stack-trace retrieval-and-reranking (Shibaev et al., 2024) and record deduplication for entity-distribution modeling in ASR transcripts (Huang et al., 2023), but dedupT as a named method is primarily associated with crash-report deduplication.

1. Problem setting and conceptual scope

Crash deduplication addresses the problem that automated crash reporting systems generate large volumes of duplicate reports, overwhelming issue-tracking systems and increasing developer workload (Mamun et al., 26 Aug 2025). In this setting, the object to be deduplicated is not arbitrary text but a crash report whose most informative artifact is usually one or more stack traces. dedupT models those traces holistically rather than as isolated frames, with the explicit goal of improving both duplicate ranking and unique crash detection (Mamun et al., 26 Aug 2025).

The pipeline consists of four stages: stack-trace preprocessing, domain-specific adaptation of a pretrained LLM (PLM) to produce stack-trace embeddings, feature-level aggregation of possibly multiple traces per crash report, and a lightweight, fully-connected ranking network that scores candidate crash pairs (Mamun et al., 26 Aug 2025). This organization places dedupT in a family of learned deduplication systems, but its defining characteristic is the combination of transformer adaptation and pairwise ranking over crash-report representations.

A useful contrast is provided by “Stack Trace Deduplication: Faster, More Accurately, and in More Realistic Scenarios” (Shibaev et al., 2024), which uses an embedding model with byte-pair encoding and approximate nearest neighbor search to retrieve candidates, followed by a reranker that takes repeated frames into account. This suggests that recent stack-trace deduplication research has increasingly emphasized representation learning and multi-stage ranking rather than only string similarity, rule-based heuristics, or alignment-based matching.

2. Stack-trace preprocessing and document construction

dedupT begins by normalizing raw stack traces into a form suitable for transformer encoding (Mamun et al., 26 Aug 2025). Consecutive frames with the same subroutine, such as recursive calls, are collapsed. This duplicate removal step reduces redundant local repetition before any embedding is computed.

The method then applies top-$N$ sampling. Empirically, the top 10–15 frames carry the most diagnostic signal, and all later frames are dropped to respect the PLM’s maximum token length, typically 512 (Mamun et al., 26 Aug 2025). The retained trace is therefore a deliberately pruned representation of the crash, optimized for the length and inductive biases of sentence-transformer models.

Frame cleaning is language-aware. Language-specific syntax, including dots in Java packages and debugger prefixes in C/C++, is replaced by whitespace, and all tokens are lower-cased (Mamun et al., 26 Aug 2025). Each stack frame is treated as a “sentence,” and the entire pruned trace is treated as one passage. dedupT also prepends or appends special tokens to each frame to preserve its ordinal position in the call stack (Mamun et al., 26 Aug 2025). This positional encoding is central to the method’s claim to holistic modeling: frame identity is not used in isolation, but in sequence context.

The preprocessing choices are not merely implementation details. The reported ablations show that frame count matters: MRR peaked using the top 10–12 frames, while retaining too many ($>20$) or too few ($<5$) degraded performance (Mamun et al., 26 Aug 2025). Frame trimming was also dataset-dependent: no trimming was best on Netbeans, whereas aggressive trimming improved Eclipse (Mamun et al., 26 Aug 2025). These results indicate that representation quality depends on a balance between preserving diagnostic context and avoiding length-induced noise.

3. PLM adaptation and contrastive embedding learning

dedupT starts from a sentence-transformer PLM and, in the reported experiments, uses bge-base-en with 512-dimensional embeddings (Mamun et al., 26 Aug 2025). The model is fine-tuned on labeled crash data to better capture structural and semantic relationships in stack traces. Training examples are constructed from duplicate buckets: an anchor $x_a$ is any stack trace from bucket $B_i$, a positive $x_p$ is sampled from the same bucket, and negatives $\{x_n^j\}$ are sampled from other buckets $B_j, j \neq i$ (Mamun et al., 26 Aug 2025).

The embedding model is optimized with the Multiple Negatives Ranking loss:

$$L_{\mathrm{MNR}} = - \frac{1}{N} \sum_{i=1}^{N} \log \frac{\exp\!\left(\mathrm{CosSim}(f_\theta(x_a^i), f_\theta(x_p^i))/\tau\right)} {\sum_{j=1}^{N} \exp\!\left(\mathrm{CosSim}(f_\theta(x_a^i), f_\theta(x_n^j))/\tau\right)}.$$

Cosine similarity is defined as

$$\mathrm{CosSim}(u,v) = \frac{\langle u,v\rangle}{\|u\|\,\|v\|}.$$

Fine-tuning sharpens the embedding space so that duplicates become closer than non-duplicates (Mamun et al., 26 Aug 2025). The reported adaptation effect is substantial. On Netbeans, the cosine-Pearson correlation between model similarity scores and ground truth rose from $>20$0, Spearman rose from $>20$1, and retrieval MRR improved from $>20$2 (Mamun et al., 26 Aug 2025). Fine-tuning is done on the training split only, and once converged, all crash-report embeddings in train, validation, and test sets can be precomputed and cached for efficiency (Mamun et al., 26 Aug 2025).

The paper also reports that after fine-tuning, different pretrained PLMs, including mpnet, distilroberta, and bge, performed within approximately $>20$3–$>20$4 of each other, with sequence-length limits partly explaining the small differences (Mamun et al., 26 Aug 2025). This suggests that domain adaptation is more important than any single off-the-shelf encoder choice, provided the encoder can represent stack-trace sequences with adequate length budget.

4. Multi-trace aggregation and duplicate ranking

A crash report $>20$5 may contain multiple stack traces $>20$6. dedupT first embeds each trace individually, $>20$7, and then merges them using Parametric Max-Mean aggregation (Mamun et al., 26 Aug 2025). The method first identifies the most similar cross-report pair:

$>20$8

It then computes mean-pooled report embeddings and defines the fused report representation as

$>20$9

where $<5$0 is learnable and $<5$1 denotes concatenation (Mamun et al., 26 Aug 2025). In the reported ablation, Parametric Max-Mean outperformed max-only, mean-only, and multi-head-attention pooling on Netbeans, with MRR $<5$2 versus $<5$3, $<5$4, and $<5$5, respectively (Mamun et al., 26 Aug 2025).

For pairwise scoring, dedupT constructs the joint feature vector

$<5$6

that is, absolute difference, element-wise average, and element-wise product (Mamun et al., 26 Aug 2025). This vector is passed through a two-layer fully connected network with dropout $<5$7:

$<5$8

Training uses RankNet loss on triplets $<5$9:

$x_a$0

The classifier is trained with Adam using learning rate $x_a$1 and batch size $x_a$2, on an Intel Xeon Gold 6148 CPU and NVIDIA V100 16 GB GPU, with peak GPU memory below $x_a$3 GB (Mamun et al., 26 Aug 2025). Classification training converges in minutes to a few hours, and end-to-end embedding adaptation plus classifier training completes in under 3 hours per dataset (Mamun et al., 26 Aug 2025).

5. Empirical performance, ablations, and operating characteristics

dedupT was evaluated on four public datasets: Netbeans and Eclipse for Java, and Gnome and Ubuntu for C/C++ (Mamun et al., 26 Aug 2025). The reported summary metrics are as follows.

Dataset	MRR	ROC-AUC
Netbeans	0.771	0.933
Eclipse	0.791	0.891
Gnome	0.742	0.905
Ubuntu	0.786	0.905

On duplicate retrieval, Netbeans achieved MRR $x_a$4 versus best traditional $x_a$5 and best deep-learning baseline $x_a$6; Eclipse achieved $x_a$7 versus $x_a$8 and $x_a$9; Gnome achieved $B_i$0 versus $B_i$1 and $B_i$2; and Ubuntu achieved $B_i$3 versus $B_i$4 and $B_i$5 (Mamun et al., 26 Aug 2025). On unique-report detection, ROC-AUC reached $B_i$6 on Netbeans, $B_i$7 on Eclipse, $B_i$8 on Gnome, and $B_i$9 on Ubuntu (Mamun et al., 26 Aug 2025). The paper states that on four public datasets, dedupT improves Mean Reciprocal Rank often by over $x_p$0 compared to the best DL baseline and up to $x_p$1 over traditional methods, while achieving higher ROC-AUC in detecting unique crash reports (Mamun et al., 26 Aug 2025).

The ablations clarify where these gains come from. Fine-tuning the PLM materially improves correlation and retrieval quality; Parametric Max-Mean is the strongest aggregation mechanism among those tested; and frame-count selection has a nontrivial optimum in the 10–12 range (Mamun et al., 26 Aug 2025). The paper also reports that transformer inference is heavier than purely IR methods, though parallelization and cached embeddings mitigate runtime (Mamun et al., 26 Aug 2025). This is a limitation rather than a contradiction of the method’s practicality, because dedupT shifts much of the computational burden into offline adaptation and embedding precomputation.

6. Relation to adjacent uses of the term and broader deduplication research

The label “dedupT” is not completely unique across the literature. In ASR, Huang et al. describe “dedupT” as a record-deduplication-based pipeline for modeling and biasing the true entity distribution in ASR transcripts, using blocking, pairwise similarity, and graph clustering rather than transformer adaptation over stack traces (Huang et al., 2023). In ontology integration, the term appears as a descriptive name for a deduplication function inspired by OntoMerger, where mappings, source priority, and hierarchy reconnection determine canonical merges in knowledge graphs (Geleta et al., 2022). These usages are technically distinct from crash-report deduplication.

Within stack-trace deduplication more specifically, Shibaev et al. present a two-stage system composed of an embedding model with byte-pair encoding and approximate nearest neighbor search, plus a reranker that accounts for repeated frames (Shibaev et al., 2024). Their evaluation includes Acc@1, ROC-AUC, and inference time, and on SlowOps the two-stage model reaches Acc@1 $x_p$2 and ROC-AUC $x_p$3 (Shibaev et al., 2024). A plausible implication is that the field now contains two complementary design lines: candidate-retrieval pipelines with ANN indexing and reranking (Shibaev et al., 2024), and transformer adaptation with cached embeddings and pairwise ranking networks (Mamun et al., 26 Aug 2025).

dedupT’s reported strengths are holistic modeling of stack traces as contextual sequences rather than isolated frames, domain adaptation via contrastive fine-tuning, a parametric aggregation mechanism that balances the most similar frame pair with global trace context, and significant gains in both ranking and unique-crash detection over traditional and deep-learning baselines (Mamun et al., 26 Aug 2025). Its reported limitations are the requirement for labeled duplicate buckets, sensitivity to stack-trace length and trimming policies, and heavier transformer inference than purely IR methods (Mamun et al., 26 Aug 2025). Taken together, these properties position dedupT as a technically specific instance of a broader shift toward representation-learning-based deduplication, while preserving the practical objective that motivates the field: reducing manual triage effort in large-scale issue-tracking systems (Mamun et al., 26 Aug 2025).