DCAT: Deduplicated Cross-Attention Transformer

• DCAT is a neural architecture that integrates dual-branch feature extraction and bidirectional cross-attention to deduplicate redundant information.
• It employs adaptive gating and convolutional block attention to filter irrelevant features while providing uncertainty estimation for robust decision-making.
• The model achieves high diagnostic accuracy in medical imaging by combining complementary outputs from networks like EfficientNetB4 and ResNet34.

The Deduplicated Cross-Attention Transformer (DCAT) refers to a class of neural architectures that leverage cross-attention mechanisms to integrate and deduplicate information across multiple sources, modalities, or hierarchical representations in a computationally efficient and interpretable manner. DCAT models are particularly prevalent in advanced medical image classification, multimodal integration, and scenarios where complementary feature extraction and uncertainty estimation are essential for reliable decision-making (Borah et al., 14 Mar 2025).

1. Architectural Overview

DCAT architectures are characterized by a dual-branch or dual-stream design, each branch employing a distinct backbone network—such as EfficientNetB4 and ResNet34—to extract multi-scale, complementary representations from the same input image (e.g., radiological or OCT images). After initial feature extraction, the feature maps from corresponding layers of both networks are concatenated to form multi-scale hierarchical tensors:

$$F^1 = [X_1^{(\mathrm{eff})}, X_1^{(r)}] \in \mathbb{R}^{2C_1 \times 28 \times 28}, \quad F^2 = [X_2^{(\mathrm{eff})}, X_2^{(r)}] \in \mathbb{R}^{2C_2 \times 14 \times 14}$$

where $X_k^{(\mathrm{eff})}$ and $X_k^{(r)}$ represent features from EfficientNetB4 and ResNet34 at resolution level $k$. The most distinctive component is a bidirectional cross-attention fusion module, in which each branch leverages the features from the other as the source for queries, keys, and values. This results in a fusion operation such as:

$$\begin{align*} Q_{\mathrm{eff}} &= W_q \cdot X_{\mathrm{img}}^{(r)} \ K_{\mathrm{eff}} &= W_k \cdot X_{\mathrm{img}}^{(\mathrm{eff})}, \quad V_{\mathrm{eff}} = W_v \cdot X_{\mathrm{img}}^{(\mathrm{eff})} \ Q_{r} &= W_q \cdot X_{\mathrm{img}}^{(\mathrm{eff})} \ K_{r} &= W_k \cdot X_{\mathrm{img}}^{(r)}, \quad V_{r} = W_v \cdot X_{\mathrm{img}}^{(r)} \ \end{align*}$$

The fused representation is obtained by combining the two attention outputs:

$$F_{\mathrm{fusion}} = \mathrm{Attention}(Q_{\mathrm{eff}}, K_r, V_r) + \mathrm{Attention}(Q_r, K_{\mathrm{eff}}, V_{\mathrm{eff}})$$

This bidirectional formulation enables mutual refinement and deduplication of redundant or non-informative features, resulting in more discriminative representations.

2. Mechanism of Deduplication via Cross-Attention

Deduplication in DCAT is realized by the selective gating and filtering properties of the cross-attention mechanism. In the context of skip connections or multi-network fusion, cross-attention acts as an adaptive filter that reweights features based on their mutual relevance, suppressing redundant or non-semantic information. Specifically, the attention operation can be interpreted mathematically as:

$$\mathrm{Attention}(Q, K, V) = \mathrm{Softmax}\left(\frac{QK^{\top}}{\sqrt{d_k}}\right) V$$

Following attention, further refinement is achieved via attention modules such as the Convolutional Block Attention Module (CBAM), which applies:

• Channel attention: Aggregates global information using average and max pooling across spatial dimensions, followed by an MLP and sigmoid activation, producing a vector $M_c \in \mathbb{R}^{C \times 1 \times 1}$ that gates channel importance.
• Spatial attention: Aggregates information across channels, followed by convolution and sigmoid activation, yielding $M_s \in \mathbb{R}^{1 \times H \times W}$ for spatial reweighting.

These steps enable the model to promote discriminative, non-duplicative features and diminish irrelevant or repetitive signals, thus "deduplicating" the final representation (Petit et al., 2021).

3. Bidirectional Fusion and Multi-Network Context

A key aspect of DCAT is bidirectional cross-attention, where each network's extracted features serve alternately as queries and sources for key-value pairs. This is formally captured in the dual attention equations above, and ensures that context mutually informs both branches:

• EfficientNetB4 queries ResNet34's feature space and vice versa.
• The outputs are symmetrically fused, allowing the unique strengths of each backbone to complement the weaknesses of the other.

This dynamic bidirectional information flow helps capture subtle context and dependencies that would not be possible using simple concatenation or self-attention alone. It is particularly impactful in tasks where critical disease patterns might only be evident at certain scales or in highly specific feature subspaces (Borah et al., 14 Mar 2025).

4. Uncertainty Estimation and Interpretability

DCAT integrates Monte Carlo Dropout for uncertainty quantification. During inference, dropout is applied and the model outputs are sampled multiple times, producing a set of stochastic softmax predictions. The empirical predictive distribution is:

$$p(y|x) \approx \frac{1}{M} \sum_{m} \mathrm{Softmax}(f_\theta^{(m)}(x))$$

The entropy of this distribution quantifies uncertainty for each input:

$H(p(y|x)) = -\sum_{c=1}^C p(y=c|x) \log p(y=c|x)$

High-entropy samples indicate greater epistemic uncertainty, which allows clinicians to focus human expertise where the model is less confident. Visualizations of entropy maps and attention weights further support interpretability, highlighting which spatial regions most contribute to the model's prediction and which cases warrant additional review.

5. Performance Metrics and Empirical Evaluation

DCAT demonstrates strong empirical results on medical image classification tasks, including chest X-rays and optical coherence tomography (OCT) images for diseases such as Covid-19, tuberculosis, pneumonia, and retinal disorders. The architecture achieves:

• Area Under ROC Curve (AUC) up to 100%
• Area Under Precision-Recall Curve (AUPR) up to 100%
• High accuracy, F1 scores, MCC, and Cohen’s Kappa

These scores are consistently achieved across imbalanced and multi-class datasets, indicating that the deduplicated cross-attention mechanism robustly captures relevant features while suppressing redundancy. The addition of uncertainty estimation does not degrade base performance, instead contributing to increased model transparency and clinical safety (Borah et al., 14 Mar 2025).

6. Comparative Analysis and Domain Applications

Compared to earlier architectures that use single backbone networks or naïve feature concatenation, DCAT’s dual cross-attention fusion delivers superior accuracy, interpretability, and reliability. The combination of multi-network feature extraction, bidirectional attention fusion, and explicit uncertainty modeling addresses both overfitting and overconfidence, which are common pitfalls in clinical machine learning pipelines.

DCAT is most impactful in applications demanding high diagnostic sensitivity and specificity under complex, multi-scale visual conditions. Its design is particularly suited to:

• Medical diagnostics, where complementary visual cues must be fused and uncertainty estimation is vital for clinical decision support.
• Scenarios requiring the explicit elimination of redundant information while preserving fine-scale discriminative patterns.

7. Broader Implications and Extensions

The principles underlying DCAT—namely, cross-attention driven deduplication, bidirectional fusion of heterogeneous features, and integration with uncertainty quantification—generalize beyond medical imaging. The architecture provides a template for building transparent, efficient, and robust classifiers in domains where multiple complementary data streams or representations are available and where decision reliability is critical.

A plausible implication is that similar deduplicated cross-attention mechanisms could be extended to multi-modal data fusion, sequential decision making in high-stakes domains, or any context where interpretability and explicit uncertainty estimation are essential requirements alongside high model accuracy.

---

Table: Core DCAT Components and Functions

Component	Function	Reference Section
Dual-branch backbone	Multi-scale, complementary feature extraction	1, 3
Bidirectional cross-attention	Contextual feature fusion and deduplication	1, 2, 3
Channel & spatial attention (CBAM)	Refinement of discriminative features	2
Monte Carlo Dropout	Uncertainty estimation and transparency	4
Multi-metric evaluation	Robustness across disease types & dataset imbalance	5