---
title: 'CPAC: Causal Prototype Attention Classifier'
url: https://www.emergentmind.com/topics/causal-prototype-attention-classifier-cpac
type: topic
---

# CPAC: Causal Prototype Attention Classifier

Searching arXiv for the cited CPAC-related papers and closely related context.
Found the main ERP-XTTN paper and the fraud-detection CPAC paper; also checking baseline/context papers for naming consistency.
The **Causal Prototype Attention Classifier (CPAC)** denotes a prototype-based attention paradigm that appears in two distinct arXiv formulations. In **ERP-XTTN**, CPAC is the classifier component of an interpretable, calibration-free architecture for cross-subject event-related potential classification, implemented through **query-key-only cross-attention with no value projection** so that **classification depends entirely on attention routing and attention faithfulness is structural rather than post-hoc** [2606.02939]. In **fraud detection**, CPAC is an interpretable latent classifier coupled to a VAE-GAN encoder, using **prototype-based attention mechanisms** to promote **class-aware clustering and improved latent space structure** under extreme class imbalance [2507.14706]. The shared theme is prototype-guided decision making; the operational form, learning objective, and meaning of “causal” differ materially across the two settings.

## 1. Terminological scope and architectural variants

The term CPAC does not refer to a single invariant neural module. In the ERP setting, it is instantiated as a **cross-attention architecture that routes input EEG patches to fixed difference-wave prototypes via query-key-only cross-attention with no value projection**. In the fraud setting, it is instantiated as a **prototype bank** together with an **attention module** that produces a **feature-wise attention mask** and a classifier defined by weighted distances to class prototypes [2606.02939].

| Aspect | ERP-XTTN CPAC | Fraud-detection CPAC |
|---|---|---|
| Input representation | EEG epoch patches | Latent vector $z$ |
| Prototype type | Fixed difference-wave prototypes | Two learnable vectors in $\mathbb{R}^d$ |
| Decision mechanism | Flattened attention routing matrix $A$ | Softmax over negative weighted distances |

This suggests a family resemblance rather than a single canonical architecture. In both formulations, prototypes act as anchors for class-relevant structure, but the ERP variant makes attention itself the sole evidentiary pathway, whereas the fraud variant uses attention to gate per-dimension distances to prototypes. A plausible implication is that “CPAC” is best understood as a design pattern centered on prototype-conditioned routing or distance weighting, rather than as one fixed network topology.

## 2. Query-key-only CPAC in ERP-XTTN

In ERP-XTTN, an input EEG epoch is defined as $X\in\mathbb{R}^{C\times T}$, with $C$ channels and $T$ time-samples. The epoch is chopped into **$N$ non-overlapping patches of width $p$ samples**; the configuration reported is **$p=8$ at 256 Hz $\rightarrow N=25$**. Each patch is flattened and linearly projected into a **$d$-dimensional patch embedding space**, with **$d=64$**, yielding
$$
E=[e_1,\dots,e_N]^\top\in\mathbb{R}^{N\times d}.
$$
For **$K$ fixed prototypes**, prototype embeddings are formed as
$$
P=[p_1,\dots,p_K]^\top\in\mathbb{R}^{K\times d}.
$$
After a shared LayerNorm, ERP-XTTN computes multi-head query and key projections, with **$H=4$** heads and **$d_h=d/H=16$**:
$$
Q^{(h)} = E\,W_Q^{(h)}, \qquad K^{(h)} = P\,W_K^{(h)}.
$$
The scaled dot-product attention weights are then
$$
A^{(h)} = \mathrm{softmax}\!\Bigl(\tfrac{Q^{(h)}(K^{(h)})^\top}{\sqrt{d_h}}\Bigr)\in\mathbb{R}^{N\times K},
$$
where the softmax is taken row-wise over prototypes, and the final routing matrix is the head average
$$
A=\tfrac1H\sum_{h=1}^H A^{(h)}\in\mathbb{R}^{N\times K}.
$$
The interpretation given is that $A_{n,k}$ is the fraction of attention that patch $n$ routes to prototype $k$ [2606.02939].

The defining restriction is the removal of the value pathway: **No separate “value” projection is used: $A$ itself is the sole input to the classifier head.** After averaging heads, the model flattens $A$ to a vector $a\in\mathbb{R}^{NK}$ and applies one linear layer,
$$
s = w^\top\,\mathrm{vec}(A)+b,\qquad \hat p=\sigma(s)=\frac{1}{1+e^{-s}}.
$$
Training uses **binary cross-entropy-with-logits loss on $s$**, and inference thresholds **$\hat p>0.5$** for a class label. The implementation details specify **Self-attention: one layer, pre-norm, $H=4$ heads, dropout=0.3 on weights, residual connection**; **Cross-attention: separate LayerNorm for queries/keys, $W_Q,W_K$ projections, no values**; and **Classifier: flatten $A\in\mathbb{R}^{25\times K}\rightarrow$ vector of length $25K\rightarrow \mathrm{Linear}(25K\rightarrow 1)$**. Because the classifier has no alternative access path to the input, the paper characterizes attention faithfulness as structural rather than post-hoc.

## 3. Prototype construction and deployment-compatible preprocessing

ERP-XTTN derives prototypes automatically from the **grand-average difference wave** within each **leave-one-subject-out (LOSO)** training fold. For a single **detection channel** $c_0$, the fold produces two class means,
$\mu^+(t)$ for positive trials and $\mu^-(t)$ for negative trials, and defines the difference wave as
$$
\Delta(t)=\mu^+(t)-\mu^-(t).
$$
This signal is **smoothed by a Gaussian ($\sigma=2$ samples)**, after which the algorithm detects **up to $K_{\max}=4$ extrema of both polarities after $t_0=50$ ms**, requiring **prominence $\ge 0.02$** and **within-polarity separation $\ge 80$ ms**. For each selected peak at time $t_p$, the method identifies the previous and next zero crossings, clamps the window width to **$[40\ \mathrm{ms}, 200\ \mathrm{ms}]$** about $t_p$, and stores **the full-montage $\Delta$ over $C$ channels restricted to the resulting window** as a prototype tensor $P_k$ [2606.02939].

Prototype embedding uses **exactly the same patch-splitting + linear projection as on $X$**, followed by mean pooling over the prototype’s own patches to yield the final $p_k\in\mathbb{R}^d$. The implementation note states that **each prototype $k$ is stored as a $C\times T_k$ tensor (raw $\Delta$-wave segments); embedding is computed on the fly with the same projection as for input patches**.

The preprocessing pipeline is explicitly **causal**. Every dataset is processed by a **4th-order Butterworth bandpass at 1–10 Hz**, implemented as a **causal IIR filter** such as `lfilter(b,a,x)`, with difference equation
$$
y[n]=\sum_{i=0}^4 b_i\,x[n-i]-\sum_{i=1}^4 a_i\,y[n-i],
$$
and **no backward pass**, so $y[n]$ depends only on past and present samples. The data are then **downsampled to 256 Hz (where needed)** and epoched from **0–800 ms post-trigger**. The paper frames these choices as **deployment-compatible conditions** and evaluates them under **zero calibration** and LOSO generalization. A common misconception would be to read “causal” here as causal discovery; in this formulation, the primary concrete use of the term is causal signal processing and temporally valid deployment.

## 4. Latent prototype-distance CPAC for fraud detection

In the fraud-detection formulation, CPAC is defined around four components: an **Encoder $E_p$**, a **prototype bank $\{p_0,p_1\}$**, an **Attention module $\mathrm{Att}(\cdot)$**, and a **Distance Aggregator + Classifier head**. The encoder produces a **$d$-dimensional latent vector $z$ (or $\mu$)**; the prototype bank consists of **two learnable vectors in $\mathbb{R}^d$ that serve as class “anchors” (non-fraud vs. fraud)**; and the attention module is **a small MLP that ingests $z$ and outputs a per-dimension gating vector $w\in(0,1)^d$** [2507.14706].

The mathematical specification is explicit. The attention mechanism is
$$
h=\mathrm{ReLU}(W_1z+b_1),\qquad w=\sigma(W_2h+b_2),
$$
and, for each class $k\in\{0,1\}$, the weighted distance is
$$
d_k(z)=\alpha\cdot\sum_{j=1}^d w_j\cdot (z_j-p_{kj})^2,
$$
where **$\alpha>0$ is a learned scalar controlling sensitivity**. These distances are converted to soft assignments by
$$
a_k(z)=\exp(-d_k(z)),\qquad a_k(z)\leftarrow \frac{a_k(z)}{a_0(z)+a_1(z)}.
$$
The final fraud probability is
$$
\hat y = a_1(z)=\mathrm{softmax}_k(-d_k(z))\big|_{k=1},
$$
or equivalently, with logits $\ell(z)=[-d_0(z),-d_1(z)]^\top$,
$$
\hat y=\mathrm{softmax}(\ell(z))_1.
$$

The paper states that **causal reasoning is invoked by viewing each attention weight $w_j$ as an estimate of the “causal effect” of latent feature $j$ on the final decision, while prototypes provide counterfactual anchors**. In this setting, therefore, “causal” is not the same notion as in ERP-XTTN’s causal IIR preprocessing. Rather, the term is used to organize interpretation around feature-level influence and counterfactual prototype comparison. This suggests that the causal semantics of CPAC are domain-specific and should not be treated as a single formal causal framework.

## 5. Training objectives, latent shaping, and empirical behavior

The ERP-XTTN training regimen specifies **AdamW with lr=$1\times10^{-3}$, weight\_decay=$1\times10^{-4}$**, **batch size 128**, **class-weighted BCE-with-logits; pos\_weight=(\#negative)/(\#positive)**, **linear warmup for 5 epochs $\rightarrow$ cosine annealing for 100 epochs $\rightarrow$ constant at $1\times10^{-5}$**, **gradient clipping $\lVert g\rVert_2\le 1.0$**, **dropout 0.3 after patch embedding and on attention weights**, training-only augmentation through **temporal jitter in $[-10,+10]$ samples** and **additive Gaussian noise $\sigma=0.1$**, and **early stopping** with a **15%** holdout stratified by **subject$\times$class**, monitored by **LOSO-validation AUROC**, with **patience=15 epochs** and **max\_epochs=250**. The final model is **retrained from scratch on train+val for the chosen epoch count** [2606.02939].

The fraud formulation introduces a broader objective because CPAC is attached to a VAE-GAN encoder. The stated losses are
$$
L_{\mathrm{rec}}=\mathbb{E}_{x\sim p_{\mathrm{data}}}\big[\lVert x-D(G(x))\rVert_2^2\big],
$$
$$
L_{\mathrm{KL}}=\beta\cdot \mathrm{KL}[q(z|x)\|N(0,I)],
$$
$$
L_{\mathrm{GAN}}=-\mathbb{E}_x[\log D(x)]-\mathbb{E}_{\hat x}[\log(1-D(\hat x))],
$$
$$
L_{\mathrm{adv}}=-\mathbb{E}_{\hat x}[\log D(\hat x)].
$$
For CPAC itself, the paper uses **Binary Cross-Entropy or Focal Loss**:
$$
L_{\mathrm{FL}}(y,\hat y)=-\alpha_{\mathrm{FL}}(1-\hat y)^\gamma y\log \hat y-(1-\alpha_{\mathrm{FL}})\hat y^\gamma(1-y)\log(1-\hat y),
$$
together with an **attention-scale penalty**
$$
L_{\mathrm{scale}}=\lambda_{\mathrm{scale}}\cdot \lVert \alpha\rVert^2
$$
and a **prototype-anchoring penalty**
$$
L_{\mathrm{anchor}}=\lambda_{\mathrm{anchor}}\cdot [\lVert p_0-\bar\mu_0\rVert^2+\lVert p_1-\bar\mu_1\rVert^2].
$$
The combined CPAC head loss is
$$
L_{\mathrm{proto}}=L_{\mathrm{scale}}+L_{\mathrm{anchor}},\qquad L_{\mathrm{CPAC}}=L_{\mathrm{cls}}+L_{\mathrm{proto}}.
$$
When attached to the VAE-GAN encoder, training alternates between minimizing **$L_{\mathrm{rec}}+L_{\mathrm{KL}}+L_{\mathrm{adv}}$** and minimizing **$L_{\mathrm{CPAC}}$**, or, in a single-objective view,
$$
L_{\mathrm{total}}=L_{\mathrm{rec}}+L_{\mathrm{KL}}+\lambda_{\mathrm{adv}}L_{\mathrm{adv}}+\lambda_{\mathrm{cls}}L_{\mathrm{cls}}+\lambda_{\mathrm{proto}}L_{\mathrm{proto}}.
$$

The latent-space interpretation is stated directly: **The BCE/Focal loss through CPAC backpropagates into the encoder, explicitly pulling latent vectors $z$ toward their class prototype $p_k$**; **the anchoring term $L_{\mathrm{anchor}}$ further encourages each prototype $p_k$ to track the empirical centroid of class-$k$ latents**; and **as a result, $z$’s form two tight, well-separated clusters, each around $p_0$ or $p_1$**. Cluster separation is described as quantifiable by **the Silhouette Score or Davies–Bouldin Index**, while the paper visualizes overlap using **PCA and 3D plots**.

## 6. Performance, interpretability, limitations, and common misconceptions

In ERP-XTTN, CPAC is benchmarked against **EEGNet** and **xDAWN+Riemannian-geometry (xDAWN+RG)** under a **targeted 3-channel montage** and a **full available montage**. The reported **mean LOSO AUROC across 9 datasets** is:

| Setting | xDAWN+RG | EEGNet | ERP-XTTN |
|---|---:|---:|---:|
| 3-channel | .715 | .731 | .716 |
| full montage | .777 | .794 | .766 |

The corresponding **interpretability cost** is reported as **$\Delta=(\text{best baseline}-\text{ERP-XTTN})=.018$** at **3 channels** and **$\Delta=.034$** at **full montage**. The paper states that the mean gap arises from **two largely distinct sources: a temporal-flexibility cost relative to EEGNet and a spatial-exploitation cost relative to xDAWN+RG, the latter driven by signal-to-noise ratio at full montage**, while also noting that **on some tasks (e.g. ERN, P300, HRI-ErrP at 3ch) ERP-XTTN actually slightly outperformed xDAWN+RG** [2606.02939].

ERP-XTTN’s interpretability is quantified using two trial-level routing metrics: **Attention entropy**
$$
H_{\mathrm{attn}}=\frac{-1}{\log(NK)}\sum_{n,k}A_{n,k}\log A_{n,k},
$$
averaged over trials, and **Routing discriminability $R_{\mathrm{disc}}$**, defined as the **mean cosine distance between class-mean attention vectors**. The paper reports that the average interpretability cost $\Delta$ against EEGNet correlates **(Spearman $\rho\approx 0.5$)** with **$H_{\mathrm{attn}}$, $R_{\mathrm{disc}}$, and an SNR proxy (grand-average $\Delta$-wave amplitude / trial-wise SD)**, suggesting that **the fixed-prototype constraint costs most when the temporal signal is strong and narrowly peaked**. Against xDAWN+RG, $\Delta$ correlates most with **SNR in the full-montage case**, indicating that **spatial filtering outperforms static prototypes when there is rich spatial information**. The grand-average waveform analysis further reports that **the TP–FP correlation (mean Pearson $r\approx 0.6$) exceeds the TP–TN correlation** on most datasets, which the paper interprets as evidence that **misclassifications occur on trials whose morphology genuinely resembles the target prototype set**.

In the fraud setting, the reported standalone CPAC metrics on raw data are **Precision=87.20%, Recall=73.65%, F1=79.85%, AUC=95.80%**. When used in **VAE-GAN+CPAC as an oversampler**, the paper states that **the best setting (75 pre-SMOTE + VAE-GAN+CPAC) yields: Precision≈96.4%, Recall≈90.2%, F1≈93.1%, AUC≈96.9%**, and the abstract summarizes this as **an F1-score of 93.14\% percent and recall of 90.18\%, along with improved latent cluster separation**. The ablation study reports that removing the **CPAC head** causes **latent clusters collapse, heavy overlap, poor downstream accuracy**; removing **attention ($w\equiv 1$)** degrades separation to **a near-linear manifold**; removing **prototypes** causes clusters to **overlap badly**; omitting **scale/anchor penalties** leaves clusters separated but with **fuzzier boundaries**; and swapping **BCE$\rightarrow$Focal Loss** gives **sharper latent separation visually but slight drop in raw F1 on hold-out** [2507.14706].

Several misconceptions can be addressed directly from these results. First, CPAC is not synonymous with lossless interpretability: the ERP evidence reports a measurable **interpretability cost**. Second, the causal label is not uniform across domains: ERP-XTTN grounds it in **causal IIR preprocessing** and deployment compatibility, whereas the fraud paper grounds it in **feature-level “causal effects”** and **counterfactual anchors**. Third, prototype guidance does not eliminate the need for domain structure: the ERP paper identifies limits associated with **temporal flexibility** and **spatial exploitation**, while the fraud paper explicitly lists **one prototype per class**, **hyperparameter sensitivity**, and possible extensions to **multiple prototypes**, **contrastive or manifold regularization**, **denoising autoencoders**, and application to **network intrusion** or **medical outliers**. Taken together, these studies position CPAC as an interpretable prototype-attention framework whose practical behavior depends strongly on how prototypes are defined, how attention is constrained, and which notion of causality is being invoked.

Source: https://www.emergentmind.com/topics/causal-prototype-attention-classifier-cpac