CTNet: Cross-modal Transition-Guided Network

• The paper introduces CTNet, a framework that models feature transitions among RGB, IR, and depth modalities to distinguish live faces from spoof attacks.
• CTNet combines modality-specific feature extractors with auxiliary branches to predict missing modality information, ensuring robust performance under sensor absence.
• Empirical results reveal that adaptive fusion of prototype and transition scores significantly enhances multi-modal face anti-spoofing and out-of-distribution detection.

Searching arXiv for the cited papers to ground the article. Cross-modal Transition-guided Network (CTNet) is a multimodal representation-learning framework introduced for multi-modal face anti-spoofing (FAS) that models not only modality-specific features but also the transitions between modalities, especially among RGB, infrared (IR), and depth representations. Its central premise is that live samples exhibit more consistent cross-modal feature transitions than spoof samples, and that this transition structure can be exploited both for generalized liveness modeling and for out-of-distribution (OOD) attack detection. In the formal usage documented in "Multi-Modal Face Anti-Spoofing via Cross-Modal Feature Transitions" (Chong et al., 8 Jul 2025), CTNet combines modality-specific feature extractors, live-prototype-based transition alignment, spoof-transition inconsistency learning, and auxiliary RGB-driven completion of missing IR and depth features. A broader, non-nominally identical interpretation also appears in MRI-to-CT synthesis, where a multi-task Transformer U-Net framework decomposes translation into skull segmentation, Hounsfield Unit (HU) prediction, and sequential reconstruction, thereby functioning as a transition-guided cross-modal synthesis model rather than a direct pixel-to-pixel regressor (Xin et al., 2023).

1. Definition and problem setting

CTNet was proposed to address two difficulties in multi-modal FAS: cross-modal distribution discrepancies and missing modalities at inference time (Chong et al., 8 Jul 2025). In this setting, RGB, IR, and depth are treated as distinct domains rather than interchangeable channels, and the model must determine whether a face is live or spoof under both intra-domain and cross-domain conditions. The motivating observation is twofold: within a single modality, live faces are more similar to one another than spoof faces are, and across modalities, feature transitions are more consistent for the live class than for live-versus-spoof comparisons (Chong et al., 8 Jul 2025).

This formulation departs from conventional multimodal fusion approaches by making the transition itself a primary learning target. Rather than only aggregating RGB, IR, and depth cues into a joint embedding, CTNet explicitly models directional relations such as RGB $\rightarrow$ IR, RGB $\rightarrow$ D, and IR $\rightarrow$ D. The resulting framework is simultaneously a multimodal alignment model, a transition-based OOD detector, and a modality completion model when deployed under sensor absence (Chong et al., 8 Jul 2025).

A broader use of the term can be inferred from related cross-modal medical imaging work. In "Enhancing CT Image synthesis from multi-modal MRI data based on a multi-task neural network framework" (Xin et al., 2023), the phrase “Cross-modal Transition-guided Network” is not used as a formal module name. However, the described MRI-to-CT synthesis method functions as a CT synthesis network guided by skull segmentation and HU prediction, which suggests a CTNet-style interpretation in which anatomically meaningful intermediate targets constrain the modality transition (Xin et al., 2023).

2. Architectural organization and feature flow

In its formal FAS instantiation, CTNet comprises three modality-specific branches, one each for RGB, IR, and depth, together with auxiliary completion branches that predict IR-like and depth-like features from RGB (Chong et al., 8 Jul 2025). The feature extractors are denoted $F_{\text{RGB}}$, $F_{\text{IR}}$, and $F_{\text{D}}$, and each produces a modality-specific latent feature. Each modality also has its own classifier, $C_{\text{RGB}}$, $C_{\text{IR}}$, and $C_{\text{D}}$. The backbone is ResNet-34, and the implementation uses AdamW with learning rate $5e\text{-}4$ for 50 epochs (Chong et al., 8 Jul 2025).

The auxiliary branches, $\rightarrow$0 and $\rightarrow$1, are designed specifically for missing-modal settings. They do not reconstruct raw sensor observations; rather, they predict feature-level modality semantics from RGB. At inference time, if IR or depth is unavailable, the corresponding predicted features replace the absent real features, and the same scoring pipeline is applied (Chong et al., 8 Jul 2025). This design makes CTNet flexible under both fixed-modal and missing-modal protocols.

The contrast with the MRI-to-CT multi-task framework is instructive. There, the core engine is an enhanced 3D Transformer U-Net operating on 3D patches, with positional encoding, multi-head self-attention, LayerNorm, dropout, convolutional feature fusion at the encoder stage, skip connections, MLP or linear projection in the transformer blocks, and 3D residual convolution blocks with upsampling in the decoder (Xin et al., 2023). The framework contains two coordinated pipelines: a segmentation pipeline for skull mask prediction and a pixel-value prediction pipeline for CT intensities or HU values. Features from different MRI channels are combined by convolution in the encoder, and patch outputs are sequentially restored into full CT volumes with overlapping voxels averaged (Xin et al., 2023). This suggests a structurally analogous transition-guided organization, though in an overview rather than FAS context.

3. Transition modeling, prototypes, and objective functions

The central object in CTNet is the cross-modal transition vector

$\rightarrow$2

The paper reports that, empirically, these transition vectors are highly consistent across live samples, whereas spoof transitions are inconsistent relative to live prototypes (Chong et al., 8 Jul 2025). This observation is operationalized by maintaining a live prototype for each modality using exponential moving average (EMA), where the current live-batch mean is

$\rightarrow$3

and the prototype update is

$\rightarrow$4

A cosine-similarity-based criterion is also given as a conceptual characterization of live-feature proximity to these modality prototypes (Chong et al., 8 Jul 2025).

CTNet divides transition learning into two complementary regimes. For live samples, it uses a Consistent Transition Loss,

$\rightarrow$5

where

$\rightarrow$6

This aligns live-sample transitions with live-prototype transitions and thereby constructs a generalized latent space (Chong et al., 8 Jul 2025).

For spoof samples, CTNet explicitly seeks inconsistency relative to live transitions. First, a modality-discriminative loss pushes spoof features away from live modality centroids:

$\rightarrow$7

Second, an Inconsistent Transition Loss pushes spoof transitions away from live prototype transitions:

$\rightarrow$8

Together, these losses encode the paper’s claim that live samples should be transition-consistent while spoof samples should be transition-inconsistent relative to live geometry (Chong et al., 8 Jul 2025).

The full training objective additionally includes a modality-specific contrastive loss $\rightarrow$9, three modality-wise binary cross-entropy losses, and a complementary feature loss $\rightarrow$0 for RGB-to-IR and RGB-to-depth auxiliary prediction. The total loss is

$\rightarrow$1

with $\rightarrow$2 and $\rightarrow$3 (Chong et al., 8 Jul 2025). In fixed-modal testing, $\rightarrow$4 is not included, whereas in missing-modal testing it is included and the auxiliary branches are trained (Chong et al., 8 Jul 2025).

4. Inference, OOD scoring, and missing-modality handling

At test time, CTNet uses an OOD score that fuses a prototype-distance term with a transition-consistency term:

$\rightarrow$5

The prototype-distance score is

$\rightarrow$6

and the transition score is

$\rightarrow$7

The decision threshold is selected using the Youden Index (Chong et al., 8 Jul 2025).

A notable result of the ablation on $\rightarrow$8 is that $\rightarrow$9 alone outperforms $F_{\text{RGB}}$0 alone, while the best performance occurs around $F_{\text{RGB}}$1; adaptive fusion is therefore better than either score in isolation (Chong et al., 8 Jul 2025). This finding is significant because it indicates that transition information is not merely an auxiliary regularizer but a principal discriminative signal.

Missing modalities are handled by substituting $F_{\text{RGB}}$2 for IR and $F_{\text{RGB}}$3 for depth when those inputs are absent (Chong et al., 8 Jul 2025). The paper’s complementary feature loss,

$F_{\text{RGB}}$4

trains these auxiliary branches to align predicted IR-like and depth-like features with the true features. This suggests that CTNet treats flexible modality availability as a first-class deployment condition rather than a post hoc robustness test.

5. Experimental protocols and empirical performance

CTNet is evaluated on three multi-modal FAS datasets: WMCA, CASIA-SURF, and CASIA-SURF CeFA (Chong et al., 8 Jul 2025). WMCA includes 7 attack types and both 2D and 3D attacks; CASIA-SURF contains 21,000 videos and 1,000 subjects with print attacks and partial-face attacks; CASIA-SURF CeFA includes 1,607 subjects, multiple ethnicities, and supports unseen-attack and cross-ethnicity protocols (Chong et al., 8 Jul 2025). Evaluation covers fixed-modal and missing-modal scenarios, as well as intra-domain and cross-domain testing. The missing-modal test settings are P1: RGB, P2: RGB + D, P3: RGB + IR, and P4: RGB + D + IR (Chong et al., 8 Jul 2025). Reported metrics are APCER, BPCER, ACER, and AUC, with

$F_{\text{RGB}}$5

In fixed-modal intra-domain evaluation, CTNet achieves strong results on WMCA, including 0.98 ACER on Seen, 2.59 ACER on Flexiblemask, and 0.00 ACER on Replay, Fakehead, and Prints (Chong et al., 8 Jul 2025). On CASIA-SURF it attains APCER 0.39, BPCER 0.39, and ACER 0.39, slightly better than FM-CLIP’s 0.43 ACER as reported in the paper (Chong et al., 8 Jul 2025). In missing-modal intra-domain results on CASIA-SURF and CeFA, the reported values are 5.79 for P1 RGB, 1.33 for P2 RGB + D, 4.71 for P3 RGB + IR, and 1.17 for P4 RGB + D + IR (Chong et al., 8 Jul 2025).

Cross-domain results are more demanding and are presented as a central test of robustness. Representative fixed-modal results include 5.54 for CASIA-SURF CeFA $F_{\text{RGB}}$6 CASIA-SURF, $F_{\text{RGB}}$7 for CASIA-SURF $F_{\text{RGB}}$8 CeFA, 5.00 for WMCA $F_{\text{RGB}}$9 CASIA-SURF, $F_{\text{IR}}$0 for WMCA $F_{\text{IR}}$1 CeFA, 24.31 for CeFA $F_{\text{IR}}$2 WMCA, and 14.96 for CASIA-SURF $F_{\text{IR}}$3 WMCA (Chong et al., 8 Jul 2025). On the harder missing-modal cross-domain setting $F_{\text{IR}}$4, CTNet reports 22.56 for P1 RGB, 17.16 for P2 RGB + D, 18.76 for P3 RGB + IR, and 16.51 for P4 RGB + D + IR (Chong et al., 8 Jul 2025). The paper interprets these outcomes as evidence that transition consistency, transition inconsistency, and auxiliary modality completion work synergistically.

6. Ablations, interpretation, and limitations

The ablation study reported for the missing-modal cross-domain protocol 1 shows a monotonic performance progression as loss terms are added: $F_{\text{IR}}$5 only yields 35.12 ACER, adding $F_{\text{IR}}$6 gives 32.11, then $F_{\text{IR}}$7 gives 30.44, $F_{\text{IR}}$8 gives 27.70, $F_{\text{IR}}$9 gives 25.04, and $F_{\text{D}}$0 gives 22.56 (Chong et al., 8 Jul 2025). This indicates that completion improves missing-modal operation, modality-specific alignment improves representation structure, consistent transition learning improves generalization, and inconsistent transition learning produces the final gain in OOD robustness.

A second ablation compares transition pairs. RGB $F_{\text{D}}$1 IR alone is weaker, whereas RGB $F_{\text{D}}$2 D and IR $F_{\text{D}}$3 D are more discriminative; using all transitions together gives the best performance (Chong et al., 8 Jul 2025). This is consistent with the paper’s broader claim that CTNet benefits from modeling multiple transition directions rather than relying on a single inter-modal relation. The t-SNE visualizations are reported to show distinct modality-specific clusters and alignment between IR-like or depth-like predicted features and real IR or depth features (Chong et al., 8 Jul 2025).

Several limitations are described as implicit rather than formally isolated in a dedicated section. The method depends on live prototype quality, so noisy live batches or prototype drift could affect performance. It uses predefined pairwise transitions among RGB, IR, and depth, which is effective for three modalities but may require a more general transition graph as the number of modalities grows. The auxiliary completion branches predict feature-level surrogates rather than raw modality data. Finally, the hardest cross-domain WMCA-targeted protocols remain challenging even though CTNet improves over prior methods (Chong et al., 8 Jul 2025). A plausible implication is that the transition-based formulation improves robustness without eliminating the broader domain-generalization problem in FAS.

The MRI-to-CT work provides a conceptually adjacent perspective on transition guidance. Its multi-task framework argues that cross-modal synthesis benefits from decomposing the task into skull segmentation, HU value prediction, and 3D sequential reconstruction rather than treating it as pure pixel-to-pixel regression (Xin et al., 2023). The loss combines Dice loss, binary cross-entropy, and local ROI-restricted MSE, patch extraction uses size $F_{\text{D}}$4 from standardized volumes of $F_{\text{D}}$5, and reconstruction averages overlapping voxels (Xin et al., 2023). This suggests that, beyond face anti-spoofing, CTNet-like ideas can also denote cross-modal systems in which anatomically or semantically meaningful intermediate structures guide the transition from one modality to another.