Adaptive Bidirectional Pyramid Difference Convolution

• The paper introduces ABPDC, combining standard and difference convolution via an adaptive gate to enhance multimodal CT/MRI feature fusion.
• ABPDC is a 3D convolution module designed to address modality-dependent biases and spatial misalignments by using a z-axis pyramid for thick-slice data.
• Empirical results show that integrating ABPDC with FPRAN improves accuracy, F1-score, and AUC across various CT/MRI modality pairs in postoperative neck pain prediction.

Adaptive Bidirectional Pyramid Difference Convolution (ABPDC) is a 3D convolutional module introduced for multimodal cross-view CT/MRI integration in postoperative neck-pain prediction for cervical spondylosis patients. It was proposed in a framework that uses paired preoperative imaging from axial CT bone-window, axial CT soft-tissue-window, axial MRI T2-weighted, and sagittal MRI T1/T2-weighted acquisitions, where orthogonal acquisition planes, modality-specific grayscale distributions, distinct texture signatures, and anisotropic voxel spacing jointly make direct feature fusion unstable. ABPDC addresses these conditions by combining standard convolution with difference convolution through a spatially adaptive gate, while constraining the difference branch with a z-axis pyramid designed for thick-slice 3D data. In the full model, ABPDC is deployed in both modality-specific backbones and is coupled with a Feature Pyramid Registration Auxiliary Network (FPRAN) for coarse-to-fine deformable alignment (Shan et al., 2 Sep 2025).

1. Problem setting and design rationale

ABPDC was developed for a clinical task with a binary endpoint: predicting whether a patient with cervical spondylosis will experience postoperative neck pain relief following anterior cervical surgery. The underlying dataset, MMCSD, provides paired preoperative multimodal and multiview imaging, specifically axial CT bone-window, axial CT soft-tissue-window, axial MRI T2-weighted, and sagittal MRI T1/T2-weighted studies. The central representational problem is cross-view multimodal fusion: CT is typically acquired in the axial plane, whereas MRI is often acquired in the sagittal plane, so the data are not only multimodal but also orthogonally oriented.

Two difficulties motivate the operator design. First, imaging differences introduce modality-dependent biases in contrast, noise, and texture signatures, which degrade direct correspondence and confuse shared semantic representations. Second, spatial discrepancies produced by axial-versus-sagittal acquisition, inconsistent anatomical detail, and anisotropic voxel spacing hinder voxel-wise alignment and destabilize multimodal learning. The paper therefore couples a feature-fusion operator, ABPDC, with an alignment module, FPRAN, assigning feature robustness and deformable registration to distinct parts of the pipeline.

The use of difference convolution is explicitly motivated by its relation to LBP and Central Difference Convolution (CDC). Central differencing emphasizes local gradient-like structures by subtracting the central pixel from its neighbors, thereby increasing texture sensitivity and reducing reliance on absolute intensity. Its grayscale invariance under additive intensity shifts is stated directly: for $T \to T + c$, central differencing yields $(T-T_c)\to(T+c-(T_c+c))=T-T_c$, cancelling global shifts. In the CT/MRI setting, this is critical because the modalities differ in intensity scales. ABPDC extends this principle to thick-slice 3D volumes, adds a pyramid strategy that prefers richer $x$-$y$ textures over $z$, and replaces the fixed CDC mixing coefficient with a learned, spatially adaptive weight (Shan et al., 2 Sep 2025).

2. Formal definition of the operator

At pyramid level $i \in \{1,2,3,4\}$, the CT and MRI backbones output $f_{\mathrm{CT}}^i$ and $f_{\mathrm{MRI}}^i$. Within each backbone, standard convolutional layers are replaced by ABPDC blocks. The module consumes a local 3D feature patch $T$, centered at $T_c$, and produces an enhanced feature map by mixing a standard convolution over a full kernel region $(T-T_c)\to(T+c-(T_c+c))=T-T_c$0 with a difference convolution over a pyramidally shrunk subset $(T-T_c)\to(T+c-(T_c+c))=T-T_c$1.

The z-axis pyramid is defined for odd kernel size $(T-T_c)\to(T+c-(T_c+c))=T-T_c$2 as

$(T-T_c)\to(T+c-(T_c+c))=T-T_c$3

This receptive field shrinks its allowable in-plane offsets as $(T-T_c)\to(T+c-(T_c+c))=T-T_c$4 increases. The stated motivation is that, due to anisotropic voxel spacing, medical volumes exhibit richer textures in the $(T-T_c)\to(T+c-(T_c+c))=T-T_c$5-$(T-T_c)\to(T+c-(T_c+c))=T-T_c$6 plane than along the $(T-T_c)\to(T+c-(T_c+c))=T-T_c$7-axis; the pyramid therefore de-emphasizes through-plane accumulation.

The adaptive gate is computed from local texture strength $(T-T_c)\to(T+c-(T_c+c))=T-T_c$8, estimated with a 3D Sobel operator, followed by centered average pooling, a learnable $(T-T_c)\to(T+c-(T_c+c))=T-T_c$9 convolution, and sigmoid normalization:

$x$0

The operator output is then

$x$1

where $x$2 denotes kernel weights, $x$3 the full convolutional footprint, and $x$4 the pyramid-shrunk subset used for the difference term.

This formulation makes the two branches complementary. The first branch performs additive aggregation over the full kernel support; the second performs subtractive, gradient-focused aggregation over a pyramidally constrained support. Because the difference term is invariant to global additive shifts, large $x$5 in low-contrast or blurred regions suppresses absolute intensity and highlights textures and edges. The paper also states that the module does not require a globally tuned CDC hyperparameter $x$6; instead, $x$7 is learned and spatially adaptive (Shan et al., 2 Sep 2025).

3. Bidirectionality, anisotropy awareness, and adaptive behavior

In ABPDC, “bidirectional” does not denote explicit CT$x$8MRI and MRI$x$9CT equations inside the convolution itself. The paper is explicit that bidirectionality is realized within each block as the joint use of two complementary operations: standard convolution, described as additive aggregation, and difference convolution, described as central differencing. The adaptive gate allows the local computation to flow toward either operator depending on texture strength, rather than enforcing a single operator globally.

The adaptive mechanism is coupled to a 3D Sobel-derived texture estimate $y$0. Because $y$1 has the form $y$2, the gate decreases as $y$3 increases. Sharper regions are therefore assigned higher weight to standard convolution, while smoother or blurred regions are assigned higher weight to the difference term. This is presented as a response to the observation that texture strength varies across anatomy and modalities. A plausible implication is that the module behaves as a local operator selector whose bias depends on regional image sharpness, although the paper formulates this in terms of adaptive weighting rather than discrete selection.

The anisotropy-aware pyramid is equally central. The difference branch does not use the full $y$4 neighborhood; it uses the z-contracted subset $y$5, thereby prioritizing in-plane textural evidence over through-plane context. This design is specifically justified for thick-slice 3D data with anisotropic resolution. In the paper’s framing, ABPDC therefore combines three distinct principles: grayscale-invariant differencing, anisotropy-aware receptive-field shaping, and spatially adaptive mixing between additive and subtractive local evidence (Shan et al., 2 Sep 2025).

4. Integration with FPRAN and the full multimodal architecture

The overall network contains two modality-specific backbones, one for CT and one for MRI, each producing a four-level feature pyramid:

$y$6

ABPDC replaces standard convolutions in both backbones. The resulting feature pyramids are then aligned by the Feature Pyramid Registration Auxiliary Network (FPRAN), which performs coarse-to-fine deformable alignment across four pyramid levels via Spatial Alignment Modules (SAMs). At the coarsest scale,

$y$7

where $y$8 denotes STN-based warping.

At level 4, the backbone features are first enhanced by 3D convolution to obtain $y$9 and $z$0. CT features are then transformed from axial to sagittal by a differentiable affine transform,

$z$1

with $z$2 a base rotation matrix and $z$3 a 3-layer convolutional refinement network. Cross-attention produces the initial displacement field,

$z$4

which is propagated downward to refine $z$5, $z$6, and $z$7. The final warped CT image is

$z$8

The classifier consumes concatenated top-level features and produces a prediction vector $z$9 with $i \in \{1,2,3,4\}$0 for pain versus non-pain. The paper characterizes the interaction between ABPDC and FPRAN as complementary: ABPDC mitigates inter-modal imaging differences by making features more texture-centric and less intensity-dependent, whereas FPRAN mitigates spatial discrepancies by pyramidal registration. A common misconception is therefore ruled out by the paper itself: cross-modal influence is not implemented as explicit cross-modal ABPDC equations, but by symmetric ABPDC deployment in both backbones together with FPRAN-based alignment (Shan et al., 2 Sep 2025).

5. Optimization, implementation, and reproducibility

The training objective is the unweighted sum of three losses:

$i \in \{1,2,3,4\}$1

The classification term is cross-entropy,

$i \in \{1,2,3,4\}$2

where $i \in \{1,2,3,4\}$3 is the one-hot label and $i \in \{1,2,3,4\}$4 the predicted probability vector. The similarity term is a normalized cross-correlation loss between MRI and the warped CT image,

$i \in \{1,2,3,4\}$5

with $i \in \{1,2,3,4\}$6 the voxel domain and $i \in \{1,2,3,4\}$7 the number of voxels. The deformation regularizer is

$i \in \{1,2,3,4\}$8

The paper does not specify weighting hyperparameters such as $i \in \{1,2,3,4\}$9 or $f_{\mathrm{CT}}^i$0.

Reported implementation details include PyTorch and MONAI as frameworks, AdamW as optimizer, batch size 16, initial learning rate $f_{\mathrm{CT}}^i$1, Step Decay (0.9) over 100 epochs, and training on two NVIDIA RTX 3090 GPUs. Standard convolutions in the backbones are replaced by ABPDC blocks, and FPRAN consists of four SAMs, one per pyramid level. The paper does not enumerate the specific kernel size $f_{\mathrm{CT}}^i$2, channel counts, or normalization and activation details. It also does not report analytical complexity, memory footprint, or runtime comparisons.

The only complexity statement given is comparative rather than absolute: standard 3D convolution over a $f_{\mathrm{CT}}^i$3 kernel is $f_{\mathrm{CT}}^i$4, whereas ABPDC performs one full convolution over $f_{\mathrm{CT}}^i$5 and one difference convolution over the pyramidally shrunk set $f_{\mathrm{CT}}^i$6, which is smaller than $f_{\mathrm{CT}}^i$7 because of the z-dependent constraint. Code availability is not specified, and no reproducibility checklist is provided, although the paper supplies model diagrams, explicit loss definitions, training settings, and quantitative results tables (Shan et al., 2 Sep 2025).

6. Empirical behavior on MMCSD and interpretive significance

MMCSD contains 250 patients who underwent anterior cervical surgery. Labels are derived from postoperative Visual Analog Scale scores, and patients are divided into pain versus non-pain groups. The original train/test split of the dataset was adopted. Evaluation uses Accuracy (ACC), F1-score, and area under the ROC curve (AUC).

The empirical comparison among standard convolution, CDC with $f_{\mathrm{CT}}^i$8, and ABPDC shows a clear overall pattern. CDC and ABPDC generally outperform standard convolution, and ABPDC is consistently superior to CDC. ABPDC achieves the best performance in modality pairs T2sag+T2tra, T2sag+CT tissue, and T2tra+T1sag. In T2sag+T1sag, T2tra+CT tissue, and T1sag+CT tissue, CDC underperforms standard convolution, but ABPDC still improves over standard convolution. For modality pairs involving CT bone, both CDC and ABPDC initially fall below standard convolution; introducing the adaptive differential weighting algorithm into CDC and ABPDC improves accuracy in most pairs and yields the largest gains on CT bone, with the adaptively selected $f_{\mathrm{CT}}^i$9 reported as close to optimal.

The ablation study isolates ABPDC (A), FPRAN (F), and their combination across six modality pairs. The combined configuration A+F yields the following results:

Modality pair	A+F results	Improvement pattern
T2sag+T2tra	76.00 ACC, 76.00 F1, 74.83 AUC	Above baseline and either module alone in ACC/F1
T2sag+CTbone	74.00 ACC, 73.46 F1, 74.86 AUC	Matches F-only ACC, improves over baseline
T2sag+CTtissue	70.00 ACC, 70.58 F1, 72.37 AUC	Best among listed variants
T1sag+T2tra	80.00 ACC, 80.77 F1, 82.64 AUC	Best reported triplet in the ablation table
T1sag+CTbone	82.00 ACC, 83.01 F1, 80.35 AUC	Highest ACC reported in the study
T1sag+CTtissue	78.00 ACC, 78.43 F1, 80.24 AUC	Best among listed variants

The overall baseline comparison in Table 2 places the proposed framework above several generic and medical backbones on the reported modality pairs. For T1sag+T2tra, ResNet obtains ACC 60.0, AUC 66.3, F1 52.3; DenseNet 74.0, 74.2, 73.5; PBTC TransNet 76.0, 79.5, 76.0; CoPAS 74.0, 81.7, 72.3; and the proposed method 80.0, 82.6, 80.7. For T1sag+CTbone, ResNet obtains ACC 64.0, AUC 66.1, F1 57.1; DenseNet 66.0, 66.2, 65.3; PBTC TransNet 70.0, 86.5, 76.9; CoPAS 74.0, 82.3, 72.3; and the proposed method 82.0, 80.3, 83.0. The peak accuracy reported for the framework is therefore 82%.

These findings are interpreted in the paper as evidence for several advantages of ABPDC: stronger texture extraction, grayscale invariance of the central-difference term, robustness to anisotropic volumes via z-pyramidal contraction, and per-location adaptation between additive and subtractive evidence. The paper also states that generic backbones such as ResNet, ViT, EfficientNet, and DenseNet underperform on this multimodal cross-view task, while medical backbones such as PBTC TransNet and CoPAS fare better but are surpassed by the CT/MRI-specific combination of ABPDC and FPRAN (Shan et al., 2 Sep 2025).

7. Limitations, scope, and future directions

Several constraints are stated directly. CT bone modality pairs initially challenge difference-based operators, and improvements require adaptive gating. Detailed architectural hyperparameters, including kernel sizes per layer, channel counts, normalization, and code, are not provided, which can hinder exact reproduction. Complexity and runtime are likewise not reported beyond the qualitative observation that the difference branch uses the smaller set $f_{\mathrm{MRI}}^i$0 rather than the full region $f_{\mathrm{MRI}}^i$1.

The paper presents ABPDC as broadly applicable to multimodal medical imaging tasks with heterogeneous contrast, such as PET/CT, CT/MRI, and multi-sequence MRI, and to anisotropic clinical volumes with thick slices. This is framed as generalization potential rather than as an experimentally validated transfer result. Similarly, FPRAN’s coarse-to-fine deformable alignment is described as useful wherever orthogonal or misaligned views are standard. These statements should therefore be read as scope claims and design implications, not as independent benchmarks outside MMCSD.

Future directions enumerated in the paper include explicit cross-modal ABPDC interactions, such as co-attentive ABPDC between backbones; learnable, scale-specific pyramid designs and dynamic z-contraction; explicit reporting and optimization of loss weights and complexity/runtime; and extension to transformer-based fusion while retaining grayscale invariance properties. Taken together, these directions clarify the current status of ABPDC: it is a modality-aware, anisotropy-aware convolutional operator validated within a specific CT/MRI cross-view prediction framework, rather than a fully standardized drop-in replacement for arbitrary 3D convolutions (Shan et al., 2 Sep 2025).