Bridging MRI and PET physiology: Untangling complementarity through orthogonal representations

Abstract: Multimodal imaging analysis often relies on joint latent representations, yet these approaches rarely define what information is shared versus modality-specific. Clarifying this distinction is clinically relevant, as it delineates the irreducible contribution of each modality and informs rational acquisition strategies. We propose a subspace decomposition framework that reframes multimodal fusion as a problem of orthogonal subspace separation rather than translation. We decompose Prostate-Specific Membrane Antigen (PSMA) PET uptake into an MRI-explainable physiological envelope and an orthogonal residual reflecting signal components not expressible within the MRI feature manifold. Using multiparametric MRI, we train an intensity-based, non-spatial implicit neural representation (INR) to map MRI feature vectors to PET uptake. We introduce a projection-based regularization using singular value decomposition to penalize residual components lying within the span of the MRI feature manifold. This enforces mathematical orthogonality between tissue-level physiological properties (structure, diffusion, perfusion) and intracellular PSMA expression. Tested on 13 prostate cancer patients, the model demonstrates that residual components spanned by MRI features are absorbed into the learned envelope, while the orthogonal residual is largest in tumour regions. This indicates that PSMA PET contains signal components not recoverable from MRI-derived physiological descriptors. The resulting decomposition provides a structured characterization of modality complementarity grounded in representation geometry rather than image translation.

• The paper introduces an orthogonal subspace decomposition framework to quantify irreducible PET signals from MRI in prostate cancer imaging.
• It leverages an implicit neural representation with SIREN to map seven MRI features to PET values, isolating shared and unique signal components.
• Results show that 99.9% of tumour PET error is orthogonal to MRI, indicating a unique molecular signature not captured by standard MRI.

Geometric Decomposition of Modality Complementarity in Prostate Cancer Imaging

Introduction and Motivation

This paper presents a formal analysis of modality complementarity in prostate cancer imaging, focusing on the relationship between multiparametric MRI (mpMRI) and Prostate-Specific Membrane Antigen (PSMA) PET. Rather than approaching multimodal fusion through naïve translation or joint latent representation, the authors introduce a subspace decomposition framework. They interrogate the recoverability of PET signals from MRI-derived physiological descriptors, explicitly quantifying the irreducible PET signal with respect to MRI through orthogonal residuals. This paradigm shift reframes cross-modal synthesis as the separation of orthogonal subspaces, delineating the boundaries between shared and unique signal components.

Figure 1: Seven MRI sequences are used to synthesize a PSMA PET image using an INR, enforcing orthogonality via SVD-based loss to isolate the unique PET signal not recoverable from MRI.

Methods: Orthogonal Subspace Decomposition

The authors operationalize their framework by training an implicit neural representation (INR) with a Sinusoidal Representation Network (SIREN) coupled with Gaussian Fourier Feature encoding. The network learns the mapping from seven-dimensional MRI feature vectors ($T_1$, $T_2$, ADC, $K^{\mathrm{trans}}$, $v_e$, $v_p$, TTP) to PET Standard Uptake Values (SUV) as independent intensity-wise regressions, eschewing spatial convolutional priors to avoid bias.

Residuals from PET prediction are geometrically decomposed via singular value decomposition:

• Parallel Residual ($r_\parallel$): Projection into the span of the MRI feature manifold, penalized in the objective to drive learnable signal into the physiological envelope.
• Orthogonal Residual ($r_\perp$): Constitutes PET signal orthogonal to the MRI feature manifold, representing information irrecoverable from MRI, regardless of modeling capacity.

The training objective penalizes the parallel component via MSE weighted loss, thereby structuring the learned representation to absorb MRI-explainable signal into the envelope and isolate orthogonal PET contributions.

Results: Signal Separation and Tissue Stratification

Empirical validation was conducted on a cohort of 13 prostate cancer patients with mpMRI and PSMA-PET. Rigorous registration and normalization protocols ensured robust feature extraction. Tissue-specific residual analysis yielded strong numerical findings:

• Tumour regions: The total MSE ($0.9723$) is nearly $10\times$ higher than non-tumour regions ($0.0875$–$T_2$0). Critically, $T_2$1 of PET signal error in tumours manifests as orthogonal residual ($T_2$2), indicating that the malignant PET signal is almost entirely irreducible from the MRI feature space.
• Non-tumour prostate and pelvic tissue: The orthogonal residual is substantially reduced, corroborating the hypothesis that MRI feature space encapsulates physiological processes in healthy tissue, while tumour biology yields unique molecular signatures.

  Figure 2: Orthogonal and parallel error metrics across tissue classes reveal dominance of irreducible PET signal in tumours.

Visualizations at the patient level further reinforce the partitioning: the squared orthogonal residual localizes to tumour regions, while parallel residuals remain minimal. Ablation studies systematically removing MRI modalities or groups show increased MSE predominantly affecting dynamic and perfusion features, confirming their central role in PET envelope prediction.

Theoretical and Practical Implications

This geometric separation has far-reaching implications:

• Theoretical rigor: The method grounds modality complementarity in representation geometry, moving beyond latent factor disentanglement toward explicit signal partitioning. The orthogonal residual provides a principled ceiling for virtual PET synthesis from MRI.
• Clinical workflow optimization: Identifying regions of irreducible PET signal supports targeted acquisition strategies. Molecular imaging can be deployed selectively, focusing on areas where MRI fails to capture the full physiological spectrum.
• Extensibility: This subspace decomposition is generalizable to other modality combinations (e.g., CT-to-PET, FET PET/mpMRI) and lays a foundation for systematic mapping of signal boundaries in multimodal imaging.
• Interpretability: The orthogonal residual provides a concrete interpretive axis, associating molecular PET signal in tumours with processes unobservable by MRI-derived descriptors, aligning with known receptor-mediated pathophysiology.

Speculation on Future Developments

Future avenues include:

• Deployment at scale across diverse cancer types and modalities to formally map cross-modal information boundaries.
• Integration with diagnostic workflows to facilitate selective molecular imaging, reducing unnecessary radiation exposure.
• Extension to domain separation in world modeling, potentially impacting unsupervised representation learning in AI for medical imaging.
• Incorporation into federated and multi-site models to improve robustness and interpretability across heterogeneous cohorts.

Conclusion

The paper reframes multimodal medical imaging as a problem of structured signal separation in geometric feature space. By enforcing orthogonality between MRI physiological envelopes and PET molecular signals, it quantifies irreducible information content and informs theoretical limits for cross-modal synthesis. This framework advances representational clarity in medical AI, offering practical benefit in optimizing diagnostic protocols for prostate cancer and beyond.