TomoGraphView: 3D Medical Classification

• TomoGraphView is a two-stage framework that decomposes 3D medical images into omnidirectional 2D slices and aggregates features via a spherical GNN to preserve volumetric geometry.
• It leverages a frozen DINOv2 2D encoder to extract 384-dimensional features from each 224x224 slice, which are assembled into a graph reflecting inter-slice relationships.
• Evaluated on six oncology tasks, TomoGraphView achieves superior AUROC compared to canonical slicing and traditional 3D backbones, demonstrating its practical effectiveness.

Searching arXiv for the specified paper and closely related context. TomoGraphView is a two-stage framework for 3D medical image classification that combines omnidirectional volume slicing with graph-based feature aggregation in order to preserve volumetric geometry while exploiting pretrained 2D vision models (Kiechle et al., 12 Nov 2025). It is designed for CT and MRI volumes containing segmented lesions, and addresses a central limitation of slice-based decomposition methods: canonical planes such as axial, sagittal, and coronal may fail to capture the spatial extent of target structures when those structures are misaligned with standardized viewing planes, while conventional slice-wise aggregation can lose spatial coherence across slices (Kiechle et al., 12 Nov 2025). In TomoGraphView, a 3D volume is first decomposed into semantically rich omnidirectional 2D slices, then the resulting slice embeddings are re-assembled by a spherical graph neural network prediction head. The method is evaluated on six oncology classification problems and is presented as a way to bridge pretrained 2D vision foundation models and volumetric medical imaging (Kiechle et al., 12 Nov 2025).

1. Conceptual basis and problem setting

3D medical image classification is described as challenging because volumetric data exhibit complex spatial relationships and long-range dependencies (Kiechle et al., 12 Nov 2025). The same source identifies two additional constraints: training models from scratch suffers from low data regimes, and the absence of 3D large-scale multimodal datasets has limited the development of 3D medical imaging foundation models (Kiechle et al., 12 Nov 2025). Recent work has nevertheless highlighted the utility of 2D vision foundation models, originally trained on natural images, as feature extractors for medical image analysis, but direct application of 2D models to 3D volumes through standard slice-based decomposition is characterized as suboptimal (Kiechle et al., 12 Nov 2025).

The central design of TomoGraphView is therefore to decouple feature extraction from volumetric aggregation. A frozen 2D encoder extracts representations from individual slices, while a graph-structured head restores structural context at the level of the volume (Kiechle et al., 12 Nov 2025). This suggests a methodological compromise between purely 3D architectures, which may be constrained by data availability, and purely 2D slice pipelines, which may underrepresent three-dimensional geometry.

The framework assumes a CT or MRI volume $V \subset \mathbb{R}^3$ containing a segmented lesion. A tight bounding box is extracted around the lesion, and the volume is resampled to isotropic voxel size $1\,\mathrm{mm}$ before omnidirectional slice generation (Kiechle et al., 12 Nov 2025). This preprocessing step is not incidental: the entire sampling procedure is defined geometrically relative to the lesion-centered volume.

2. Omnidirectional volume slicing

The first stage encloses the preprocessed volume in a sphere

$S = \{ x \in \mathbb{R}^3 : \|x\|_2 \le r \}$

whose center coincides with the center of $V$ and whose radius $r$ just covers the bounding box (Kiechle et al., 12 Nov 2025). TomoGraphView then samples $N$ viewpoints on this sphere. Three fixed canonical points are defined as

$$p_1 = (r,0,0), \quad p_2 = (0,r,0), \quad p_3 = (0,0,r),$$

corresponding to the axial, coronal, and sagittal normals (Kiechle et al., 12 Nov 2025). The remaining $N-3$ points are initialized uniformly on the sphere and optimized by minimizing the total electrostatic Thomson energy

$E = \sum_{i \ne j} \frac{1}{\|x_i - x_j\|^2},$

subject to $\|x_i\| = r$ for all $1\,\mathrm{mm}$0 (Kiechle et al., 12 Nov 2025). In practice, each nonfixed point is moved along the gradient of $1\,\mathrm{mm}$1 and re-projected onto the sphere until convergence, yielding a near-uniform distribution of normals $1\,\mathrm{mm}$2 (Kiechle et al., 12 Nov 2025).

Each normal $1\,\mathrm{mm}$3 defines a slicing plane through the lesion. For each direction, the method aligns a plane whose normal vector is $1\,\mathrm{mm}$4 and scans along that direction to find the single cross-section that maximizes the lesion’s 2D area, or equivalently, intersects the lesion mask most fully (Kiechle et al., 12 Nov 2025). Every viewpoint therefore yields one 2D slice $1\,\mathrm{mm}$5 of size $1\,\mathrm{mm}$6 after cropping and resampling (Kiechle et al., 12 Nov 2025).

This construction is the defining departure from canonical slicing. Rather than restricting the representation to standardized anatomical planes, it samples the lesion from a near-uniform set of directions on a sphere. A plausible implication is that the representation becomes less sensitive to orientation mismatch between lesion morphology and canonical viewing planes, particularly in cases of anisotropic acquisition where the diagnostically salient cross-section may not align with axial, sagittal, or coronal directions.

3. Spherical graph construction and feature aggregation

After slice generation, TomoGraphView forms an undirected graph $1\,\mathrm{mm}$7 whose $1\,\mathrm{mm}$8 nodes correspond to the sampled slice planes (Kiechle et al., 12 Nov 2025). Let $1\,\mathrm{mm}$9 be the set of unit-norm normals. A spherical Delaunay triangulation is performed on $S = \{ x \in \mathbb{R}^3 : \|x\|_2 \le r \}$0, and two points $S = \{ x \in \mathbb{R}^3 : \|x\|_2 \le r \}$1 are connected if they share an edge in the triangulation:

$S = \{ x \in \mathbb{R}^3 : \|x\|_2 \le r \}$2

This produces a sparse, near-regular mesh on the sphere (Kiechle et al., 12 Nov 2025).

To enable longer-range message passing, the method can also add all cross-connections $S = \{ x \in \mathbb{R}^3 : \|x\|_2 \le r \}$3 for $S = \{ x \in \mathbb{R}^3 : \|x\|_2 \le r \}$4 in complete-graph fashion, weighting each edge by the inverse of the hop distance in the local mesh (Kiechle et al., 12 Nov 2025). If $S = \{ x \in \mathbb{R}^3 : \|x\|_2 \le r \}$5 is the shortest-path length between nodes $S = \{ x \in \mathbb{R}^3 : \|x\|_2 \le r \}$6 and $S = \{ x \in \mathbb{R}^3 : \|x\|_2 \le r \}$7 in the local mesh, then

$S = \{ x \in \mathbb{R}^3 : \|x\|_2 \le r \}$8

and the weighted adjacency matrix $S = \{ x \in \mathbb{R}^3 : \|x\|_2 \le r \}$9 is defined with $V$0 for $V$1 and zero diagonal (Kiechle et al., 12 Nov 2025).

Each node is attributed with a 384-dimensional feature vector $V$2 obtained from a frozen DINOv2-ViT-small encoder:

$V$3

The graph aggregation module is a message-passing GNN in the style of GraphSAGE (Kiechle et al., 12 Nov 2025). With $V$4, layer $V$5 computes

$V$6

where $V$7 are the neighbors of node $V$8 and $V$9 is a permutation-invariant aggregator; TomoGraphView uses mean and max aggregations in parallel and concatenates their outputs (Kiechle et al., 12 Nov 2025). Node states are updated as

$r$0

After $r$1 layers, typically $r$2, a global readout takes the dimension-wise mean over nodes,

$r$3

and a linear classifier $r$4 produces the predicted logit $r$5 for binary classification (Kiechle et al., 12 Nov 2025). Training uses standard binary cross-entropy loss (Kiechle et al., 12 Nov 2025).

The graph component is not merely a classifier appended to slice embeddings. Its stated role is to preserve volumetric geometry during aggregation. This suggests that the spherical adjacency is intended to encode inter-view relationships intrinsic to the sampling procedure itself, rather than treating slices as an unordered set or as a simple sequence.

4. Integration of pretrained 2D vision models

TomoGraphView uses a DINOv2 vision transformer, specifically the small variant with 21 M parameters, pretrained on 142 M natural images via self-supervised learning (Kiechle et al., 12 Nov 2025). Each $r$6 slice is passed through the frozen DINOv2 backbone, and the 384-dimensional $r$7 token embedding is used as the node feature $r$8 (Kiechle et al., 12 Nov 2025).

No fine-tuning of DINOv2 is performed. Only the GNN head and a small MLP classification head, consisting of two layers with ReLU in between and approximately 100 k trainable parameters, are trained end-to-end on each task (Kiechle et al., 12 Nov 2025). This sharply constrains the trainable portion of the system relative to full backbone optimization.

The architecture therefore relies on transfer from natural-image self-supervised pretraining while avoiding task-specific adaptation of the 2D backbone. This suggests that TomoGraphView treats DINOv2 as a generic slice encoder and places most domain adaptation burden on the graph-based prediction head. A plausible implication is that the framework is especially suited to settings where dataset size is insufficient for robust full-model fine-tuning.

5. Training protocol and evaluation setting

The framework is evaluated on six public oncological tasks: brain tumor grading using T1-c MRI with $r$9, head–neck HPV status using CT with $N$0, breast tumor grade using DCE-MRI with $N$1, liver tumor subtype using CT with $N$2, kidney tumor grade using CT with $N$3, and soft-tissue sarcoma grade using T2FS MRI with $N$4 (Kiechle et al., 12 Nov 2025). Volumes are reoriented to RAS+, resampled to $N$5 isotropic resolution, and cropped around lesion masks (Kiechle et al., 12 Nov 2025).

Evaluation uses 5-fold stratified cross-validation with three folds for training, one for validation, and one for testing (Kiechle et al., 12 Nov 2025). Optimization is performed with SGD using weight decay $N$6, momentum $N$7, batch size $N$8, and $N$9 epochs (Kiechle et al., 12 Nov 2025). The learning rate is linearly warmed up over $$p_1 = (r,0,0), \quad p_2 = (0,r,0), \quad p_3 = (0,0,r),$$0 epochs to a peak of $$p_1 = (r,0,0), \quad p_2 = (0,r,0), \quad p_3 = (0,0,r),$$1, then reduced by a factor of $$p_1 = (r,0,0), \quad p_2 = (0,r,0), \quad p_3 = (0,0,r),$$2 on any validation-AUROC stall of five epochs (Kiechle et al., 12 Nov 2025). The best model by validation AUROC is evaluated on the held-out test fold, and no additional data augmentation is applied beyond cropping and resizing (Kiechle et al., 12 Nov 2025).

The primary metric is Area Under the Receiver-Operating Characteristic Curve (AUROC), with balanced accuracy, F1-score, and Matthews correlation coefficient (MCC) used in ablation tables (Kiechle et al., 12 Nov 2025). The emphasis on AUROC is consistent with the binary classification formulation used throughout the reported experiments.

6. Quantitative performance and ablation results

Across all six tasks, TomoGraphView achieves an average test AUROC of $$p_1 = (r,0,0), \quad p_2 = (0,r,0), \quad p_3 = (0,0,r),$$3 with $$p_1 = (r,0,0), \quad p_2 = (0,r,0), \quad p_3 = (0,0,r),$$4 viewpoints (Kiechle et al., 12 Nov 2025). This is compared with $$p_1 = (r,0,0), \quad p_2 = (0,r,0), \quad p_3 = (0,0,r),$$5 when omnidirectional slices are pooled by a simple MLP, and $$p_1 = (r,0,0), \quad p_2 = (0,r,0), \quad p_3 = (0,0,r),$$6 for canonical slice-based MLP baselines (Kiechle et al., 12 Nov 2025). In head–neck, kidney, and liver classification, the GNN head yields gains of $$p_1 = (r,0,0), \quad p_2 = (0,r,0), \quad p_3 = (0,0,r),$$7 to $$p_1 = (r,0,0), \quad p_2 = (0,r,0), \quad p_3 = (0,0,r),$$8 AUROC over the omnidirectional+MLP variant (Kiechle et al., 12 Nov 2025).

The method is also compared against five large-scale pretrained 3D backbones—Models Genesis, VOCO, VISTA3D, FMCIB, and SwinUNETR—using frozen embeddings plus a 100 k-parameter classifier (Kiechle et al., 12 Nov 2025). Under this comparison, TomoGraphView surpasses the best 3D model, FMCIB, with average AUROC $$p_1 = (r,0,0), \quad p_2 = (0,r,0), \quad p_3 = (0,0,r),$$9 versus $N-3$0 (Kiechle et al., 12 Nov 2025). The report further states that even FMCIB fine-tuned for 200 epochs with large batch sizes does not match TomoGraphView’s average AUROC of $N-3$1 (Kiechle et al., 12 Nov 2025).

Three ablation axes are reported (Kiechle et al., 12 Nov 2025). First, varying the number of omnidirectional views $N-3$2 shows steady improvement as $N-3$3 increases, especially on datasets with high slice anisotropy, for example $N-3$4-spacing $N-3$5 (Kiechle et al., 12 Nov 2025). Second, graph topology is varied among local Delaunay-only connectivity and complete graphs with uniform, inverse-hop, and inverse-hop$N-3$6 weighting. The complete graph with inverse-hop weighting yields the best mean AUROC of $N-3$7, while the sparse local mesh still attains $N-3$8 (Kiechle et al., 12 Nov 2025). Third, alternative slice-aggregation baselines also benefit from omnidirectional slicing: LSTM AUROC rises from $N-3$9, and Medical Slice Transformer AUROC rises from $E = \sum_{i \ne j} \frac{1}{\|x_i - x_j\|^2},$0, but neither matches the GNN head (Kiechle et al., 12 Nov 2025).

Comparison	Reported average AUROC
TomoGraphView, $E = \sum_{i \ne j} \frac{1}{\|x_i - x_j\|^2},$1	0.8372
Omnidirectional slices + simple MLP	0.8154
Canonical slice-based MLP baseline	0.7701
Best 3D model (FMCIB)	0.7170
Local mesh only	0.8305

These results isolate two contributions: the omnidirectional slicing strategy itself and the graph-based aggregation mechanism. The ablations indicate that both matter. Omnidirectional views improve multiple non-graph aggregators, while the GNN further improves performance over omnidirectional slices pooled without explicit geometry (Kiechle et al., 12 Nov 2025).

7. Limitations, clinical implications, and projected extensions

The framework currently requires a lesion segmentation mask or bounding box to center slice sampling, which introduces a dependency on pre-existing delineations (Kiechle et al., 12 Nov 2025). The source notes, however, that in many clinical workflows bounding boxes suffice and could be auto-generated (Kiechle et al., 12 Nov 2025). Another limitation is that slice interpolation across $E = \sum_{i \ne j} \frac{1}{\|x_i - x_j\|^2},$2 orientations adds modest preprocessing cost (Kiechle et al., 12 Nov 2025).

The reported clinical relevance is framed around improved AUROC on tumor grading and HPV status prediction, which is said to suggest potential to assist radiologists in risk stratification, especially when 3D models are too data-hungry to train from scratch (Kiechle et al., 12 Nov 2025). This should not be read as a claim of clinical deployment; rather, it indicates the intended downstream role of the framework in oncology-oriented decision support.

Several future directions are explicitly identified. One is integration of a detection module to propose bounding boxes, potentially making the pipeline end-to-end (Kiechle et al., 12 Nov 2025). Another is the possibility of leveraging multimodal LLMs for in-context 3D localization (Kiechle et al., 12 Nov 2025). The framework is also described as flexible with respect to stronger 2D encoders or 3D feature extractors, and as potentially extensible to multi-class problems or survival-time prediction (Kiechle et al., 12 Nov 2025).

A common misconception would be to interpret TomoGraphView as a native 3D foundation model. The reported formulation does not do so. Instead, it is a wrapper that couples omnidirectional 2D sampling to a spherical-mesh GNN and a frozen pretrained 2D encoder (Kiechle et al., 12 Nov 2025). Its contribution lies in how 3D structure is sampled and reconstructed at the representation level, not in end-to-end volumetric pretraining.