Depth-Sequence Transformer for CT Landmark Mapping

• The paper introduces a hybrid CNN–Transformer framework that reformulates dense 3D segmentation into a 1D axial sequence learning problem for ICAC landmark localization.
• It employs parallel probabilistic landmark localization to predict six independent probability distributions over axial slices, achieving MAE near 0.1 slices with 96% accuracy.
• The DST architecture leverages anisotropic 3D convolutions and depth-axis self-attention to preserve full resolution and global anatomical context for precise subsegment mapping.

Searching arXiv for the named method and closely related uses of the term. Searching arXiv for "Depth-Sequence Transformer" and related "depth-wise LSTM transformer" formulations. Depth-Sequence Transformer (DST) is a hybrid CNN–Transformer framework that linearizes the CT depth axis into a sequence and solves segment-specific intracranial carotid artery calcification (ICAC) mapping by predicting six probabilistic landmark locations rather than directly segmenting all calcified voxels. In the formulation introduced for non-contrast head CT, DST processes a full-resolution volume as an ordered sequence of axial slices, learns $N=6$ independent probability distributions over slice indices, and uses the resulting landmark positions to assign calcified voxels to ICA subsegments. This reformulation replaces a dense 3D segmentation problem with a 1D axial-depth sequence learning problem, while preserving full axial resolution and global anatomical context (Hou et al., 10 Jul 2025).

1. Reformulation as axial sequence learning

The defining idea of DST is to treat a 3D CT volume not as a tensor to be exhaustively modeled by full 3D attention, but as a sequence of 2D slices ordered along the axial dimension. Let the input CT volume be

$\mathcal{V} \in \mathbb{R}^{H \times W \times D}.$

Instead of predicting a dense 3D label map directly, DST predicts the slice locations of a small set of anatomical reference planes. For each landmark $j \in \{1,\dots,N\}$, with $N=6$, the ground-truth slice index is $z_j$, and the target is encoded as a one-hot vector over depth,

$$\mathcal{Y}_i^{(j)} = \delta_{i,z_j} \qquad \forall i \in \{1, \dots, D\}.$$

The model learns

$$f_\theta(\mathcal{V}) = \mathcal{P} = \{\mathcal{P}^{(1)}, \dots, \mathcal{P}^{(N)}\},$$

where each $\mathcal{P}^{(j)} \in \mathbb{R}^{D}$ is a probability distribution over slice indices for landmark $j$. The predicted landmark location is the argmax over depth for each distribution (Hou et al., 10 Jul 2025).

This formulation is termed Parallel Probabilistic Landmark Localization: it is parallel because all landmarks are predicted at once, probabilistic because each landmark is modeled as a distribution over slices, and localization-oriented because each distribution is expected to peak at the landmark slice. The paper’s motivation is explicitly anatomical. Aggregate ICAC volume is described as too coarse because it ignores where the calcium sits along the intracranial internal carotid artery, whereas segment-specific patterns may better capture local wall biology, stroke mechanisms, and procedural difficulty. The reformulation is therefore not merely computational; it is aligned to a clinical claim that location-specific calcium burden may carry prognostic and procedural information that total volume does not (Hou et al., 10 Jul 2025).

A central technical rationale is the failure mode of conventional 3D models on this task. Downsampled full volumes preserve context but lose the fine detail required for boundary localization; high-resolution patches preserve detail but lose global positional context, creating left/right ambiguity and segment mislabeling. DST addresses this tradeoff by preserving full axial ordering and learning over the depth sequence directly. This suggests that the method’s core contribution is the choice of representation as much as the choice of backbone.

2. Architectural pipeline and sequence modeling

DST begins with slice-wise feature extraction using a 3D CNN backbone with anisotropic 3D convolutions and stride $(2,2,1)$. This aggressively downsamples the in-plane dimensions $\mathcal{V} \in \mathbb{R}^{H \times W \times D}.$0 while preserving the full axial resolution $\mathcal{V} \in \mathbb{R}^{H \times W \times D}.$1. After several stages, the spatial dimensions are pooled away, producing a feature map of shape $\mathcal{V} \in \mathbb{R}^{H \times W \times D}.$2, which is interpreted as a sequence of $\mathcal{V} \in \mathbb{R}^{H \times W \times D}.$3 feature vectors, one per axial slice (Hou et al., 10 Jul 2025).

The sequence is then passed through a stack of DST blocks implementing global self-attention over the depth dimension. Because attention is applied only along the 1D slice axis after CNN compression, the paper states that the attention burden is reduced from roughly

$\mathcal{V} \in \mathbb{R}^{H \times W \times D}.$4

A boolean attention mask ensures that padded positions are ignored in self-attention. The model does not resample the CT volume to a canonical depth; instead, it pads sequences to a fixed maximum length. The paper explicitly argues that resampling would distort the physical mapping between slice index and anatomy. It also reports that left-padding matters empirically, claiming that left-padding gives the Transformer a stable coordinate anchor at the end of the sequence and preserves alignment of anatomical positions across patients (Hou et al., 10 Jul 2025).

DST has two output heads. The main head is a Parallel Landmark Localization Head: a linear layer is applied to each output slice token, predicting $\mathcal{V} \in \mathbb{R}^{H \times W \times D}.$5 logits per token and producing an output tensor of shape $\mathcal{V} \in \mathbb{R}^{H \times W \times D}.$6. A Softmax is then applied independently along the depth dimension $\mathcal{V} \in \mathbb{R}^{H \times W \times D}.$7 for each landmark channel, yielding the landmark distributions $\mathcal{V} \in \mathbb{R}^{H \times W \times D}.$8. For external benchmarking, the architecture also includes a classification head acting on a prepended $\mathcal{V} \in \mathbb{R}^{H \times W \times D}.$9 token; the paper notes that the $j \in \{1,\dots,N\}$0 token can serve as a global aggregator of sequence information, consistent with the “attention sink” idea where attention tends to concentrate on the first token (Hou et al., 10 Jul 2025).

A diagram in the paper also indicates a DepthAttentionBlock with convolution-enhanced self-attention. This suggests a hybridization of sequence attention with convolutional inductive bias, although the core architectural principle remains depth-axis sequence modeling rather than volumetric tokenization.

3. Anatomical priors, landmark definition, and supervision

DST is embedded in a larger pipeline that includes a deterministic preprocessing stage for left/right separation of the ICA territories. This step thresholds the CT and extracts the skull, finds the skull centroid, uses DICOM metadata—especially ImageOrientationPatient—to define the midsagittal plane, and splits the volume into left and right masks. The purpose is to impose a reproducible anatomical coordinate system before landmark localization, since segment labels are side-specific (Hou et al., 10 Jul 2025).

The paper describes a four-segment ICA scheme in the introduction—cervical, petrous, cavernous, and supraclinoid—but the localization task itself is defined through six landmarks, specifically on each side: left carotid canal, left petrolingual ligament, left anterior clinoid ligament, right carotid canal, right petrolingual ligament, and right anterior clinoid ligament. These landmarks define the anatomical boundaries used for segment-specific labeling of calcified voxels. The exact segment mapping is built by using the predicted landmark slice positions to assign the binary ICAC mask into segment-specific classes (Hou et al., 10 Jul 2025).

For landmark localization, the training objective is the sum of cross-entropies over all landmarks,

$j \in \{1,\dots,N\}$1

For the Clean-CC-CCII classification benchmark, the classification head is trained with standard cross-entropy loss, denoted $j \in \{1,\dots,N\}$2 (Hou et al., 10 Jul 2025).

The clinical dataset consists of 100 non-contrast head CT scans from one institution; 1 scan was excluded due to incorrect gantry angle; patient ages were 40–90 years with mean age: 72.5 ± 10.8; and ground truth was created by two trained expert readers. The annotations included binary calcification masks and six landmark slice indices. Evaluation used 5-fold patient-level cross-validation, with patient-level splitting to prevent leakage and a validation set for early stopping. Optimization used SGD with learning rate 2e-3 (Hou et al., 10 Jul 2025).

4. Quantitative performance and ablation evidence

For landmark localization, DST is reported to achieve excellent precision. The abstract reports a Mean Absolute Error (MAE) of 0.1 slices, with 96\% of predictions falling within a $j \in \{1,\dots,N\}$3 slice tolerance. In the detailed results, the paper reports overall MAE = 0.13 slices and Acc$j \in \{1,\dots,N\}$4 ≈ 0.96, along with near-perfect Top-1 and Top-2 accuracy and a quadratic weighted Cohen’s kappa approaching 1.0 (Hou et al., 10 Jul 2025).

The per-landmark errors are all on the order of a few tenths of a slice: left carotid canal 0.06 ± 0.08, left petrolingual ligament 0.17 ± 0.12, left anterior clinoid ligament 0.15 ± 0.21, right carotid canal 0.13 ± 0.17, right petrolingual ligament 0.17 ± 0.14, and right anterior clinoid ligament 0.12 ± 0.12. The paper also reports strong agreement between predicted and ground-truth calcium volumes across intracranial segments, supporting the utility of landmark localization for downstream segment-specific ICAC quantification (Hou et al., 10 Jul 2025).

The baseline comparison is framed as evidence for the reformulation itself. nnU-Net-style end-to-end segmentation approaches do reasonably on binary ICAC segmentation with Dice about 0.66, but performance drops substantially on left-versus-right ICA differentiation, and a direct 8-class segmentation model fails to converge. In the paper’s interpretation, conventional dense segmentation methods struggle because local appearance is anatomically ambiguous and fine-grained side-specific labeling depends on global context (Hou et al., 10 Jul 2025).

The ablations isolate two design choices. First, replacing the DST attention blocks with plain 1D convolutions degrades performance, with MAE worsening to about 0.27 slices and top-1 and tolerance accuracy also dropping. Second, right-padding performs worse than left-padding, supporting the claim that padding choice affects coordinate stability and alignment. On the external Clean-CC-CCII 3-class CT classification benchmark, the DST backbone yields AUC: 0.99 ± 0.01, F1: 0.92 ± 0.01, Accuracy: 0.92 ± 0.01, and 35.57 GFLOPs; the paper describes this as the best reported performance in its comparison table. A classification-depth ablation further reports that 0 layers gives near-random performance, 1 layer gives a large jump, and 8 layers gives the best or near-best performance (Hou et al., 10 Jul 2025).

Taken together, these results support a specific conclusion: the transformer component is not a marginal add-on to a 3D CNN, and the decisive architectural change is the conversion of volumetric localization into sequence modeling over depth.

5. Nomenclature, related formulations, and the broader “DST-style” family

The acronym DST is not unique across the literature. In multi-object tracking, DST denotes Dense Spatio-Temporal position encoding, a transformer positional encoding applied directly to dense 2D CNN feature maps so that target association can exploit motion and location continuity beyond appearance matching. That work argues that standard transformer positional encodings, designed for 1D token sequences, lose positional information when applied after projection, whereas dense encoding on feature maps preserves pixel-level geometry and can represent both single-frame location and multi-frame trajectory evolution uniformly (Cao et al., 2022). This is a distinct use of the acronym from Depth-Sequence Transformer.

Several other papers are relevant not because they use the exact name, but because they instantiate a similar principle: a transformer is applied along a nonstandard sequence axis associated with depth, temporal depth, or network depth. DepthFormer uses depth-discretized epipolar sampling and alternating self-/cross-attention to refine matching over a per-pixel sequence of depth hypotheses; the paper explicitly states that it does not use the exact term “Depth-Sequence Transformer (DST),” but describes the architecture as “DST-like” in spirit (Guizilini et al., 2022). STDepthFormer is described as a self-supervised depth-sequence transformer for video, predicting a sequence of present and future depth states from monocular context and enforcing spatio-temporal consistency across the output sequence rather than a single frame (Boulahbal et al., 2023). FocDepthFormer separates spatial reasoning from stack-wise aggregation by combining a Transformer encoder with a latent-space LSTM that fuses ordered focal-stack frames of arbitrary length; its detailed description explicitly notes that, in a “DST-style” sense, the sequence is the focal-stack order and the LSTM acts as the sequence-aware aggregator across depth/focus positions (Kang et al., 2023).

A different but conceptually related reinterpretation appears in “Rewiring the Transformer with Depth-Wise LSTMs”, where stacked transformer layers are treated as a vertical sequence over network depth and connected by a depth-wise LSTM rather than simple residual addition. That paper characterizes the model as DST-like because sequence modeling happens along the depth axis of the network rather than time or token position (Xu et al., 2020). A theoretical complement is provided by “What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks”, which argues that increasing transformer depth enables progressively richer compositions of simple operations: one layer can memorize, two layers suffice for reasoning and template generalization, and three layers are needed for contextual generalization (Chen et al., 2024).

This broader usage suggests that “Depth-Sequence Transformer” functions in two ways in the literature. In the strict sense, it names the ICAC landmark-localization framework on CT (Hou et al., 10 Jul 2025). In a looser sense, it denotes an architectural pattern in which a transformer models an ordered depth-related sequence—axial slices, depth hypotheses, focal positions, future depth states, or even layer depth—rather than only a conventional token sequence. A common misconception is therefore to treat all such systems as one method family with identical semantics. The available papers do not support that collapse; they support an analogy across formulations, not an identity of architecture or task.

6. Significance, scope, and open interpretation

Within medical imaging, DST’s stated significance is practical as well as methodological. The paper presents it as a feasible way to automate segment-specific ICAC analysis on routine non-contrast CT, a task described as previously impractical because reliable localization required both fine detail and full anatomical context (Hou et al., 10 Jul 2025). The clinical motivation is explicit: cervical calcification may interfere with balloon or stent expansion, petrous calcification may complicate catheter navigation and cause “ledge effect,” and cavernous/supraclinoid calcification is associated with arterial stiffness, perforator compromise, and embolic infarction patterns. Segment-specific mapping is therefore positioned as relevant to diagnosis, prognosis, and procedural planning (Hou et al., 10 Jul 2025).

The architectural significance is broader. DST replaces direct dense 3D segmentation with prediction of a small set of probabilistic anatomical landmarks, then composes those predictions into downstream segment labels. This suggests a general design principle for anatomy-dependent tasks in which local appearance is ambiguous but global positional context is decisive. The paper itself argues that the same framework could be adapted to vertebra labeling, cardiac phase detection, or vessel bifurcation localization (Hou et al., 10 Jul 2025).

At the same time, the scope of the evidence is bounded by the reported setting: 100 non-contrast head CT scans from one institution, with 5-fold patient-level cross-validation (Hou et al., 10 Jul 2025). A plausible implication is that external validation, scanner heterogeneity, and broader anatomical variation remain important questions, although the paper’s Clean-CC-CCII result does provide evidence that the backbone is not narrowly specialized to one vascular task. More generally, the related literature indicates that the most durable idea behind DST is not any single implementation detail, but the decision to align the transformer’s sequence axis with the latent structure of the problem—axial order in CT, epipolar depth candidates in monocular geometry, focus order in focal stacks, future depth states in video, or stacked layers in model depth (Guizilini et al., 2022, Kang et al., 2023, Boulahbal et al., 2023, Xu et al., 2020).

In that sense, DST occupies both a narrow and a broad place in the recent transformer literature: narrowly, as a specific landmark-localization architecture for segment-specific ICA calcification mapping; broadly, as an instance of a research trend in which transformers are repurposed to model structured depth sequences rather than flat token streams.