SKGE-Swin Navigation Architecture

• The architecture integrates Swin Transformer blocks with a novel skip-stage mechanism to retain high-resolution details while capturing global context for improved waypoint prediction.
• It combines RGB, depth, and LiDAR-BEV inputs to generate semantic segmentation and BEV representations, which are processed by a GRU-based controller for vehicle control.
• Empirical evaluations on CARLA indicate that SKGE-Swin boosts driving scores and reduces collision infractions compared to CNN baselines in challenging, adversarial scenarios.

Searching arXiv for the primary paper and the original Swin Transformer paper to ground citations. SKGE-Swin is an end-to-end autonomous vehicle waypoint prediction and navigation architecture that combines Swin Transformer blocks with a skip-stage mechanism to preserve fine spatial information while expanding global contextual representation across multiple network levels. In the reported formulation, the model begins from a forward RGB image, a corresponding depth map, and, for some variants, a LiDAR-BEV histogram; it performs semantic segmentation, constructs a semantic-depth cloud and top-down bird’s-eye-view representation, extracts BEV features with a second SKGE-Swin backbone, and predicts waypoints and vehicle-control signals through a GRU-based controller. The architecture was evaluated on CARLA under adversarial scenarios, where it achieved a higher Driving Score than the reported baseline methods (Kartiman et al., 28 Aug 2025).

1. Architectural definition and system pipeline

The architecture is organized as a sequential perception-to-control pipeline with two major encoder stages and an autoregressive controller. The input specification comprises a forward RGB image $R\in\{0,\dots,255\}^{3\times256\times256}$, a depth map $D\in\mathbb{R}^{1\times256\times256}$, and, in some variants, a LiDAR-BEV histogram in $\mathbb{R}^{2\times256\times256}$. Encoder A functions as a semantic segmentation backbone based on Swin Transformer blocks augmented with skip-stage connections from shallow to deep layers. Its stated output is a 23-class pixel-wise segmentation map (Kartiman et al., 28 Aug 2025).

The segmentation output is fused with depth to form a Semantic Depth Cloud, which is then projected into a top-down BEV image. Encoder B applies an identical SKGE-Swin backbone to this BEV input, optionally together with LiDAR-BEV, and produces a spatial feature map $F\in\mathbb{R}^{C\times H\times W}$. A controller then consumes these features using a GRU loop of length 3. At each iteration, the hidden state $h_t$ is updated and linear heads predict $\Delta x,\Delta y$ waypoints, steering, throttle, brake, and traffic-light and stop-sign logits. The waypoints are represented in local coordinates and form the final trajectory.

A common misconception is to equate “end-to-end” here with direct control prediction from raw RGB alone. In the reported system, the end-to-end designation coexists with explicit intermediate structure: semantic segmentation, semantic-depth cloud formation, BEV generation, and a second feature-extraction backbone all appear in the forward path. This suggests that the architecture occupies an intermediate position between strictly monolithic sensor-to-control models and modular autonomy stacks.

2. Skip-stage feature propagation

The defining modification is the skip-stage mechanism inserted into both Encoder A and Encoder B. Features from an early stage $F^{(s)}$, exemplified by stage 1, are routed to a later stage $F^{(t)}$, exemplified by stage 4, by first bilinearly resizing the shallow feature map to the spatial resolution of the deeper feature and then combining the two by element-wise addition (Kartiman et al., 28 Aug 2025).

The resized feature map is written as

$$\widetilde{F}^{(s\to t)}(x,y) =\sum_{i\in\{x_1,x_2\}}\sum_{j\in\{y_1,y_2\}} Q_{ij}\,\frac{(x_{i+1}-x)(y_{j+1}-y)}{(x_2-x_1)(y_2-y_1)},$$

and the enrichment step is

$$F^{(t)}_{\rm enriched}=F^{(t)}+\widetilde{F}^{(s\to t)}.$$

The stated motivation is that early high-resolution local features would otherwise be attenuated by Swin Transformer patch-merging. By forwarding shallow representations into the deepest stage, the model retains fine spatial detail from the initial to the final phases of feature extraction. In the navigation setting, this is intended to improve comprehension of complex scene structure while preserving pixel-level cues that are relevant for lane geometry, object boundaries, and traffic-scene layout.

The ablation summary reported for the paper identifies the stage $D\in\mathbb{R}^{1\times256\times256}$0 routing as the most effective choice. Table A is described as showing that this routing yields the lowest semantic segmentation test loss and the highest IoU in downstream driving, while Table B is described as showing that BEV + LiDAR with stage $D\in\mathbb{R}^{1\times256\times256}$1 skip yields the best test loss. These results position the skip-stage mechanism not as an auxiliary regularizer, but as a central architectural determinant of performance.

3. Swin Transformer components and navigation-specific adaptation

SKGE-Swin reuses the original Swin Transformer building blocks introduced by Liu et al. (Liu et al., 2021), while introducing skip-stage links tailored to navigation (Kartiman et al., 28 Aug 2025). The core attention mechanism partitions the token sequence $D\in\mathbb{R}^{1\times256\times256}$2 into non-overlapping windows of size $D\in\mathbb{R}^{1\times256\times256}$3, applies multi-head self-attention within each window, and then shifts windows by $D\in\mathbb{R}^{1\times256\times256}$4 in the next layer to enable cross-window interaction.

The attention computation is expressed as

$D\in\mathbb{R}^{1\times256\times256}$5

followed by

$D\in\mathbb{R}^{1\times256\times256}$6

where $D\in\mathbb{R}^{1\times256\times256}$7 is a learnable relative-position bias. Hierarchical patch-merging follows every two SW-MSA blocks, merging neighboring $D\in\mathbb{R}^{1\times256\times256}$8 patches, halving spatial resolution and doubling the feature dimension.

The paper reports two model sizes, Swin-Tiny and Swin-Base, both using the same MLP, LayerNorm, and window settings as the original Swin configuration. The navigation-specific modifications are therefore concentrated in the skip-stage interconnection pattern and the integration of Swin-based feature extractors into a perception-to-waypoint-control pipeline, rather than in a redesign of the underlying attention block itself.

This design choice implies a specific division of labor: windowed and shifted-window attention provide the long-range relational modeling, while the skip-stage pathway re-injects the local high-resolution evidence that patch-merging would suppress. The paper’s qualitative description of early detection of lateral vehicles and improved anticipatory braking is consistent with that division of labor, though those behavioral interpretations remain downstream observations rather than direct proofs of mechanism.

4. Block structure, positional handling, and controller outputs

Each transformer block follows a standard residual arrangement: LayerNorm, SW-MSA, residual addition, then LayerNorm, MLP, and a second residual addition (Kartiman et al., 28 Aug 2025). The MLP is specified as a two-layer feed-forward transformation:

$D\in\mathbb{R}^{1\times256\times256}$9

Positional information is not injected through absolute positional embeddings. Instead, it is represented through the relative bias $\mathbb{R}^{2\times256\times256}$0 inside each attention head. This is a technically consequential detail because it aligns the model’s positional encoding strategy with the original Swin formulation while avoiding absolute token-coordinate encodings.

The controller is structurally simple but operationally broad. Over a GRU loop of length 3, it predicts three local waypoints together with steering, throttle, brake, traffic-light logits, and stop-sign logits. The coupling of trajectory outputs with control and traffic-state outputs indicates a multi-task supervision regime in which geometric planning and scene-action interpretation are optimized jointly. A plausible implication is that the controller’s hidden state acts as the locus where BEV scene representation is translated into short-horizon action structure.

5. Training procedure and objective functions

Training is performed by imitation learning, specifically behavior cloning, on CARLA 0.9.10.1 with samples taken every 500 ms (Kartiman et al., 28 Aug 2025). The preprocessing pipeline crops inputs to $\mathbb{R}^{2\times256\times256}$1, and the depth image is decoded by

$\mathbb{R}^{2\times256\times256}$2

The learning objective is multi-task. For segmentation, the reported loss is

$\mathbb{R}^{2\times256\times256}$3

and for traffic-light, stop-sign, steering, throttle, brake, and waypoint prediction, the loss is

$\mathbb{R}^{2\times256\times256}$4

The total objective is

$\mathbb{R}^{2\times256\times256}$5

with adaptive weights $\mathbb{R}^{2\times256\times256}$6 determined by Modified Gradient Normalization.

Optimization uses AdamW with weight decay $\mathbb{R}^{2\times256\times256}$7, initial learning rate $\mathbb{R}^{2\times256\times256}$8, ReduceLROnPlateau with patience 3, early stopping with patience 15, and batch size 8. Taken together, these details indicate a training regime designed to stabilize a heterogeneous loss landscape spanning dense segmentation, regression of vehicle control, and waypoint prediction.

6. Empirical results, adversarial evaluation, and interpretation

The principal quantitative evaluation reported for CARLA Town05 long routes uses Route Completion (RC), Infraction Penalty (IP), and Driving Score (DS) (Kartiman et al., 28 Aug 2025). The x13 CNN baseline is reported at RC $\mathbb{R}^{2\times256\times256}$9, IP $F\in\mathbb{R}^{C\times H\times W}$0, and DS $F\in\mathbb{R}^{C\times H\times W}$1. SKGE-Swin-tiny with stage $F\in\mathbb{R}^{C\times H\times W}$2 skip and float32 inference is reported at RC $F\in\mathbb{R}^{C\times H\times W}$3, IP $F\in\mathbb{R}^{C\times H\times W}$4, and DS $F\in\mathbb{R}^{C\times H\times W}$5, which is the best DS among the listed learned models. The float16 version is reported at RC $F\in\mathbb{R}^{C\times H\times W}$6, IP $F\in\mathbb{R}^{C\times H\times W}$7, and DS $F\in\mathbb{R}^{C\times H\times W}$8. The expert human reference is reported at RC $F\in\mathbb{R}^{C\times H\times W}$9, IP $h_t$0, and DS $h_t$1.

The paper’s own interpretation is that the tiny Swin variant with stage $h_t$2 skip closes roughly two-thirds of the DS gap to expert performance, with DS $h_t$3 versus $h_t$4, and improves over x13 by $h_t$5 DS points. It also interprets the lower RC together with higher IP as evidence of a tradeoff in which route completion decreases modestly while infractions are reduced, indicating safer navigation.

Under adversarial scenarios involving dense traffic, sudden vehicles or pedestrians, and red-light violations, the model is reported to produce 25% fewer collision infractions than x13, 15% higher stopping accuracy at red lights, and an overall DS improvement from $h_t$6 to $h_t$7 under identical test routes. The accompanying qualitative examples are described as showing anticipatory braking at sharp turns, earlier detection of lateral vehicles, and fewer side-swipe collisions than the CNN baseline. These observations are consistent with the paper’s central claim that richer fusion of local and global spatial cues materially affects downstream driving behavior.

7. Position within end-to-end autonomous driving research

Within end-to-end autonomous driving, SKGE-Swin is most precisely understood as a Swin-based waypoint-prediction system that incorporates hierarchical attention, explicit semantic-to-BEV transformation, and skip-stage feature preservation into a unified training objective (Kartiman et al., 28 Aug 2025). It does not merely substitute a transformer for a convolutional backbone; rather, it combines the Swin Transformer’s shifted-window self-attention with a specific cross-stage information pathway intended to counteract the loss of fine detail caused by hierarchical downsampling.

The architecture also clarifies a recurrent distinction in the literature between global context modeling and local geometric fidelity. In SKGE-Swin, global context is attributed to SW-MSA and shifted-window interactions, while local fidelity is attributed to stage $h_t$8 skip propagation. The empirical pattern reported in the ablation summaries and adversarial driving results suggests that both are necessary for the observed gains.

A further implication is methodological rather than purely architectural. Because the system is trained with behavior cloning and supervised multi-task losses, its reported gains are achieved without introducing reinforcement learning into the optimization loop. This places the contribution primarily in representational design, feature fusion, and task coupling, rather than in a new learning paradigm.