Token-Guided 2D Patch Distillation (TGPD)

• The paper introduces a targeted distillation technique that aligns multimodal BEV features through patch tokenization and attention mechanisms.
• TGPD decomposes dense BEV maps into local patch tokens and a global token, using Transformer-style self-attention for focused cross-modal supervision.
• The method significantly accelerates student convergence and improves mIoU and mAP in online HD map construction scenarios.

Token-Guided 2D Patch Distillation (TGPD) is a targeted knowledge distillation strategy for bird’s eye view (BEV) feature alignment introduced in MapKD for efficient online HD map construction (Yan et al., 21 Aug 2025). Within MapKD’s Teacher-Coach-Student (TCS) paradigm, a camera-LiDAR fusion model with SD/HD map priors serves as the teacher, a vision-centric coach model with prior knowledge and simulated LiDAR bridges the cross-modal knowledge transfer gap, and a lightweight vision-based student is optimized for low-cost inference. TGPD addresses the mismatch between the teacher’s multimodal, prior-enriched BEV representations and the student’s camera-only BEV representations by decomposing BEV feature maps into local patch tokens and a global token, applying Transformer-style self-attention to these tokens, and supervising the student through a KL-based alignment of token-attention distributions together with a light MSE penalty on patch features (Yan et al., 21 Aug 2025).

1. Role within cross-modal BEV distillation

TGPD is motivated by a specific difficulty in cross-modal knowledge distillation for BEV-map construction: the teacher’s BEV features, fused from LiDAR, camera, and map priors, and the student’s BEV features, derived from cameras only, live in quite different representation spaces (Yan et al., 21 Aug 2025). The method is designed as a response to two limitations stated explicitly in the MapKD description. First, vanilla feature-level distillation, such as MSE on whole feature maps, often forces a low-capacity student to regress dense LiDAR-rich patterns it cannot synthesize from images, leading to slow convergence or suboptimal minima. Second, response-level distillation, such as matching logits, cannot fully convey the rich spatial structures embedded in intermediate BEV representations (Yan et al., 21 Aug 2025).

A common misconception in this setting is that denser supervision is necessarily better. TGPD is framed against that assumption. Rather than flooding the student with unnecessary signal over the full BEV tensor, it reduces the regression burden to compact local subregions and shifts emphasis toward where the teacher focuses. Another misconception is that logit matching alone is sufficient for transferring BEV structure. The formulation of TGPD explicitly rejects that view by treating the attention structure among local regions as a distillation target in its own right (Yan et al., 21 Aug 2025).

2. Tokenization of BEV feature maps

TGPD operates on intermediate BEV feature maps from teacher and student. Let the teacher’s intermediate BEV feature map be $F_T \in \mathbb{R}^{C \times H \times W}$ and the student’s BEV feature map be $F_S \in \mathbb{R}^{C \times H \times W}$. In the reported instantiation on nuScenes with BEV range $60 \times 30$ m and voxel resolution $0.15$ m, $(H, W) \simeq (400, 200)$ and $C = 256$ (Yan et al., 21 Aug 2025).

The first step is 2D patch extraction. A BEV feature map $F$ is partitioned into non-overlapping $p \times p$ cells, and the $i$th patch is defined as

$$P_i(F) = \mathrm{flatten}\!\left(F[:, (i_h \cdot p):(i_h \cdot p + p - 1), (i_w \cdot p):(i_w \cdot p + p - 1)]\right) \in \mathbb{R}^{p^2 \cdot C},$$

where $F_S \in \mathbb{R}^{C \times H \times W}$0 indexes the patch and the total patch count is

$F_S \in \mathbb{R}^{C \times H \times W}$1

Each patch vector is then linearly projected into a token embedding:

$F_S \in \mathbb{R}^{C \times H \times W}$2

TGPD also constructs a scene-level global token. The spatial average of $F_S \in \mathbb{R}^{C \times H \times W}$3 is

$F_S \in \mathbb{R}^{C \times H \times W}$4

which is projected as

$F_S \in \mathbb{R}^{C \times H \times W}$5

For each model, the token sequence is therefore

$F_S \in \mathbb{R}^{C \times H \times W}$6

This tokenization step is central to TGPD’s design because it converts dense BEV tensors into a structured set of local and global descriptors suitable for relational supervision (Yan et al., 21 Aug 2025).

3. Attention-guided patch selection

The distinctive mechanism in TGPD is its use of attention distributions as soft patch selectors. On each token set, TGPD applies one Transformer-style attention layer:

$F_S \in \mathbb{R}^{C \times H \times W}$7

and

$F_S \in \mathbb{R}^{C \times H \times W}$8

where $F_S \in \mathbb{R}^{C \times H \times W}$9 is the temperature (Yan et al., 21 Aug 2025).

The attention matrix $60 \times 30$0 encodes how each patch token and the global token attend to all others. TGPD treats these attention distributions as indicators of importance in the BEV representation. In the wording of the paper, the central insight is to use the attention distributions themselves as “soft” selectors that highlight which patches the teacher deems important in the BEV. Distillation therefore targets not only local feature values but also the pattern of inter-patch attentions (Yan et al., 21 Aug 2025).

This design has two immediate consequences. First, it preserves long-range dependencies, including relations such as object-to-lane relations, because every token can attend to every other token. Second, it turns supervision into a focus-aware signal: the student is encouraged to reproduce the teacher’s allocation of attention rather than merely its raw activation magnitudes. A plausible implication is that this can better tolerate modality gaps than direct full-map regression, because the supervisory signal is organized around relational saliency rather than exhaustive pixelwise equivalence.

4. Loss formulation

TGPD combines two terms: a KL-divergence term on token-attention and an MSE term on local patch features. Let $60 \times 30$1 and $60 \times 30$2 be the attention matrices for teacher and student, respectively, after temperature scaling. The KL term

$60 \times 30$3

penalizes discrepancies in where the student “looks” in its BEV representation (Yan et al., 21 Aug 2025).

Using the notation from the paper, with $60 \times 30$4 as the patch-extraction operator, $60 \times 30$5 as the global-average operator, $60 \times 30$6, $60 \times 30$7 as learned projections, and $60 \times 30$8 as the weighting factor on the MSE patch-feature term, the teacher-to-student TGPD loss is

$60 \times 30$9

The coach-to-student branch has the analogous form:

$0.15$0

The overall TGPD objective is

$0.15$1

where $0.15$2 and $0.15$3 are the weights for the teacher-to-student and coach-to-student branches (Yan et al., 21 Aug 2025).

This formulation makes TGPD a multi-branch distillation mechanism inside the broader TCS framework. Factually, it aligns the student simultaneously to the teacher and the coach; contextually, this means that the coach functions as an intermediate source of supervision that reduces the cross-modal transfer gap.

5. Instantiation and implementation details

The reported implementation uses patch size $0.15$4, token embedding dimension $0.15$5, temperature $0.15$6, MSE weight $0.15$7, and TGPD weights $0.15$8, $0.15$9, with the latter selected via ablation (Yan et al., 21 Aug 2025). For $(H, W) \simeq (400, 200)$0 and $(H, W) \simeq (400, 200)$1, this yields approximately

$(H, W) \simeq (400, 200)$2

patches.

The attention module is implemented as a single-layer multi-head attention block with $(H, W) \simeq (400, 200)$3 heads, and $(H, W) \simeq (400, 200)$4 is shared across teacher, coach, and student. All TGPD losses are computed only during training. The reported computational overhead is that tokenization plus one-layer attention adds $(H, W) \simeq (400, 200)$5 training time and zero cost at inference, since TGPD is used only during training (Yan et al., 21 Aug 2025).

The paper also reports practical observations about optimization. TGPD generally accelerates student convergence, with the student reaching $(H, W) \simeq (400, 200)$6 mIoU in $(H, W) \simeq (400, 200)$7 epochs versus $(H, W) \simeq (400, 200)$8 for the baseline. A warm-up of $(H, W) \simeq (400, 200)$9 from $C = 256$0 target over the first $C = 256$1 epochs stabilizes training. For transfer to other domains, the description recommends always including a global token to capture scene-level context, tuning $C = 256$2 between $C = 256$3 and $C = 256$4, and, when distilling from very high-capacity teachers, considering $C = 256$5 to avoid over-constraining local features (Yan et al., 21 Aug 2025). These are reported as practical tips rather than universal guarantees.

6. Empirical behavior on nuScenes

The reported quantitative evidence is given on nuScenes for a camera-only HDMapNet student with a 30-epoch baseline of $C = 256$6 mIoU and $C = 256$7 mAP (Yan et al., 21 Aug 2025). TGPD alone improves this baseline by $C = 256$8 mIoU and $C = 256$9 mAP, reaching $F$0 mIoU and $F$1 mAP. Combining TGPD with MSRD, even without the coach, yields $F$2 mIoU and $F$3 mAP. Within the full Teacher-Coach-Student framework, adding TGPD alone yields $F$4 mIoU and $F$5 mAP (Yan et al., 21 Aug 2025).

Setting	mIoU	mAP
Camera-only HDMapNet student, 30-epoch baseline	31.16	23.13
TGPD alone	36.00	31.36
TGPD + MSRD, without the coach	37.40	31.90
Full TCS framework, adding TGPD alone	37.73	32.29

These results are presented in the source as evidence that TGPD is by far the single most effective component for aligning cross-modal BEV features (Yan et al., 21 Aug 2025). Interpreted conservatively, the pattern suggests that attention-guided local alignment accounts for a substantial fraction of the student’s recovery of teacher-side performance, and that the gain is not confined to one particular supervision path: it remains strong both in direct teacher-student distillation and in the full TCS setting.

7. Significance within MapKD

In MapKD as a whole, the student model is improved by $F$6 mIoU and $F$7 mAP while simultaneously accelerating inference speed (Yan et al., 21 Aug 2025). TGPD is one of the two targeted distillation strategies introduced to support that outcome, the other being Masked Semantic Response Distillation (MSRD). TGPD’s specific function is BEV feature alignment; MSRD is described as semantic learning guidance. The distinction matters because TGPD addresses intermediate relational structure rather than terminal semantic responses alone (Yan et al., 21 Aug 2025).

Conceptually, TGPD occupies a specific niche within distillation for autonomous driving. It does not replace multimodal teachers, prior knowledge, or coach-based transfer; rather, it provides a mechanism for making their BEV representations legible to a vision-centric student. Its structure—patch tokenization, a global token, self-attention, KL alignment of attention matrices, and a light patchwise MSE term—defines a focused recipe for cross-modal BEV distillation. A plausible implication is that the method is relevant beyond the original MapKD setting wherever the teacher contains sensing modalities or priors unavailable to the student but the transfer target remains a spatially organized BEV representation (Yan et al., 21 Aug 2025).