Dual-Granularity Dynamic Weighted Scoring

• The paper introduces a unified Feature Map Quality Score (FMQS) that fuses global task-level and local feature metrics for module evaluation.
• It employs a dual-granularity approach with dynamic weighting and contrastive alignment using CLIP-FMQE-Net to supervise and improve model training.
• Empirical results on BEV-based 3D detection with nuScenes demonstrate significant NDS gains, validating the method's impact on interpretability and performance.

The Dual-Granularity Dynamic Weighted Scoring System (DG-DWSS) is a quantitative framework for evaluating and supervising intermediate feature maps in multi-module, end-to-end autonomous vehicle perception architectures. DG-DWSS introduces a fusion of global task-level and local feature-structure similarity metrics, producing a unified Feature Map Quality Score (FMQS) that supports both independent functional evaluation and in-loop training supervision. This methodology was developed and validated in the context of BEV (bird’s-eye view)-based 3D detection using the nuScenes dataset, and has been demonstrated to enhance interpretability and downstream model performance (Zhang et al., 11 Aug 2025).

1. Motivation and Conceptual Rationale

Intermediate feature maps in hierarchical, end-to-end autonomous driving models—such as BEVFormer—lack explicit independent supervision and are typically regarded as “black boxes.” Traditional evaluation pipelines focus either on overall task loss (e.g., 3D detection metrics) or on representation similarity between intermediate feature maps and ground-truth data. However, previous methods do not offer a unified metric for both coarse (task-level) and fine (feature-level) assessment, nor incorporate such metrics as composite losses during model optimization.

DG-DWSS is designed to address this deficiency by providing:

• A unified, quantitative FMQS that integrates task effectiveness with representational structure quality.
• An auxiliary loss derived from FMQS, enabling continuous and module-level functional supervision during end-to-end training.

2. Mathematical Structure of DG-DWSS

DG-DWSS quantifies feature map quality by dynamically combining macro- and micro-level metrics across modules, training stages, and data samples.

2.1 Macro-Level (Coarse-Granularity) Score

At the global scale, task performance is indexed by the normalized 3D detection score:

• For each configuration-stage pair $(i, j)$, the 3D detection metric is $\mathrm{NDS}_{i,j}$.
• Identify the SOTA configuration $(i_{\mathrm{sota}}, j_{\mathrm{sota}})$ with maximal metric:

$$(i_{\mathrm{sota}},j_{\mathrm{sota}}) = \underset{i,j}{\arg\max}\ \mathrm{NDS}_{i,j},$$

with reference score $\mathrm{NDS}_{\mathrm{sota}}$.

• Define the normalized model-level score:

$$\mathrm{Score}_{\mathrm{model}}^{i,j} = \frac{\mathrm{NDS}_{i,j}}{\mathrm{NDS}_{\mathrm{sota}}}$$

2.2 Micro-Level (Fine-Granularity) Score

Local structure is evaluated by comparing each candidate feature map $$\mathcal{F}^{i,j,k} \in \mathbb{R}^{C \times H \times W}$$ to its SOTA counterpart $\mathcal{F}_{\mathrm{sota}}^{i,j,k}$, using Channel–Spatial Cosine Similarity (CS-CosSim):

$$\mathrm{Score}_{\mathrm{Feat}}^{i,j,k} = \alpha\,\mathrm{CosSim}_{\mathrm{ch}}^{i,j,k} + (1-\alpha)\,\mathrm{CosSim}_{\mathrm{sp}}^{i,j,k}$$

with $\alpha = 0.5$.

• Channel-wise similarity:

$\mathrm{NDS}_{i,j}$0

• Spatial similarity:

$\mathrm{NDS}_{i,j}$1

2.3 Unified Fusion via Dynamic Weighting

The final FMQS combines both granularity levels:

$\mathrm{NDS}_{i,j}$2

with $\mathrm{NDS}_{i,j}$3 dynamically set; empirically, $\mathrm{NDS}_{i,j}$4 yields optimal performance.

3. DG-DWSS Architecture and System Integration

DG-DWSS is realized through the CLIP-based Feature Map Quality Evaluation Network (CLIP-FMQE-Net), interfacing with target modules and ground-truth for both evaluation and training supervision.

Component	Function	Input/Output
Feature Map Encoder	Projects $\mathrm{NDS}_{i,j}$5	Lightweight 4-layer CNN
Ground-truth Text Encoder	Maps annotation templates to $\mathrm{NDS}_{i,j}$6	CLIP text transformer
Contrastive Alignment Head	Aligns feature/ground-truth encodings	Symmetric contrastive loss
FMQS Prediction Head	Regresses FMQS	Transformer decoder

Feature maps from each module (e.g., IFEM, BFEM) are embedded into a 512-dimensional space. Structured annotation templates are encoded using CLIP’s text transformer. Contrastive learning between map and text embeddings guides alignment, while a regression head predicts FMQS for downstream loss computation.

4. Training Objectives and Loss Design

The DG-DWSS framework introduces several loss terms for representation alignment and functional supervision.

• Contrastive Alignment Loss:

$\mathrm{NDS}_{i,j}$7

with $\mathrm{NDS}_{i,j}$8.

• FMQS Regression Loss:

$\mathrm{NDS}_{i,j}$9

• Auxiliary Supervision Loss in Downstream Model:

For each feature module $(i_{\mathrm{sota}}, j_{\mathrm{sota}})$0 (IFEM, BFEM):

$(i_{\mathrm{sota}}, j_{\mathrm{sota}})$1

The total perception loss combines task and auxiliary terms:

$(i_{\mathrm{sota}}, j_{\mathrm{sota}})$2

This structure closes the evaluation-train loop: CLIP-FMQE-Net is pretrained and frozen, FMQS predictions supervise intermediate modules during BEVFormer training.

5. Implementation Protocol and Workflow

The computation and integration protocol for DG-DWSS, as applied to perception training or inference, consists of the following loop:

$(i_{\mathrm{sota}}, j_{\mathrm{sota}})$6

This scheme enables real-time evaluation and training supervision, leveraging both precomputed task-level references and dynamic, sample-level feature metrics.

6. Empirical Evaluation and Results

DG-DWSS was evaluated using nuScenes-mini (404 keyframes, 80/20 split) over a set of eight module configurations and eight training stages. The SOTA reference (VoV-SCA-RCF, stage 7) achieved $(i_{\mathrm{sota}}, j_{\mathrm{sota}})$3.

• FMQS regression performance:
  • IFEM: MSE = 0.0041; MAE ≈ 0.049; $(i_{\mathrm{sota}}, j_{\mathrm{sota}})$4; MAPE ≈ 7.4%
  • BFEM: MSE = 0.0055; MAE ≈ 0.058; $(i_{\mathrm{sota}}, j_{\mathrm{sota}})$5; MAPE ≈ 8.9%
• BEVFormer integration:
  • Using IFEM or BFEM auxiliary loss yields relative NDS improvements of approximately 5–6%.
  • Dual (IFEM + BFEM) auxiliary loss produces an average NDS gain of 3.89%, peaking at 7.36% (Res50-SCA-RCF).
• Ablation studies confirm complementary benefits and maximal gains when applying dual-granularity supervision to both IFEM and BFEM modules.

These results establish the reliability of DG-DWSS for module evaluation and as a functional loss shaping feature representations towards greater task effectiveness and structural similarity.

7. Generalization and Impact

DG-DWSS provides a principled and extensible approach for bridging the divide between task-level and feature-level supervision in deep autonomous driving architectures. While developed in the context of BEV-based 3D detection on nuScenes, the dual-granularity scoring and supervisory paradigm is applicable to:

• Pipeline architectures for depth estimation and semantic segmentation.
• End-to-end planning modules where latent representational consistency is desirable.
• Multi-sensor fusion systems where a composite of global-metric and local-structure scores advances modality alignment.

DG-DWSS enables quantitative demystification of intermediate networks, supports interpretability, and enhances overall system performance by providing continuous, module-level diagnostic and optimization signals (Zhang et al., 11 Aug 2025). A plausible implication is that this framework may accelerate adoption and reliability assessment in broader safety-critical, multi-module AI systems.