Feature Map Convergence Score (FMCS)

• FMCS is a quantitative metric that assesses the convergence and maturity of intermediate feature maps in deep neural networks.
• It computes discrete scores by segmenting the training loss trajectory using checkpoint selection and loss smoothing techniques.
• FMCS integrates as an auxiliary loss for real-time module diagnostics, boosting performance in applications like image classification and autonomous driving.

The Feature Map Convergence Score (FMCS) is a quantitative metric designed to assess the maturity and convergence quality of intermediate feature maps in deep neural networks, particularly within modular autonomous driving and image classification pipelines. FMCS enables explicit, module-level evaluation independent from end-to-end task supervision, addressing longstanding interpretability and optimization challenges posed by “black-box” deep models. Recent extensions under the umbrella of Feature Map Quality Scoring (FMQS) generalize the principle to encompass both functional convergence and feature representation similarity, supporting robust training and real-time analysis of perception models (Zhang et al., 11 Aug 2025, Zhang et al., 2024, Zhu et al., 8 Aug 2025).

1. Rationale and Motivation

Traditional deep networks optimize global task performance via end-to-end loss, obscuring the development and adequacy of individual modules (e.g., backbone, fusion layer). This obfuscation hampers diagnosis and targeted improvement, as it is unclear whether underperformance arises from architectural design, inadequate training, or the immaturity of specific feature representations (Zhang et al., 2024). Since modules exchange information through feature maps, measuring the “convergence” of these maps—i.e., how far a module’s current output resembles that of a fully trained or “mature” module—provides actionable, interpretable diagnostics for both research and practical deployment. FMCS facilitates this decoupled assessment by assigning standardized, phase-resolved convergence scores to intermediate feature maps.

2. Formal Definition and Calculation of FMCS

FMCS assigns each feature map a discrete convergence index $k \in \{1, \dots, K\}$ indicating its position relative to a reference convergence trajectory derived from the model’s loss curve. The computation involves the following sequence (Zhang et al., 2024, Zhu et al., 8 Aug 2025):

1. Checkpoint Selection: During standard training, select $K$ checkpoints (epochs), $\{c_1, \ldots, c_K\}$, linearly spanning the full trajectory from initialization ($E_0$) to convergence ($E_K$).
2. Loss Smoothing and Segmentation: Record the per-epoch training loss $L_m$, smooth via exponential filtering,

$$\widetilde L_m = \alpha L_{m-1} + (1-\alpha) L_m,\quad \alpha \in (0,1],$$

and compute the Convergence Quantification Indicator (CQI), the moving average of absolute loss changes,

$$CQI_m = \frac{1}{B}\sum_{b=1}^B | \widetilde L_{m-b+1} - \widetilde L_{m-b} |,$$

to pinpoint $E_K$ once $CQI_m$ drops below a threshold $K$0.

1. Log-Loss Trajectory: Define total log-loss drop $K$1 and partition into $K$2 uniform intervals: $K$3.
2. Score Assignment: For each target phase $K$4, select the nearest checkpoint $K$5 where $K$6, labeling all feature maps at $K$7 with FMCS $K$8.

This process produces a labeled dataset (FMCS-Dataset) consisting of $K$9 feature maps paired with convergence-phase labels.

3. FMCS Networks: Evaluation and Prediction

The original FMCE-Net and its later incarnations (including CLIP-FMQE-Net and FMCE-Net++) are lightweight convolutional (or hybrid) networks trained to predict FMCS from individual feature maps (Zhang et al., 2024, Zhu et al., 8 Aug 2025, Zhang et al., 11 Aug 2025). Typical architecture and usage include:

• Input: A module’s feature-map tensor (e.g., $\{c_1, \ldots, c_K\}$0).
• Conv Encoder: Multiple convolution–ReLU–pooling stages project maps to lower-dimensional embeddings.
• Prediction Head: For FMCE-Net, a fully-connected head outputs $\{c_1, \ldots, c_K\}$1 logits for FMCS-classification. In FMQE-Net, a regressor yields a real-valued FMQS/FMQS estimate.
• Supervision: Cross-entropy loss for discrete FMCS prediction; mean squared error for real-valued FMQS regression.
• Integration: FMCE-Net can be used for real-time maturity monitoring; CLIP-FMQE-Net extends to cross-modal (feature ↔ text) quality alignment and score prediction in autonomous driving (Zhang et al., 11 Aug 2025).

Network	Input	Output	Supervision
FMCE-Net	Feature map	FMCS class	Cross-entropy
CLIP-FMQE-Net	Feature map, Text	FMQS (scalar)	MSE, contrastive
FMCE-Net++	Feature map	FMCS class	Cross-entropy

Performance metrics obtained in (Zhang et al., 2024) show FMCE-Net achieves $\{c_1, \ldots, c_K\}$299% FMCS classification accuracy on MNIST/Mini-ImageNet and $\{c_1, \ldots, c_K\}$392–94% on CIFAR-10.

4. FMQS: Dual-Granularity Quality Scoring

Recent advancements reformulate the core convergence principle into a unified Feature Map Quality Score (FMQS) via a dual-granularity dynamic weighted scoring system (DG-DWSS) (Zhang et al., 11 Aug 2025). FMQS fuses two components:

1. Model-Level Score (Macro):

$\{c_1, \ldots, c_K\}$4

where $\{c_1, \ldots, c_K\}$5 is the NuScenes detection score for module $\{c_1, \ldots, c_K\}$6 at stage $\{c_1, \ldots, c_K\}$7.

2. Feature-Level Score (Micro): Weighted channel–spatial cosine similarity between module feature map $\{c_1, \ldots, c_K\}$8 and SOTA reference $\{c_1, \ldots, c_K\}$9:

$E_0$0

(with $E_0$1), where channel similarity averages cosine over channels and spatial over $E_0$2 locations.

3. Fusion:

$E_0$3

(with $E_0$4 in typical experiments).

The FMQS can be efficiently predicted in real-time by CLIP-FMQE-Net, supporting both global (semantic) and local (structural) feature evaluation.

5. Training Integration and Representation Losses

FMCS/FMQS-based supervision is injected into model training by freezing the prediction network (FMCE-Net or FMQE-Net) and incorporating its outputs as auxiliary losses. In FMCE-Net++ (Zhu et al., 8 Aug 2025), the Representation Auxiliary Loss (RAL) balances task classification loss and FMCS convergence loss using a Representation Abstraction Factor $E_0$5: $E_0$6 where $E_0$7 denotes backbone parameters and $E_0$8 classification head. Here, $E_0$9 encourages all representations to resemble those of the most mature phase ($E_K$0). Empirical results show optimal performance when $E_K$1, with accuracy gains up to $E_K$2 pp on CIFAR-10 and substantial gains for additional architectures and datasets.

In autonomous driving (e.g. BEVFormer), analogous auxiliary losses based on FMQS are added for each module, demonstrating up to $E_K$3\% absolute gain in detection scores (NDS) on NuScenes upon dual-module integration (Zhang et al., 11 Aug 2025).

6. Experimental Validation and Practical Impact

FMCS and its derivatives have been validated on a wide range of benchmarks:

• Image Classification (MNIST, CIFAR-10, Mini-ImageNet, FashionMNIST, CIFAR-100): FMCE-Net achieves $E_K$4 score prediction on “easy” datasets (MNIST, Mini-ImageNet) and $E_K$5 on CIFAR-10, using ResNet-50 and ShuffleNet v2 backbones (Zhang et al., 2024, Zhu et al., 8 Aug 2025).
• Autonomous Driving (NuScenes): Integrating FMQS-driven supervision into BEVFormer via CLIP-FMQE-Net yields significant NDS improvements, up to $E_K$6\% absolute when supervising both IFEM and BFEM modules, with regression $E_K$7 up to $E_K$8 and MAPE below $E_K$9 (Zhang et al., 11 Aug 2025).

Performance gains do not require architecture changes or additional data; improvement arises purely from convergence-guided feature shaping.

7. Strengths, Limitations, and Extensions

Strengths:

• Provides the first independent quantitative metric for module-level convergence.
• Enables real-time monitoring and interpretable diagnosis of feature maturity in deep pipelines.
• Supports plug-and-play integration into existing architectures.
• Demonstrated gains on standard benchmarks without compromising primary task performance.

Limitations:

• Baseline FMCS (as in (Zhang et al., 2024, Zhu et al., 8 Aug 2025)) is currently proven only on final-backbone feature maps in classification settings, not for early-stage or arbitrary intermediate modules.
• Requires parameter tuning (choice of $L_m$0, $L_m$1, $L_m$2) per dataset/task.
• Does not directly assess inter-module or compositional information flow.

Extensions and Open Directions:

• Application of FMCS/FMQS paradigms to more complex autonomous driving stacks and mid-level modules.
• Automated meta-optimization of phase partitioning and thresholds.
• Enrichment of similarity metrics (e.g., topological, cross-layer entropy) for more granular quality evaluation.
• Deeper integration with decoupled functional modularization in large-scale perception-and-planning systems (Zhang et al., 11 Aug 2025, Zhang et al., 2024, Zhu et al., 8 Aug 2025).