Soft Voting CNN Ensembles

• Soft voting ensembles of CNNs are defined as frameworks that combine independent class probability outputs via weighted averaging to yield a consensus prediction.
• They utilize modular, independent training of diverse CNN architectures with standardized preprocessing and augmentation to enhance generalization.
• Empirical results show consistent gains in accuracy, recall, and class-balance across applications, with advanced weighting and meta-learning strategies mitigating limitations.

A soft voting ensemble of Convolutional Neural Networks (CNNs) is an architecture-agnostic framework wherein multiple trained CNNs independently generate class-probability distributions for an input, and those probability vectors are fused—typically via averaging or weighted summation—to obtain a final prediction. This aggregation leverages architectural diversity and variance reduction while maintaining low implementation complexity, achieving consistent gains in accuracy, recall, and class-balance across natural, medical, and industrial classification domains.

1. Mathematical Formulation of Soft Voting Ensembles

A soft voting ensemble combines the class posterior probabilities produced by $M$ independently trained CNNs, each denoted $p_j(x) \in \mathbb{R}^K$ for input $x$ and $K$ classes. The canonical formulation is:

$p_{\rm ens}(x) = \sum_{j=1}^M w_j\, p_j(x)$

where $w_j \ge 0$ and $\sum_{j=1}^M w_j = 1$. The final predicted label is given by

$\hat{y} = \arg\max_k [p_{\rm ens}(x)]_k$

Most reported implementations use equal weighting $w_j=1/M$ for all $j$; however, weighting schemes proportional to validation accuracy or learned via held-out optimization are also effective (Ju et al., 2017, Shafi et al., 23 Dec 2025). The decision boundary thus reflects a consensus over the smoothed probability simplex, integrating multiple models' confidence.

2. Construction and Training Protocols

The workflow for soft voting ensembles is modular:

• Model Independence: Each CNN is trained separately, preserving architectural heterogeneity and preventing information leakage (Bashar et al., 12 Apr 2025).
• Data Preparation: Standard preprocessing includes input resizing to model-specific dimensions, normalization (e.g., ImageNet statistics), and augmentation (random flips, rotations, brightness/contrast jitter) to enhance generalization (Farooq et al., 2023, Shafi et al., 23 Dec 2025).
• Model Selection: Ensembles may comprise established image-classification backbones (e.g., XceptionNet, DenseNet, EfficientNet, SENet, ConvNeXt, InceptionV3, VGG19, MobileNetV2), with no fusion of intermediate activations or shared parameterization (Bashar et al., 12 Apr 2025, Shafi et al., 23 Dec 2025).
• Loss and Optimization: Each network is trained with task-specific loss (categorical cross-entropy for multiclass problems), with optimizers such as Adam or SGD and model selection based on lowest validation loss (Farooq et al., 2023, Shafi et al., 23 Dec 2025).

Architectural diversity and independent training are crucial for maximizing complementary error profiles and achieving the characteristic ensemble gains.

3. Inference Mechanism and Implementation Details

At inference, the ensemble operates according to the following pipeline:

1. Forward Pass: Each CNN generates pre-softmax logits for the input $x$.
2. Probability Conversion: Logits are transformed to probability vectors via softmax:

$$p_j^k(x) = \frac{\exp(s_j^k(x))}{\sum_{k'=1}^K \exp(s_j^{k'}(x))}$$

where $s_j^k(x)$ is the pre-softmax score of model $j$ for class $k$ (Ju et al., 2017, Bashar et al., 12 Apr 2025).

1. Soft Voting Fusion: The ensemble probability vector is computed as a weighted sum over the models. In the equal weighting scenario, this reduces to a simple average.
2. Decision Rule: The predicted label is the index of maximum probability in the fused vector.
3. Performance Metrics: Ensemble accuracy, precision, recall, F1-score, and AUC are compared to individual model baselines (Farooq et al., 2023, Shafi et al., 23 Dec 2025).

A representative pseudocode block—mirroring standard practice—is:

for i in range(num_samples): ensembled_probs = sum(w[m] * softmax(model[m](x_i)) for m in range(M)) y_hat[i] = argmax(ensembled_probs) ensemble_accuracy = (y_hat == y_true).mean()

(Bashar et al., 12 Apr 2025)

4. Empirical Performance and Comparative Results

Soft voting CNN ensembles reproducibly yield measurable improvements across a range of datasets and application domains.

Task / Dataset	Best Single	Soft Voting Acc.	Absolute Gain	Reference
Breast tumor classification	83.33 %	88.33 %	+5.0 %	(Farooq et al., 2023)
ImageNet (3 heterogeneous CNNs)	81.37 %	83.79 %	+2.42 %	(Bashar et al., 12 Apr 2025)
Skin lesion (HAM10000)	94.0 %	96.32 %	+2.3 %	(Shafi et al., 23 Dec 2025)
Mode inference (best CNN)	81.3 %	85.0 %	+3.7 %	(Yazdizadeh et al., 2019)

Performance gains are most pronounced when base models are strong and diverse. Recall, especially for under-represented classes, and AUC also improve. In highly imbalanced tasks, gains may be limited by insufficient data for rare categories, suggesting the need for complementary strategies such as data augmentation or cost-sensitive loss functions (Syarubany, 29 Sep 2025).

5. Weighting, Calibration, and Limitations

Soft voting can be implemented with uniform or nonuniform model weights:

• Uniform weighting is canonical and trivial to execute but may underperform in the presence of weak or miscalibrated base learners (Ju et al., 2017, Yazdizadeh et al., 2019).
• Weighted Voting: Weights proportional to validation accuracy or determined by convex optimization over a held-out set (as in the Super Learner approach) suppress over-confident or unreliable contributors, raising overall accuracy and stability (Ju et al., 2017, Shafi et al., 23 Dec 2025).

Limitations include:

• Sensitivity to Outliers: Over-confident or poorly performing models can dominate the ensemble and reduce accuracy, particularly in heterogeneous libraries (Ju et al., 2017).
• Inefficiency with Large Libraries: Blind averaging over many weak or partially redundant models can dilute ensemble power (Yazdizadeh et al., 2019).
• No Joint Training: Since models are trained independently, there is no explicit decorrelation or joint optimization of ensemble diversity.

Practical consensus is that learned weighting or meta-learning (via post-hoc regression or random forest meta-learners) consistently outperforms naive averaging, especially as library diversity increases (Ju et al., 2017, Yazdizadeh et al., 2019).

6. Application-Specific Considerations

Soft voting CNN ensembles have demonstrated state-of-the-art or near state-of-the-art accuracy in medical imaging, transportation mode inference, vehicle recognition, and general visual classification:

• Medical Imaging: Soft-voting CNN ensembles on mammography (Farooq et al., 2023) and dermatology (Shafi et al., 23 Dec 2025) improve detection sensitivity and class-balance, achieve +2–5% over strong baselines, and maintain real-time inference speeds suitable for clinical deployment.
• Industrial and Transportation Tasks: Even with severe class imbalance, ensemble methods on CNN-extracted features (used for Random Forest or AdaBoost) maintain robust class-wise accuracy for common classes, although rare class detection remains a challenge unless explicit rebalancing or cost-sensitive losses are introduced (Syarubany, 29 Sep 2025).
• Large-Scale Image Classification: On ImageNet, three-CNN soft voting with model heterogeneity consistently outperforms single networks by 2–3% at lower inference latency than transformer-based backbones (Bashar et al., 12 Apr 2025).

A plausible implication is that domain-specific preprocessing (segmentation, augmentation) paired with soft voting can yield high accuracy with manageable computational overhead, supporting deployment in latency-sensitive scenarios.

7. Recommendations and Future Directions

• Model Selection: Combine diverse, high-performing CNN architectures to maximize complementarity (Bashar et al., 12 Apr 2025).
• Weighting: Where possible, allocate model weights via validation-based optimization. Uniform weighting remains beneficial when constituent models are comparably calibrated and strong (Shafi et al., 23 Dec 2025).
• Calibration: Consider probability calibration post-processing (e.g., temperature scaling) especially in risk-sensitive contexts or when combining heterogeneous learners (Ju et al., 2017, Yazdizadeh et al., 2019).
• Meta-learner Extension: For large libraries, employ meta-learning (e.g., random forest or convex combination over base outputs) to optimize the ensemble’s discriminative power (Yazdizadeh et al., 2019).
• Class Imbalance: Pair soft voting with targeted rebalancing (SMOTE, cost-sensitive loss) to improve minority class performance (Syarubany, 29 Sep 2025, Shafi et al., 23 Dec 2025).
• Deployment: Maintain model independence for modular training and facilitate model updating or replacement with minimal cross-coupling (Bashar et al., 12 Apr 2025).

The empirical consensus is that soft voting ensembles deliver robust, modular, and easily optimized improvements for a wide spectrum of image classification challenges, but their ultimate efficacy is bounded by the quality and diversity of the base CNNs and the calibration of their confidence estimates. Weighting schemes and meta-learners offer systematic means to further enhance performance in heterogeneous or large-scale settings.