---
title: Ban & Pick Post-Training Strategies
url: https://www.emergentmind.com/topics/ban-pick-post-training
type: topic
---

# Ban & Pick Post-Training Strategies

Ban & Pick post-training refers to a class of post hoc methods that achieve performance gains, improved efficiency, or enhanced robustness by automatically *banning* (removing, masking, or downweighting) statistically low-impact or redundant components while *picking* (retaining, emphasizing, or reinforcing) the most informative or impactful ones. These strategies are hypothesis- and data-driven, requiring no retraining from scratch and leveraging only a pretrained model (and, if needed, small calibration datasets or ancillary models). Applications span mixture-of-experts (MoE) routing, preference optimization in language models, prioritization in RL fine-tuning, and overfitting mitigation in deep classifiers.

## 1. Key Conceptual Principles

The Ban & Pick paradigm formalizes a “hard” or “soft” selection at multiple granularities of deep learning systems:

- **Ban:** Disable or deprioritize components (experts, tokens, problems, activations) deemed low-impact, using objective statistics derived from the current model or external references.
- **Pick:** Identify and prioritize components with high optimization or inference impact, based on empirical metrics reflecting utility (alignment, accuracy, learning signal).

By decoupling post-training selection from pretraining or architecture, Ban & Pick strategies offer practical, plug-in improvements to existing models without disrupting their foundational structure or original objectives. These methods generalize over earlier notions of pruning, curriculum learning, and token/activation masking, but optimize the pick/ban decision using model-dependent, often dynamic, informativeness scores [2507.07725, 2509.06346, 2601.02648, 2309.16827].

## 2. Algorithms Across Major Application Domains

The Ban & Pick framework manifests in diverse architectures and settings. Below are canonical instantiations:

### MoE-LLMs (Post-training Routing)
In MoE LLMs, Ban & Pick post-training is used for smarter inference routing [2509.06346]:

- **Pick**: Empirically identifies per-layer, per-domain *key experts* by measuring KL-perturbation on token distributions from calibration data. These are reinforced via “range-based replacement” in the routing policy if nearly selected.
- **Ban**: Dynamically prunes token-expert assignments using sensitivity scores jointly dependent on calibration-based per-layer sensitivity (KL between maximum and minimum expert pruning) and per-token router softmax concentration. The result is a reduced, token- and layer-adaptive expert count $K_{t,l}$, preserving accuracy and improving speed.

Pseudocode for combined Ban & Pick routing is provided in the original publication, which details expert candidate selection, key expert insertion, and expert count adaptation for each token and layer.

### Preference Optimization (LLM Alignment)
Selective-DPO applies Ban & Pick at the token level for preference-based post-training alignment [2507.07725]:

- **Pick**: Selects the top $k\%$ of tokens (in win and lose responses) by absolute log-probability difference with a strong reference model. These high-impact tokens drive the optimization.
- **Ban**: All other tokens are masked (“banned”) from contributing primary gradient signal; optionally, a small KL penalty stabilizes their distributions.
- The DPO objective is then masked, summing only over selected tokens for both responses.

The quality of the reference model directly impacts the reliability of ban/pick decisions and thus alignment gains.

### RL Post-Training Curriculum (GRPO)
In RL post-training for LLMs, Ban & Pick is realized as prioritized problem replay [2601.02648]:

- **Pick**: Problems are scored by the variance of group reward advantages ($\omega_i = s_i (1-s_i)$), peaking at intermediate empirical success rates.
- Samples with high $\omega_i$ are picked (via a max-heap), focusing training on those contributing the most informative policy updates.
- **Ban**: Problems that become fully solved or unsolved (EMA success rate $\bar s_i$ near $1$ or $0$) are moved to auxiliary pools and only retested periodically, mitigating gradient waste, starvation, and forgetting.

### Overfitting Mitigation in DNNs
Ban & Pick is adapted for post-training overfitting mitigation using clipped-ReLU and maximum margin statistics [2309.16827]:

- **Ban**: Replace standard ReLU units with upper-thresholded versions, limiting neuron activations and thus curtailing excessive margins responsible for both non-malicious and malicious (backdoor) overfitting.
- **Pick**: Learn per-layer or per-neuron clip thresholds $Z$ by optimizing a bi-level objective using a small clean holdout set, balancing fidelity on clean data with a penalty on the maximum margin.

## 3. Mathematical Formalism and Optimization Objectives

A hallmark of Ban & Pick post-training is precise, quantitative selection using model-internal or reference-driven statistics. Three canonical examples:

| Domain             | Ban          | Pick                |
|--------------------|--------------|---------------------|
| MoE-LLMs           | Dynamic expert pruning based on $S_{t,l}$ | Key expert reinforcement via KL-perturbation |
| Preference Optim.  | Masking tokens with $|\Delta_t|$ below threshold | Selecting top $k\%$ tokens by $|\Delta_t|$ |
| RL Post-training   | Problems with $\bar s_i$ at extremes removed | Highest-variance problems ($s_i \approx 0.5$) |
| DNN Classifiers    | Replace ReLU with min-clip, ban large activations | Optimize thresholds $Z^*$ on small clean set |

Optimization is situated in either “hard” (set inclusion/exclusion) or “soft” (weighting, thresholding, masking) forms, always informed by a score reflecting informativeness, sensitivity, or reward variance.

## 4. Empirical Validation and Performance Summary

Comprehensive experiments demonstrate the efficacy of Ban & Pick strategies across settings:

### MoE-LLMs
- On Qwen3-30B-A3B, combined Ban & Pick improves accuracy on AIME2024 from 80.67% to 84.66% and on GPQA-Diamond from 65.66% to 68.18, while reducing expert usage from $8.0$ to $\sim4.8$ experts/token and achieving $1.25\times$ inference speedup [2509.06346].
- Pick alone gains up to $+4.66$ points (AIME2024) and Ban alone achieves $1.25\times$ speedup with minimal accuracy drop ($\leq1.5\%$).

### LLM Preference Alignment
- Selective-DPO outperforms standard DPO by more than $+9.1$pp Arena-Hard win-rate at 0.5B scale (with large reference) and by $+1.3$pp at the 3B scale. It also surpasses Distill-DPO on 3B Arena-Hard ($+2.2$pp) and matches/exceeds on MT-Bench [2507.07725].
- Empirically, optimal token selection ratio is $\sim40\%$; strong references yield marked improvements.

### RL Post-Training
- On MATH-500 and AIME-2024, prioritized Ban & Pick yields pass@1 improvements from $7.2\to10.5\%$ and $9.1\to12.3\%$ respectively [2601.02648].
- Ablations show that the banning/pool mechanism and exploration sampling are critical for preventing performance drop due to problem starvation or overfitting.

### DNN Overfitting/Backdoor Mitigation
- On imbalanced CIFAR-10 LT-100, MMOM post-training raises accuracy from $73.36\to80.98\%$. When combined with GCL, performance reaches $82.04\%$ (comparable to the best train-time methods).
- On backdoored nets, Ban & Pick (MMAC/MMDF) reduces attack success rate from $>90\%$ to $<5\%$ with $<1\%$ clean accuracy loss [2309.16827].

## 5. Theoretical and Practical Implications

Ban & Pick approaches demonstrate that post-training interventions, without model retraining or architectural changes, can provide:

- Free accuracy improvements and inference speedups (MoE-LLMs, LLMs).
- Robustness against overfitting from imbalanced data or training artifacts.
- Automatic data/task curricula driven by learning signal variance.
- Resiliency to catastrophic forgetting and starvation phenomena in RL fine-tuning, via adaptive retesting of banned (solved/unsolved) pools.
- Data–model co-adaptation, with ban/pick ratios (e.g., $\lambda$ in MoE, $k$ in token masking) empirically tunable for optimal tradeoff.

A strong reference (in alignment) or sound calibration set (for expert/tuning selection) significantly magnifies impact, underscoring the value of external, high-fidelity guides for pick/ban strategy.

## 6. Design Considerations, Limitations, and Research Directions

- **Reference Model Quality:** In token-level Ban & Pick (e.g., Selective-DPO), weak references produce noisy selection and dilute alignment; strong, well-aligned references sharpens pick/ban impact and gradient focus [2507.07725].
- **Computational Overhead:** All major implementations emphasize minimal runtime overhead—routing adjustments, masking, or heap maintenance are computationally negligible relative to model computation.
- **Granularity and Hyperparameters:** Each setting requires careful choice of selection thresholds ($k$, $\lambda$, $\epsilon$, etc.) and sometimes dual sensitivities (layer, token, problem). Empirical sweeps and ablations are standard practice [2509.06346, 2507.07725, 2601.02648].
- **Universality:** While demonstrated from per-token up to per-problem and per-neuron, the core paradigm is widely applicable, provided a suitable informativeness signal (variance, margin, KL divergence) is available.

A plausible implication is continued evolution of Ban & Pick strategies to jointly optimize multi-granular selections—integrating expert masking, data curriculum, and activation bounding in large-scale, distributed, or multi-modal systems.

---

**Key References:**  
- "Ban&Pick: Achieving Free Performance Gains and Inference Speedup via Smarter Routing in MoE-LLMs" [2509.06346]  
- "Not All Preferences are What You Need for Post-Training: Selective Alignment Strategy for Preference Optimization" [2507.07725]  
- "Prioritized Replay for RL Post-training" [2601.02648]  
- "Post-Training Overfitting Mitigation in DNN Classifiers" [2309.16827]

Source: https://www.emergentmind.com/topics/ban-pick-post-training