Dynamic Gated Stacking in Vulnerability Detection

• Dynamic Gated Stacking is an ensemble method that augments base-model predictions with code features, enabling instance-specific weighting for improved vulnerability detection.
• It applies a dynamic gating mechanism inspired by mixture-of-experts to adjust model contributions based on the semantic context of the input code.
• Empirical results show that DGS notably enhances performance on imbalanced and multi-class datasets compared to traditional stacking methods.

Searching arXiv for the specified paper and closely related work on stacking/dynamic ensemble methods. Dynamic Gated Stacking (DGS) is an ensemble learning method for code vulnerability detection in which a stacking meta-model is conditioned not only on the predictions of multiple base models but also on features extracted from the input code itself. In the formulation introduced for LLM-based vulnerability detection, DGS is described as a stacking variant inspired by Mixture of Experts (MoE): the meta-model dynamically gates or weights base-model contributions according to the characteristics of each code instance, rather than relying exclusively on prediction vectors as in traditional stacking (Sun et al., 16 Sep 2025).

1. Definition and motivation

DGS was proposed in the context of empirical evaluation of ensemble methods for LLM-based source code vulnerability detection. The underlying motivation is that vulnerability detection with LLMs exhibits prediction inconsistency across model architectures and training epochs, yet those discrepancies reveal complementary strengths that can be exploited through ensembling. Traditional ensemble methods such as Bagging, Boosting, and Stacking can improve robustness, but standard Stacking is limited because its meta-model uses only base-model predictions as inputs. For difficult settings, especially class-imbalanced datasets and multi-class outputs, that limitation can degrade performance (Sun et al., 16 Sep 2025).

Within this setting, DGS is presented as a novel ensemble stacking variant specifically designed to overcome that restriction. Its defining innovation is the augmentation of the meta-model input: instead of learning only from base-model outputs, the meta-model also receives code features. This makes the final aggregation code-aware and instance-dependent. The designation “dynamic gated” refers to the fact that the meta-model learns to select or blend experts dynamically, in a manner analogous to routing in MoE architectures (Sun et al., 16 Sep 2025).

A common misconception is to treat DGS as ordinary stacking with a larger classifier on top. The distinguishing property is not merely increased model capacity; it is the explicit inclusion of code-derived features in the meta-learner so that model weighting can vary with the input program context.

2. Core formulation

The paper contrasts DGS with traditional stacking through a direct change in the meta-model’s input space. In conventional stacking, base models $f_1, f_2, \dots, f_M$ produce predictions $\phi_m(x) = f_m(x)$, and the meta-model $g$ is trained on the prediction vector: $\hat{y} = g(\phi_1(x), \dots, \phi_M(x)).$

DGS replaces that input with a concatenation of code features and model predictions: $$\text{input} = \mathrm{Concat}(\text{Code Features}(x),\, \phi_1(x), ..., \phi_M(x)).$$

This formulation preserves the stacked ensemble structure while changing what the second-level learner can condition on. The meta-model is therefore trained to select the most suitable base model, or mixture of base models, for each code instance on the basis of both the semantic or structural representation of the code and the base-model outputs. The paper explicitly characterizes this as a dynamic gating mechanism inspired by MoE, with the gate depending on the code input rather than being static across samples (Sun et al., 16 Sep 2025).

The resulting perspective is that DGS is a context-conditioned stacker. This suggests that its main advantage should arise precisely when the informativeness of different base models varies sharply across samples, vulnerability categories, or class priors.

3. Training and inference workflow

The reported DGS workflow consists of three stages during training and a corresponding inference procedure. First, multiple LLMs are trained or fine-tuned for code vulnerability detection. The study uses heterogeneous architectures, though the formulation also allows models from different training stages of the same architecture. Second, for each code instance $x$, a feature embedding is extracted from the code; the examples given are hidden states from an LLM and syntactic or semantic features. Third, the meta-model is trained on the joint input formed by concatenating the code feature with the base-model predictions, using the ground-truth vulnerability label as the target (Sun et al., 16 Sep 2025).

At inference time, a new code snippet is passed through all base models to obtain their predictions, and its code feature is extracted. The meta-model then outputs the final class label by dynamically integrating that information. The paper’s description emphasizes that the final decision is not a fixed averaging or voting rule, but an input-dependent combination of experts.

The empirical study instantiates this framework with five LLMs: DeepSeek-Coder-6.7B, CodeLlama-7B, CodeLlama-13B, CodeQwen1.5-7B, and StarCoder2-15B. These are evaluated on three datasets: Devign, described as balanced binary classification; ReVeal, described as imbalanced binary classification; and BigVul, described as multi-class and processed to balanced for fairness. DGS is compared against Bagging, Boosting, and several traditional Stacking variants whose meta-models include Logistic Regression, Random Forest, SVM, and kNN (Sun et al., 16 Sep 2025).

4. Empirical behavior across balanced, imbalanced, and multi-class settings

The reported empirical pattern is not uniform across all data regimes. On the balanced Devign dataset, traditional stacking remains competitive: stacking with kNN as meta-model excels in Recall and F1, while Logistic Regression and Random Forest maximize Accuracy and Precision. In this regime, DGS is comparable but not always better than the best stacking configuration. That observation is important because it limits any claim that DGS universally dominates classical stacking (Sun et al., 16 Sep 2025).

The contrast is sharper on ReVeal and BigVul. On the imbalanced ReVeal dataset, traditional stacking is reported to falter, failing to improve Recall or F1 over single LLMs, whereas DGS clearly outperforms all stacking meta-models and improves both Recall and F1. On the multi-class BigVul dataset, stacking is reported to be unable to outperform single LLMs, while DGS significantly improves F1 and other metrics. The paper interprets these results as evidence that code-aware gating helps the meta-model compensate for class imbalance and better handle multi-class recognition complexity (Sun et al., 16 Sep 2025).

A concise quantitative excerpt reported in the study is as follows:

Dataset	Best Stacking	DGS
Devign (F1)	63.2	62.6
ReVeal (F1)	32.7	36.3
BigVul (W-F1)	49.3	51.5

The study further summarizes the effect size by stating that, on imbalanced and multi-class tasks, DGS yields a 2–4 percentage point F1 improvement over the best LLM and stacking. It also reports vulnerability-type differences: ensemble methods, especially Boosting and DGS, consistently outperform single LLMs for memory-related vulnerabilities such as buffer overflows and use-after-free, and DGS specifically excels at “multi-layered” vulnerabilities such as permissions or control flaws that require integrating broad contextual information (Sun et al., 16 Sep 2025).

These results support a narrower but technically consequential interpretation: DGS is most valuable when the classification problem requires meta-level reasoning over both model disagreement and code context.

5. Relation to dynamic stacking and to similarly named methods

DGS belongs to a broader lineage of dynamic ensemble methods that depart from constant-coefficient stacking. An earlier dynamic stacking formulation for node classification on networks replaced constant global stacking weights with smooth functions of node topological features such as degree or closeness centrality, using a generalized varying coefficient model: $$g(m(u_i, \boldsymbol{Z}_i)) = \beta_0 + \sum_{j=1}^p Z_{ij} \beta_j(u_i).$$ In that framework, the contribution of each classifier changes smoothly with a node-dependent feature $u_i$, rather than remaining fixed across the graph (Han et al., 2016).

The two methods address different application domains and use different conditioning signals. Dynamic stacked generalization for networks conditions on topological node features; DGS for vulnerability detection conditions on code features. The shared conceptual thread is that stacking weights need not be static. A plausible implication is that DGS can be read as an input-conditioned stacking method specialized to code analysis, where the conditioning variable is an embedding or feature representation of the program rather than a graph-topological scalar.

The acronym “DGS” is also used in unrelated arXiv contexts. In spectral graph theory, “DGS” denotes graphs determined by their generalized spectrum, as in work on rooted products and DGS-graph constructions (Wang et al., 4 Jan 2026). In mobile-edge sensor fusion, the phrase “dynamic gated” appears in connection with dynamic gated neural networks and system-wide optimization, where gating selects stems and branches under latency, quality, and energy constraints (Singhal et al., 2024). Those usages are terminologically adjacent but conceptually distinct from Dynamic Gated Stacking in vulnerability detection.

6. Interpretation, scope, and practical significance

The central significance of DGS lies in the meta-learning signal it introduces. Traditional stacking assumes that base-model predictions contain sufficient information for the second-level learner to infer how experts should be combined. DGS challenges that assumption for vulnerability detection by incorporating code semantics or structure directly into the meta-model input. The reported behavior on ReVeal and BigVul suggests that this additional conditioning signal is particularly useful when minority classes are hard to identify or when the label space requires finer discrimination (Sun et al., 16 Sep 2025).

The method should not be interpreted as eliminating the need for careful ensemble selection. The same study reports that Boosting excels in scenarios involving imbalanced datasets, and its practical guidance is correspondingly differentiated: Boosting is recommended for tasks with class imbalance, whereas DGS or stacking with meta-models tailored to the target metric is recommended for balanced or multi-class problems. This indicates that DGS is best viewed as part of an ensemble design space rather than as a universal replacement for all other schemes (Sun et al., 16 Sep 2025).

From a methodological standpoint, DGS sharpens an increasingly common research theme: expert aggregation benefits from conditioning on features that explain when each expert is reliable. In earlier dynamic stacking work, those features were graph-topological. In DGS, they are code-derived. This suggests a general principle for ensemble design in structured domains: when expert quality varies systematically with observable input attributes, a dynamic meta-learner that can access those attributes may outperform a static combiner.

7. Open questions and boundaries of the reported evidence

The current evidence for DGS is empirical and domain-specific. The paper demonstrates gains in LLM-based vulnerability detection across the particular combination of five LLMs and the Devign, ReVeal, and BigVul datasets, with especially strong results on imbalanced and multi-class settings (Sun et al., 16 Sep 2025). It does not establish that DGS is uniformly superior across all balanced binary benchmarks; on Devign, its performance is comparable but not always better than the strongest traditional stackers.

Another boundary concerns the code-feature channel itself. The study states that code features may come from hidden states from an LLM or from syntactic or semantic features, but the summary does not provide a single canonical feature extractor. This suggests that some of the practical behavior of DGS may depend materially on how those features are instantiated. Likewise, the article’s reported advantages are tied to code vulnerability detection rather than to arbitrary code understanding tasks.

Taken together, DGS is best characterized as a code-aware, MoE-inspired stacking method whose principal contribution is to make the stacker’s combination rule depend on the input program representation as well as on expert predictions. In the reported experiments, that design choice is most consequential where ordinary stacking underuses context: severe class imbalance, multi-class recognition, and vulnerability categories requiring broader contextual integration (Sun et al., 16 Sep 2025).