Meta-IR: Meta-learning for Imbalanced Regression

• Meta-IR is a meta-learning framework that recommends the best pipeline by jointly selecting a resampling strategy and a regressor for imbalanced regression tasks.
• It leverages imbalance-aware metrics like F1-scoreR and SERA to focus on rare, high-relevance target regions, ensuring performance where it matters most.
• The study demonstrates that a chained formulation improves recommendation accuracy and computational efficiency, outperforming standard AutoML approaches.

Searching arXiv for the specified papers to ground the article in the cited literature. Meta-learning for Imbalanced Regression (Meta-IR) is a meta-learning framework for imbalanced regression that recommends the best pipeline composed of a resampling strategy and learning model per task in a zero-shot fashion. It was proposed for settings in which certain target values are rare yet important, and in which the optimal choice of preprocessing and regressor is affected by the dataset, the learning model, and the evaluation metric. Meta-IR treats the pipeline as a joint object, learns offline from prior tasks, and uses imbalance-aware metrics—specifically F1-score for regression and Squared Error–Relevance Area (SERA)—to define what counts as the best recommendation (Avelino et al., 16 Jul 2025).

1. Problem setting and conceptual basis

Imbalanced regression is commonly described in a utility-based regression view. In this view, a regression problem is imbalanced when the user has disproportionate preference over certain target regions and those important regions are poorly represented in the available data. Meta-IR adopts this perspective through a relevance function

$\phi : Y \rightarrow [0,1],$

where $\phi(y)$ returns a relevance score in $[0,1]$, with 0 denoting minimum relevance and 1 maximum relevance. Rare instances are then defined by a relevance threshold $t_R$, for example $t_R = 0.8$, yielding

$$D_R = \{\langle x,y \rangle \in D : \phi(y) \ge t_R\}, \qquad D_N = \{\langle x,y \rangle \in D : \phi(y) < t_R\}.$$

The percentage of rare cases is

$p.\text{rare} = 100 \times \frac{|D_R|}{|D|}.$

In the Meta-IR study, the relevance function is built using Piecewise Cubic Hermite Interpolating Polynomials over control points derived automatically from Tukey’s boxplot rule (Avelino et al., 16 Jul 2025).

A broader theoretical account defines imbalance in regression through the relation between a relevance measure $\mu$ and the marginal target distribution $P_Y$. Under the generalized definition of target balance, the target domain is $\mu$-balanced if

$\phi(y)$0

A regression dataset is imbalanced with respect to $\phi(y)$1 if that condition is violated for some pair of sets. This formalism generalizes the usual class-balance notion in classification and isolates the central issue: regions of the target space can be simultaneously highly relevant and under-represented (Kowatsch et al., 2024).

The practical difficulty follows from the behavior of standard regression objectives. Learning algorithms tend to optimize global error and can ignore rare extremes, which can yield good global RMSE but poor performance where it matters. The theoretical account in the regression-imbalance literature makes the same point in measure-theoretic terms: regions of $\phi(y)$2 with higher density $\phi(y)$3 receive more weight in the expected loss and therefore dominate optimization. This explains why ordinary MAE, MSE, and $\phi(y)$4 can be misleading in imbalanced regression, and why Meta-IR is explicitly anchored in relevance-based metrics and resampling-aware pipeline selection (Kowatsch et al., 2024).

2. Why pipeline recommendation is difficult

The pipeline-selection problem in imbalanced regression is not limited to choosing a regressor. A common alternative is to employ balancing algorithms in preprocessing to address dataset imbalance, but the variety of resampling methods and learning models creates a combinatorial selection problem. In the Meta-IR formulation, a pipeline is a pair $\phi(y)$5, where $\phi(y)$6 is a base regressor and $\phi(y)$7 is a resampling strategy, including the option of no resampling. For a dataset $\phi(y)$8, the pipeline operates by splitting the data, applying resampling to the training partition, training the regressor, and evaluating on the test partition using F1-scoreR or SERA (Avelino et al., 16 Jul 2025).

The motivation for joint recommendation is empirical as well as conceptual. The reported evidence states that there is no single best combination of learning model and resampling strategy across tasks, and that there is a strong interdependence between resampling and learning algorithm: applying a resampling algorithm changes the optimal learning model in nearly 50% of datasets. This means that the choice of resampling alters which regressor is best, and the choice of regressor affects which resampling is most effective. A plausible implication is that model-only selection mechanisms are structurally incomplete for imbalanced regression, because they omit one of the variables that defines downstream performance (Avelino et al., 16 Jul 2025).

This also clarifies the limitations of mainstream AutoML frameworks in this context. The Meta-IR paper states that Auto-sklearn, TPOT, H2O, FLAML, LightAutoML, and Naive AutoML do not natively handle imbalanced regression in their search spaces: there are no imbalanced regression resampling strategies and no imbalance-aware evaluation metrics. Even when meta-learning is used inside an AutoML framework, a full search on the new dataset is still required. Meta-IR, by contrast, is designed to learn offline from many tasks and provide a zero-shot recommendation on a new dataset without pipeline search, training, or validation at recommendation time (Avelino et al., 16 Jul 2025).

3. Framework architecture and meta-learning formulations

Meta-IR is defined over a set of imbalanced regression datasets

$\phi(y)$9

a set of candidate learning algorithms

$[0,1]$0

and a set of resampling strategies

$[0,1]$1

Each dataset $[0,1]$2 is mapped to a meta-feature vector

$[0,1]$3

and the meta-targets are the best learning algorithm $[0,1]$4 and the best resampling strategy $[0,1]$5. The meta-dataset is

$[0,1]$6

Two meta-classifiers are then trained: $[0,1]$7 predicts the best learning algorithm, and $[0,1]$8 predicts the best resampling strategy (Avelino et al., 16 Jul 2025).

The framework is organized into three phases. Phase I constructs the meta-dataset by evaluating all candidate pipelines on each dataset, selecting the best pipeline according to the optimization metric, extracting meta-features, and storing the resulting instance in $[0,1]$9. Phase II trains the meta-classifiers on that meta-dataset. Phase III performs recommendation in a zero-shot manner for a new dataset $t_R$0: compute its meta-features, predict $t_R$1 and $t_R$2, and output the pipeline $t_R$3. No internal search is performed on $t_R$4 during recommendation (Avelino et al., 16 Jul 2025).

Two formulations are proposed. In the Independent formulation, the best regressor and best resampling strategy are predicted independently from the same meta-features: $t_R$5 In the Chained formulation, the output of one meta-classifier is used as input for the other in order to model intrinsic relationship factors. Two chaining orders are defined. In Model First, the predicted model $t_R$6 is appended to the meta-features before predicting the resampling strategy. In Strategy First, the predicted resampling strategy $t_R$7 is appended before predicting the learning model. The Chained scenario showed superior performance, suggesting a relationship between the learning algorithm and the resampling strategy per task (Avelino et al., 16 Jul 2025).

The metric used to define “best” is central to the framework. For each dataset and each pipeline, performance is computed with either F1-scoreR or SERA. F1-scoreR is maximized, whereas SERA is minimized. Only the single best pipeline per dataset is kept as the ground-truth meta-label; no top-$t_R$8 ranking is used. This design makes Meta-IR a recommendation system over discrete pipeline labels rather than a direct performance regressor (Avelino et al., 16 Jul 2025).

4. Meta-features, resampling space, and optimization criteria

Each dataset is represented by 43 meta-features. These are grouped into simple meta-features, dataset distribution meta-features, correlation meta-features, linear regression–related performance meta-features, data smoothness meta-features, and, in some chained scenarios, prediction meta-features. The simple features include n.examples, n.attributes, n.rare, and p.rare. The complexity-oriented features include $t_R$9, $t_R = 0.8$0, and $t_R = 0.8$1; correlation-based measures $t_R = 0.8$2 through $t_R = 0.8$3; linear-regression measures $t_R = 0.8$4 through $t_R = 0.8$5; and smoothness measures $t_R = 0.8$6 through $t_R = 0.8$7. Complexity-based features are extracted with the ECoL package, following the definitions of Lorena et al. (2018) as reported in the Meta-IR paper (Avelino et al., 16 Jul 2025).

The resampling space $t_R = 0.8$8 contains six strategies plus NONE: SmoteR (SMT), Random Over-sampling (RO), Random Under-sampling (RU), Introduction of Gaussian Noise (GN), SMOGN (SG), and WERCS. The learning space $t_R = 0.8$9 contains six base regressors with default hyperparameters: Bagging (BG), Decision Tree (DT), Multilayer Perceptron (MLP), Random Forest (RF), Support Vector Machine (SVR), and XGBoost (XG). The resulting configuration space comprises 42 pipelines, computed as 6 learning algorithms times 7 resampling options (Avelino et al., 16 Jul 2025).

The importance of SERA in this ecosystem is not merely evaluative. SERA was introduced as an imbalance-aware metric that emphasizes errors committed at extreme values while also accounting for performance over the overall target-variable domain. In the optimization study on gradient boosting, SERA is defined by

$$D_R = \{\langle x,y \rangle \in D : \phi(y) \ge t_R\}, \qquad D_N = \{\langle x,y \rangle \in D : \phi(y) < t_R\}.$$0

where

$$D_R = \{\langle x,y \rangle \in D : \phi(y) \ge t_R\}, \qquad D_N = \{\langle x,y \rangle \in D : \phi(y) < t_R\}.$$1

The analysis in that study shows that SERA can be embedded as a loss function into optimization-based learning algorithms for imbalanced regression scenarios, and that its gradients scale errors according to relevance. This provides the metric-level background for Meta-IR’s use of SERA as one of its two optimization scenarios (Silva et al., 2022).

A broader implication, consistent with the formal literature on imbalance in regression, is that meta-features such as p.rare, intrinsic dimension, feature–target correlation, and local smoothness are not arbitrary descriptors. They are proxies for the mismatch between relevance and representation in the target space, as well as for the geometry of the supervised mapping that determines how resampling and model inductive bias will interact on a given task (Kowatsch et al., 2024).

5. Experimental evidence and comparative performance

The Meta-IR experiments use 218 imbalanced regression datasets: 200 from OpenML filtered from an initial set of 660 to retain those with at least 2% rare cases, plus 18 additional imbalanced regression datasets from Moniz et al. (2017). The datasets range from 27 to 20,640 examples, 5 to 1024 attributes, and 2.0% to 39.6% rare cases. For meta-level evaluation, the study uses leave-one-dataset-out cross-validation: one dataset is treated as the test dataset, the remaining 217 define the meta-dataset, and the recommended pipeline is evaluated on the held-out task (Avelino et al., 16 Jul 2025).

At the meta-level, the evaluation metric is F1-macro for the meta-classification tasks. For both F1-scoreR and SERA optimization scenarios, Meta-IR significantly outperforms Random and Majority in recommending both models and resampling strategies. Under F1-scoreR optimization, Independent reaches F1-macro 0.33 for learning model prediction and Chained Strategy First improves this to 0.36; for resampling prediction, Independent reaches 0.14 and Chained Model First reaches 0.16. Under SERA optimization, Independent reaches 0.31 for model prediction and 0.21 for resampling prediction, while Chained variants perform slightly worse. Thus, the best meta-learning formulation is metric-dependent (Avelino et al., 16 Jul 2025).

At the base level, Meta-IR is compared with all 42 fixed pipelines. The reported number-of-wins analysis over 218 datasets shows that Meta-IR (Model First variant) achieves more wins than any fixed pipeline for both F1-scoreR and SERA. The Wilcoxon signed-rank analysis further states that, for F1-scoreR, Meta-IR significantly outperforms all pipelines except two DT-based combinations—DT.GN and DT.RO—whereas for SERA it is statistically significantly better than all 42 pipelines, with $$D_R = \{\langle x,y \rangle \in D : \phi(y) \ge t_R\}, \qquad D_N = \{\langle x,y \rangle \in D : \phi(y) < t_R\}.$$2-value $$D_R = \{\langle x,y \rangle \in D : \phi(y) \ge t_R\}, \qquad D_N = \{\langle x,y \rangle \in D : \phi(y) < t_R\}.$$3 everywhere (Avelino et al., 16 Jul 2025).

The comparison with AutoML frameworks is similarly explicit. For F1-scoreR, Meta-IR has the best average rank and is statistically significantly better than Random, Majority, and all evaluated AutoML frameworks. For SERA, Meta-IR also ranks best on average, although the differences are not always statistically significant against Majority, H2O, FLAML, and TPOT. The win/tie/loss counts show, for example, that under F1-scoreR Meta-IR beats Auto-sklearn on 143 of 218 datasets, H2O on 143 of 218, TPOT on 147 with 2 ties, FLAML on 145, LightAutoML on 165 with 4 ties, and Naive AutoML on 132 with 4 ties (Avelino et al., 16 Jul 2025).

The advantage of Meta-IR increases with imbalance severity. When datasets are stratified by proportion of rare examples into low, medium, and high imbalance, Meta-IR significantly outperforms all AutoML frameworks at high imbalance for both F1-scoreR and SERA. At medium and low imbalance, the differences are less consistently significant, especially for SERA. This suggests that the framework’s relevance-based design aligns more strongly with problems in which the mismatch between importance and representation is most severe (Avelino et al., 16 Jul 2025).

The study also evaluates Meta-IR as a preprocessing step for AutoML. When the recommended resampling strategy is applied before AutoML search, all AutoML frameworks improve in F1-scoreR; LightAutoML improves by +16.50%, FLAML by +8.64%, and the others by +4–6%. For SERA, improvements are reported for H2O, LightAutoML, and Naive AutoML, while Auto-sklearn, TPOT, and FLAML degrade. Time efficiency is reported on 10 representative datasets: Meta-IR is about 50× faster than AutoML frameworks on average because the heavy computation is done once offline and recommendation on a new dataset is cheap (Avelino et al., 16 Jul 2025).

6. Interpretation, limitations, and research directions

Several findings in the Meta-IR study concern the internal structure of the recommendation problem. Feature-importance analysis with Random Forest Gini importance indicates that, for resampling recommendation, the most important features are p.rare, C3.min, C4.mean, and S1.mean. For model recommendation, the most important features differ by metric: under F1-scoreR, S1.sd is highlighted; under SERA, T3 and C2.mean are emphasized. The reported general pattern is that measures about feature–target relationships and output distribution or smoothness are highly informative, while simple imbalance indicators such as p.rare remain crucial (Avelino et al., 16 Jul 2025).

These findings fit a broader theoretical picture. The foundational imbalance paper argues that loss gradients are dominated by high-density target regions and that higher imbalance is strongly associated with larger weighted MAE and lower precision, recall, and F1 for regression, with Wasserstein distance showing particularly strong correlation. This suggests that meta-features encoding imbalance severity and target-space structure can serve as meaningful task descriptors for meta-learning, not only for pipeline recommendation but also for task weighting, curriculum design, or imbalance-aware meta-objectives (Kowatsch et al., 2024).

The current Meta-IR formulation also has explicit limitations. The study fixes default hyperparameters for both regressors and resampling strategies in order to reduce computational cost and focus on pipeline selection rather than fine tuning. Only 43 meta-features are used. The chained formulation uses only a single best prediction as an additional feature. The framework predicts only one best pipeline rather than a ranked list. Future work identified in the study includes hyperparameter optimization, additional meta-features, multilabel or ranking strategies, more sophisticated chaining, and top-$$D_R = \{\langle x,y \rangle \in D : \phi(y) \ge t_R\}, \qquad D_N = \{\langle x,y \rangle \in D : \phi(y) < t_R\}.$$4 recommendations. Extensions to multi-target regression, data streams, neural architectures, and integration with more advanced AutoML frameworks are presented as natural research directions rather than established results (Avelino et al., 16 Jul 2025).

A final issue concerns metric alignment. The optimization study on SERA shows a clear metric–loss alignment phenomenon: optimizing MSE yields better global average error, whereas optimizing SERA yields better performance on extremes and high-relevance regions, with modest trade-offs in MSE. Meta-IR’s own results echo this dependence on the target metric: Chained formulations improve meta-performance for F1-scoreR but not for SERA. This indicates that Meta-IR is best understood not as a universal model selector, but as a metric-conditioned recommendation framework for imbalanced regression, where the definition of “best” is inseparable from the relevance structure of the task and the evaluation criterion used to encode it (Silva et al., 2022).