Value-Filtered Regression Methods

• Value-Filtered Regression is a statistical framework that targets specific data value ranges to improve estimation and feature selection, particularly in high-dimensional settings.
• Techniques include ℓp-loss based extreme event detection, quantile segmentation for local adaptation, and fused penalized logistic methods for threshold-sensitive classification.
• Empirical and theoretical results demonstrate improved sparse recovery, reduced error rates, and robust handling of noisy or corrupted observations in varied applications.

Value-Filtered Regression encompasses a family of statistical modeling frameworks wherein the model estimation, inference, or feature selection is explicitly "filtered" or targeted based on the values observed in either the predictors, the response, or both. This approach typically prioritizes specific regions of the outcome distribution, exceptional events, or exhibits selective adaptation to value-dependent error structures. Methodologies range from segmenting the response space and fitting distinct models within value bands, to constructing estimators that up-weight extreme observations, to filtering datasets by removing or adaptively adjusting the influence of noisy or corrupted points. Value-filtered regression has seen prominent application in high-dimensional sparse regression for rare-event identification, value-band-wise regression for distributional heterogeneity, error removal in corrupted datasets, and rigorous post-selection inference frameworks.

1. Sparse Regression for Extreme Values

A central variant of value-filtered regression is the Extreme Value Linear Regression model, which addresses the detection and feature selection of variables associated with rare, high-magnitude responses. The model postulates

$y_i = x_i^T \beta^* + \epsilon_i,$

with errors $\epsilon_i$ distributed as a generalized normal (Subbotin) density with shape parameter $\gamma = p > 2$ and scale $\sigma > 0$: $$f(\epsilon_i; \sigma, \gamma) = \frac{\gamma}{2 \sigma\,\Gamma(1/\gamma)} \exp\bigl\{ -|\epsilon_i/\sigma|^\gamma \bigr\}.$$ Estimation proceeds by minimizing the $p$-norm loss with $p > 2$: $$L_p(\beta) = \frac{1}{n} \sum_{i=1}^n |y_i - x_i^T \beta|^p,$$ which exponentially up-weights large residuals, emphasizing extreme-event signals over background noise. For high-dimensional settings, the Extreme Lasso extends this principle with an $\ell_1$ penalty: $$\hat{\beta} = \arg\min_{\beta \in \mathbb{R}^p} \left\{ \frac{1}{n} \sum_{i=1}^n |y_i - x_i^T \beta|^p + \lambda \|\beta\|_1 \right\}$$ where $\epsilon_i$0 induces sparsity.

Theoretical results under restricted strong convexity and sub-Weibull/Gamma tail assumptions deliver $\epsilon_i$1-consistency with rates $\epsilon_i$2 and exact support recovery for active features, provided nonzero signals exceed the appropriate threshold. Influence-function analysis demonstrates that extreme residuals have influence scaling as $\epsilon_i$3, conferring dramatic leverage to large-magnitude events in contrast to the conventional Lasso ($\epsilon_i$4) (Chang et al., 2020).

Simulation studies and applications (spiking-neuron calcium imaging, air-pollution TVOC monitoring) show that Extreme Lasso achieves F1-scores of $\epsilon_i$5–$\epsilon_i$6 with negligible false positives in rare-event support recovery, substantially outperforming standard Lasso, thresholding, and sparse quantile regression approaches.

2. Segment-Wise and Quantile-Band Regression

A distinct form is Value-Filtered Regression by segmentation of the target space, also referenced as Distribution Assertive or split-and-perform regression. The atomic steps are:

1. Partition the range of $\epsilon_i$7 into $\epsilon_i$8 bands using empirical quantiles.
2. For each $\epsilon_i$9, fit a regression model $\gamma = p > 2$0 on the subset for which $\gamma = p > 2$1.
3. At prediction time, the generated $\gamma = p > 2$2 is assigned according to which quantile-band it maps onto, ensuring bandwise model-consistency.

In notation: $\gamma = p > 2$3 where $\gamma = p > 2$4 gives the quantile-segment index.

This approach mitigates heteroscedasticity and distributional asymmetry by allowing both slopes and error characteristics to locally adapt. Empirical results (Boston Housing, UCI datasets) show that decile-wise mean absolute percentage error (MAPE) dropped from as high as $\gamma = p > 2$5 in tail deciles to $\gamma = p > 2$6–$\gamma = p > 2$7, with global average MAPE dropping from $\gamma = p > 2$8 to $\gamma = p > 2$9 after segmentation (Pathak et al., 2018).

3. Logistic Threshold Regression with Fusion Penalties

In high-dimensional classification, the Fusion Penalized Logistic Threshold Regression ("FILTER") offers an alternative value-filtered paradigm. Each continuous predictor is discretized into bins at unknown threshold points $\sigma > 0$0, transforming the feature space into a set of binary indicators. The objective

$\sigma > 0$1

combines penalized likelihood with a fused-lasso penalty, simultaneously encouraging level-wise fusion and feature sparsity.

Estimation proceeds via recursive threshold discovery (e.g., CART splits) followed by convex optimization. Theoretical analysis establishes $\sigma > 0$2 estimation error $\sigma > 0$3 and variable selection consistency, provided the estimated cut points are sufficiently accurate. FILTER's empirical results, including risk-score construction for diabetes prediction (AUC $\sigma > 0$4), outperform tree-based and standard lasso-type methods for classifying within threshold-sensitive regimes (Lin et al., 2022).

4. Value-Filtered Error Detection and Data Cleaning

Value-Filtered Regression can operate as a model-agnostic approach to remove likely corrupted data. In this setting, one computes veracity scores for each observation—using, for instance, standardized residuals scaled by estimated epistemic and aleatoric uncertainties,

$\sigma > 0$5

The data are filtered by excising points with the largest veracity scores, selecting the optimal cut-point by maximizing post-filtering cross-validated $\sigma > 0$6. This procedure has formal guarantees (residuals separate corrupted from clean points with probability tending to 1 as corruption magnitude grows), and demonstrably improves AUROC and AUPRC in empirical and real-error data settings (Zhou et al., 2023).

5. Selective Inference After Value-Based Screening

Value-filtered regression principles underpin modern approaches to post-selection inference, such as Valid $\sigma > 0$7-screening. Here, regression output is reported or interpreted only if an initial value-based filter—such as an $\sigma > 0$8-test for any effect—is passed. Standard inferential statistics become invalid when conditioning on this screening. The selective inference methodology constructs $\sigma > 0$9-values, confidence intervals, and point estimates conditional on passing the screen by conditioning on the screening event (and necessary projections), leveraging the joint distribution of $$f(\epsilon_i; \sigma, \gamma) = \frac{\gamma}{2 \sigma\,\Gamma(1/\gamma)} \exp\bigl\{ -|\epsilon_i/\sigma|^\gamma \bigr\}.$$0 or $$f(\epsilon_i; \sigma, \gamma) = \frac{\gamma}{2 \sigma\,\Gamma(1/\gamma)} \exp\bigl\{ -|\epsilon_i/\sigma|^\gamma \bigr\}.$$1 statistics. This preserves Type I error control and valid confidence coverage post-filtering (McGough et al., 29 May 2025).

Computationally, all adjusted inference can be performed with standard regression outputs (estimates, $$f(\epsilon_i; \sigma, \gamma) = \frac{\gamma}{2 \sigma\,\Gamma(1/\gamma)} \exp\bigl\{ -|\epsilon_i/\sigma|^\gamma \bigr\}.$$2, $$f(\epsilon_i; \sigma, \gamma) = \frac{\gamma}{2 \sigma\,\Gamma(1/\gamma)} \exp\bigl\{ -|\epsilon_i/\sigma|^\gamma \bigr\}.$$3-statistics), using Monte Carlo or analytic approximations to the relevant truncated distributions.

6. Value-Filtered Regression with Filtered Outcomes

A foundational nonparametric example occurs in nonparametric regression where the outcome $$f(\epsilon_i; \sigma, \gamma) = \frac{\gamma}{2 \sigma\,\Gamma(1/\gamma)} \exp\bigl\{ -|\epsilon_i/\sigma|^\gamma \bigr\}.$$4 is subject to left-truncation or right-censoring (as with survival or reliability data). The response variable itself is "filtered" by value. Estimators employ kernel smoothing for the conditional hazard or survivor function,

$$f(\epsilon_i; \sigma, \gamma) = \frac{\gamma}{2 \sigma\,\Gamma(1/\gamma)} \exp\bigl\{ -|\epsilon_i/\sigma|^\gamma \bigr\}.$$5

enabling estimation of conditional mean or median response even under complex filtering patterns. A two-step oracle-based approach achieves asymptotically efficient estimation when a multiplicative model holds. Bandwidth selection and practical algorithmic steps are adapted to the censored/truncated context (Linton et al., 2011).

7. Summary Table of Major Value-Filtered Regression Variants

Variant	Key Mechanism	Target Use/Regime
Extreme Lasso (Chang et al., 2020)	$$f(\epsilon_i; \sigma, \gamma) = \frac{\gamma}{2 \sigma\,\Gamma(1/\gamma)} \exp\bigl\{ -|\epsilon_i/\sigma|^\gamma \bigr\}.$$6 loss ($$f(\epsilon_i; \sigma, \gamma) = \frac{\gamma}{2 \sigma\,\Gamma(1/\gamma)} \exp\bigl\{ -|\epsilon_i/\sigma|^\gamma \bigr\}.$$7)	Sparse detection of rare/extreme event associations
Quantile-Band Regression (Pathak et al., 2018)	Piecewise model on $$f(\epsilon_i; \sigma, \gamma) = \frac{\gamma}{2 \sigma\,\Gamma(1/\gamma)} \exp\bigl\{ -|\epsilon_i/\sigma|^\gamma \bigr\}.$$8 bands	Heteroscedastic/asymmetric error, interpretable deciles
Logistic FILTER (Lin et al., 2022)	Discretized features, fusion penalty	High-dim. threshold effects, interpretable scores
Veracity Filtering (Zhou et al., 2023)	Residual-based error scoring	Corrupted responses, robust data cleaning
Selective $$f(\epsilon_i; \sigma, \gamma) = \frac{\gamma}{2 \sigma\,\Gamma(1/\gamma)} \exp\bigl\{ -|\epsilon_i/\sigma|^\gamma \bigr\}.$$9-screening (McGough et al., 29 May 2025)	Inference predicated on screening event	Post-selection hypothesis testing/inference
Nonparam. filter regression (Linton et al., 2011)	Hazard/survival integration	Left-truncated or right-censored $p$0

Each of these approaches exploits value-dependent structure—either in the loss, the data access, or the inferential target—to achieve estimation, inference, or prediction gains with respect to specific distributional features, event rarity, value-based uncertainty, or data quality.