MALD: Mean Absolute Local Derivative

• MALD is a variable importance measure defined as the average magnitude of local effects computed from a fitted regression function, applicable to both continuous and categorical features.
• It enables robust variable selection in heterogeneous, nonlinear settings by leveraging local derivatives or prediction contrasts from models like neural networks and random forests.
• Empirical evaluations demonstrate that MALD effectively controls false discovery rates and outperforms traditional measures such as LASSO and Gini importance in complex data analysis.

Searching arXiv for the cited paper and knockoff references. arxiv_search query: (Mason et al., 20 Aug 2025) arxiv_search query: "Controlling the false discovery rate via knockoffs" Mean Absolute Local Derivative (MALD) is a variable importance measure introduced for knockoff-based variable selection in heterogeneous, distribution-free settings where linear importance statistics are inadequate for complex outcome mechanisms (Mason et al., 20 Aug 2025). It is defined from a fitted regression function $g(\cdot)$ and quantifies, for each feature, the average magnitude of its local effect on the predicted outcome. In the proposed framework, MALD is designed to be compatible with nonlinear outcome functions, including random forests and neural networks, while accommodating both continuous and categorical variables. The paper positions MALD as an interpretable alternative to coefficient-based or Gini-style importance measures, particularly when variable selection must be coupled with false discovery rate (FDR) control through knockoffs (Mason et al., 20 Aug 2025).

1. Definition and motivation

MALD was proposed in the context of knockoff variable selection, where synthetic knockoff variables are constructed to mirror the correlation structure of the observed features and then compared with the original variables through a feature-importance statistic (Mason et al., 20 Aug 2025). The motivation is explicit: traditional variable importance measures in the knockoff framework, such as those based on the LASSO or linear models, assess variable importance by fitting linear relationships between the predictors and the outcome, and these approaches break down or become unreliable in the presence of complex nonlinear and heterogeneous data.

Within that setting, MALD is presented as a novel, general-purpose variable importance measure that works with nonlinear outcome models, handles both categorical and continuous variables, and is especially compatible with machine learning methods such as random forests and neural networks (Mason et al., 20 Aug 2025). Its role is not to replace knockoff construction, but to provide the importance statistic used after a predictive model has been fit to the response using original and knockoff variables jointly.

The central interpretive idea is local sensitivity. Rather than assigning importance from a global linear coefficient, MALD measures how much the prediction changes when one feature is perturbed locally, and then averages the magnitude of that change across samples. This construction is meant to avoid sign cancellation and to remain meaningful when the feature effect is nonlinear, nonconstant across the covariate space, or expressed differently for continuous and categorical inputs.

2. Mathematical formulation

Let $X$ denote the observed features and $y$ the outcome. Suppose a flexible regression function $g(\cdot)$ is fit to model the dependence of $y$ on $X$. For a continuous variable $X_j$, the local derivative at sample $i$ is defined by

$$\ell_{ij} = \frac{\partial}{\partial X_{ij}} \mathbb{E}[y_i \mid X_{i\cdot}] \approx \frac{\partial}{\partial X_{ij}} g(X_{i\cdot}).$$

MALD then aggregates absolute local derivatives, optionally raised to a power $r>0$, through

$X$0

In practice, $X$1 is most common. The use of absolute values is intended to avoid cancellation from derivatives of different signs, as can occur under quadratic or other nonlinear responses (Mason et al., 20 Aug 2025).

For categorical variables, derivatives with respect to indicator variables are not well-defined in the classical sense, so MALD uses finite contrasts in predicted values. For a binary categorical variable $X$2,

$X$3

For a multi-level categorical variable with levels $X$4,

$X$5

The corresponding importance remains

$X$6

This formulation yields a unified statistic across continuous and categorical predictors. The paper states that the measure quantifies the magnitude of the variable’s local effect on the prediction regardless of sign or variable type. In a linear model $X$7, the derivative is constant, so MALD reduces to $X$8, making it consistent with the absolute value of a classical coefficient-based importance measure (Mason et al., 20 Aug 2025).

3. Role in the knockoff workflow

The MALD-based procedure is embedded in a knockoff pipeline. A predictive model $X$9 is first fit to the outcome using both original and knockoff variables. Then, for each variable $y$0 and each sample $y$1, one computes a local effect $y$2: for a continuous variable, the local derivative of $y$3 with respect to $y$4; for a categorical variable, the range of fitted values obtained by varying that variable over its possible levels while keeping the remaining inputs fixed (Mason et al., 20 Aug 2025).

The resulting per-variable importance is

$y$5

when $y$6 has already been converted to an absolute quantity, or equivalently $y$7 in the notation above. The same computation is repeated for the knockoff variables to obtain $y$8. The knockoff statistic is then

$y$9

Selection is performed by applying the knockoff filter to the $g(\cdot)$0 statistics.

The paper summarizes this workflow in pseudocode form. For numeric variables, it uses $g(\cdot)$1. For categorical variables, it uses

$g(\cdot)$2

The importance is averaged across samples, the same construction is applied to knockoff features, and $g(\cdot)$3 is passed to the knockoff filter for FDR control (Mason et al., 20 Aug 2025).

This places MALD in the part of the knockoff framework where a valid importance statistic must satisfy the flip-sign property. The paper explicitly states that exchangeability arguments in knockoff theory carry over when MALD is used in the difference form $g(\cdot)$4.

4. Compatibility with nonlinear models

A central feature of MALD is model-agnostic compatibility with nonlinear outcome estimators. The paper lists neural networks, random forests, and other machine learning models that provide conditional mean predictions. For neural networks, automatic differentiation can efficiently compute $g(\cdot)$5 for all variables and samples. For random forests, gradients are estimated through finite differences, using

$g(\cdot)$6

for small $g(\cdot)$7, where $g(\cdot)$8 is the standard basis vector (Mason et al., 20 Aug 2025).

This model compatibility is one of the main reasons MALD is proposed. Most prior knockoff statistics and variable importance measures depend on linearity or on specific properties of estimators such as the LASSO path. MALD is instead formulated in terms of the fitted conditional mean surface, so it can be used with any estimator that can be queried locally. This suggests a broader operational domain than coefficient-based statistics, especially when the outcome mechanism is smooth only piecewise, highly nonlinear, or learned by a nonparametric model.

The paper also connects MALD to interpretability. It states that MALD is directly interpretable as average local sensitivity, or effect size, and that it gives meaningfully comparable importance across variables of different types and scales (Mason et al., 20 Aug 2025). For categorical features, this comparison is enabled by replacing derivatives with level-wise prediction contrasts. For continuous features, the derivative-based construction preserves a localized notion of responsiveness. The same section of the paper contrasts MALD with random-forest Gini importance, stating that MALD is not affected by known biases in Gini/MDI importance, which can favor variables with more categories or higher variance.

5. Theoretical properties and assumptions

Several theoretical properties are stated for MALD in the paper. First, there is a baseline consistency claim: in linear models, MALD corresponds to the classical coefficient-based importance, preserving the familiar interpretation of absolute effect magnitude (Mason et al., 20 Aug 2025). Second, for nonlinear models, averaging absolute local derivatives is intended to reflect actual, possibly non-constant, influence on the outcome throughout the feature space rather than a single global coefficient.

Third, the paper links MALD to knockoff validity through the form $g(\cdot)$9. Because the knockoff filter requires a variable importance statistic with the flip-sign property, computing MALD for original and knockoff variables separately and then taking their difference is presented as compatible with the standard exchangeability arguments used in knockoff theory. The paper further states that, for knockoff FDR control, the exchangeability requirement on knockoff construction is needed, whereas MALD itself only requires an outcome estimator (Mason et al., 20 Aug 2025).

The assumptions attached to MALD are comparatively modest but operationally important. The outcome model $y$0 must be sufficiently flexible and well fitted to capture the true $y$1. For categorical variables, the measure assumes that the outcome model can be queried by varying one input at a time. For random forests and other models without analytic gradients, finite-difference approximations must be numerically workable. A plausible implication is that MALD inherits both the strengths and the estimation errors of the fitted predictor: if $y$2 misses the response surface, MALD will reflect the misspecified surface rather than the true conditional mean.

The paper also characterizes MALD as scale-invariant by construction, through averaging absolute derivatives, and as directly interpretable as average local sensitivity (Mason et al., 20 Aug 2025). Those statements are part of the proposed rationale for using MALD in settings with mixed data types and heterogeneous covariates.

6. Empirical behavior, comparisons, and applications

The empirical evaluation reported in the paper includes simulation studies and a DNA methylation application in mouse tissues. In simulations varying signal strength, number of mixture modes, and covariate correlations, MALD-based importance measures combined with nonlinear outcome models such as random forests or neural networks showed higher or comparable statistical power than linear measures when the outcome was nonlinear or the data were heterogeneous (Mason et al., 20 Aug 2025). The paper also reports effective FDR control when MALD was paired with suitable knockoff generators, including conditional residual or Forest SCIP constructions.

The same experiments are used to compare MALD against alternative combinations of importance statistics and knockoff generators. According to the paper, conservative or inappropriately lenient FDR control was more common for second-order (linear) knockoffs or GAN-based knockoffs, depending on the scenario (Mason et al., 20 Aug 2025). The comparison with traditional LASSO-based statistics is explicit: those methods rely on linearity, do not capture nonlinear effects, and their FDR control can break in nonlinear or heterogeneous settings. The comparison with Gini importance is also explicit: Gini/MDI can exhibit scale, category-count, and correlation biases, whereas MALD is proposed as a unified and bias-resistant alternative.

The real-data demonstration uses a DNA methylation dataset from mouse tissues with more than 5,000 mouse samples and more than 730 CpG sites, along with tissue and sex covariates; the outcome is chronological or transformed age (Mason et al., 20 Aug 2025). In that setting, traditional LASSO-based knockoff selection performed well when the true outcome was linear, but failed to maintain FDR or to identify meaningful structure when the outcome was nonlinear. MALD with a random forest or neural network outcome model enabled identification of CpG sites relevant for age prediction even with nonlinear relationships, achieved practical FDR control, and yielded greater power than other nonlinear importance measures. The paper further states that the MALD-based knockoff approach produced smaller sets of selected features but better cross-validated mean squared error than using the full set or sets from standard LASSO selection, and that the selected features corresponded to biologically reasonable and previously reported CpG sites (Mason et al., 20 Aug 2025).

Software accompanying the method is available in R under the package name rangerKnockoff and in Python under MALDimportance (Mason et al., 20 Aug 2025). This indicates that MALD was presented not only as a theoretical importance statistic, but also as an implemented component of a broader workflow for knockoff generation and nonlinear variable selection in mixed-type data.