MAD-OPT: Mahalanobis Optimization Framework

• MAD-OPT is a data-centric optimization framework that integrates machine learning surrogate models with Mahalanobis distance constraints derived from historical data.
• It exploits covariance geometry via positive semidefinite matrices to construct ellipsoidal feasible regions, enhancing metric learning and anomaly detection.
• The approach is applied across domains—from gas turbine optimization to object detection—demonstrating improved robustness and operational plausibility.

Searching arXiv for the cited papers and closely related Mahalanobis-optimization work. arXiv_search("Mahalanobis Distance-based Optimization MAD-OPT gas turbine (Ashraf et al., 11 Jul 2025)") Mahalanobis Distance-based Optimization (MAD-OPT) denotes, in the explicit usage of "Domain-Informed Operation Excellence of Gas Turbine System with Machine Learning," a data-centric optimization framework that couples machine-learned surrogate models with an explicit multivariate feasibility constraint derived from historical operating data (Ashraf et al., 11 Jul 2025). A broader usage is suggested by a range of arXiv papers that do not necessarily use the acronym, but do formulate the Mahalanobis matrix, covariance geometry, or Mahalanobis score itself as the primary optimization object in local metric learning, large-margin metric learning, robust pairwise-constraint fitting, out-of-distribution detection, rotated object detection, optimal transport, and speaker verification (Fetaya et al., 2015, Shen et al., 2010, Ihara et al., 2019, Kamoi et al., 2020, Wen et al., 2022, Zhang et al., 2024, Bai et al., 2019). In that broader sense, MAD-OPT refers to methods that optimize decisions, representations, or feasible regions using covariance-weighted quadratic geometry rather than isotropic Euclidean geometry.

1. Scope and conceptual interpretation

The narrow and explicit meaning of MAD-OPT is the one given in the gas-turbine study: an optimization framework that uses ANN surrogate models for performance variables and a Mahalanobis-distance-based constraint to keep the optimizer inside a domain-consistent ellipsoidal region of the input space (Ashraf et al., 11 Jul 2025). The wider interpretation is inferential but well supported by the literature provided here. Several papers are best read as Mahalanobis-distance-driven optimization schemes even when the term itself is absent. "Learning Local Invariant Mahalanobis Distances" formulates the metric matrix $M$ as the decision variable of a constrained margin maximization problem and derives an equivalent hard-margin SVM optimization (Fetaya et al., 2015). "Scalable Large-Margin Mahalanobis Distance Metric Learning" learns a PSD matrix $X$ by a convex large-margin program over triplet constraints (Shen et al., 2010). "Robust Mahalanobis Metric Learning via Geometric Approximation Algorithms" optimizes agreement with similarity and dissimilarity constraints under adversarial label corruption (Ihara et al., 2019).

This broader usage also includes post-hoc scoring systems in which Mahalanobis geometry is the main design target. Kamoi and Kobayashi’s reinterpretation of the Mahalanobis detector shows that anomaly performance can come from covariance-weighted low-variance directions rather than from classifier confidence, thereby shifting attention from posterior confidence to feature anisotropy (Kamoi et al., 2020). In other application areas, Mahalanobis geometry becomes the loss itself: in rotated object detection it appears as Mahalanobis Distance Loss over box vertices (Wen et al., 2022), and in multi-view crowd localization it appears inside an entropic optimal-transport cost with camera-aware anisotropy (Zhang et al., 2024). A useful working definition, therefore, is that MAD-OPT covers optimization procedures in which a Mahalanobis form, a Mahalanobis score, or a covariance-induced feasible region is the central mechanism rather than a secondary diagnostic.

2. Core mathematical formulations

The common primitive is the quadratic form

$d_M(x,y)^2=(x-y)^T M (x-y), \qquad M \succeq 0,$

with the PSD constraint ensuring nonnegativity and permitting the factorization $M=L^T L$, so that Mahalanobis distance is Euclidean distance after a learned linear embedding (Fetaya et al., 2015, Shen et al., 2010). In optimization-oriented formulations, the variable is usually either the PSD matrix itself, a covariance or inverse covariance, or a transformed feature representation whose geometry is then measured by such a quadratic form.

A second recurring primitive is the covariance-weighted ellipsoid

$$(\mathbf{x}-\mu)^\top \Sigma^{-1}(\mathbf{x}-\mu)\le \tau^2,$$

which operationalizes admissibility as proximity to a historical operating cloud under the joint covariance structure of the variables. This is the central feasibility constraint in the gas-turbine MAD-OPT framework (Ashraf et al., 11 Jul 2025). A third primitive is class-conditioned scoring. In the anomaly-detection formulation analyzed by Kamoi and Kobayashi, penultimate features $f(x)$ are compared to class means by

$$M(x)=\max_c -\bigl(f(x)-\hat\mu_c\bigr)^\top \hat\Sigma^{-1}\bigl(f(x)-\hat\mu_c\bigr),$$

while a marginal variant discards class identity and uses a single Gaussian over all features (Kamoi et al., 2020). These formulations are mathematically close—each uses inverse-covariance weighting—but they optimize different downstream quantities: local margins, triplet separation, robust feasibility, or in-/out-distribution scoring.

Setting	Optimization object	Representative form
Local metric learning	PSD matrix $M(x_0)$	$\min_M \frac12\|M\|^2$ s.t. $(x_i-x_0)^T M (x_i-x_0)\ge 2,\; M\succeq 0$ (Fetaya et al., 2015)
Large-margin triplet learning	PSD matrix $X$0 or $X$1	maximize margin under $X$2 (Shen et al., 2010)
Robust pairwise constraint fitting	PSD matrix $X$3	$X$4 number of violated similarity/dissimilarity constraints (Ihara et al., 2019)
Explicit domain-informed MAD-OPT	Decision vector $X$5 under ANN surrogates	optimize TE/THR subject to power set point and Mahalanobis ellipsoid (Ashraf et al., 11 Jul 2025)
Post-hoc anomaly scoring	Feature-space distance or score	nearest-class or marginal Mahalanobis score (Kamoi et al., 2020)

The important conceptual distinction is that a Mahalanobis formulation does not by itself specify the optimization goal. The same geometry can support discrimination, feasibility control, ranking, transport, or density surrogates. This suggests that MAD-OPT is better understood as a family of covariance-aware optimization designs than as a single algorithm.

3. Optimization paradigms over Mahalanobis matrices and metrics

In local metric learning, Frome et al. learn a distinct Mahalanobis matrix $X$6 for each anchor datum $X$7. Their core program minimizes Frobenius norm subject to local margin constraints, and the computational contribution is an exact reduction from the semidefinite formulation to a hard-margin SVM with quadratic kernel $X$8. The optimal matrix has the representer-style form

$X$9

and local invariance to transformations is enforced by null-space constraints

$d_M(x,y)^2=(x-y)^T M (x-y), \qquad M \succeq 0,$0

which reduce to projecting training differences onto the orthogonal complement of the transformation subspace before solving the same optimization (Fetaya et al., 2015). This is a concrete instance of MAD-OPT in which the Mahalanobis matrix is the primary decision variable and nuisance directions are explicitly removed by geometric constraints.

In scalable large-margin metric learning, Ying and Li formulate Mahalanobis learning as a convex SDP over triplet comparisons. The learned matrix $d_M(x,y)^2=(x-y)^T M (x-y), \qquad M \succeq 0,$1 satisfies $d_M(x,y)^2=(x-y)^T M (x-y), \qquad M \succeq 0,$2 and $d_M(x,y)^2=(x-y)^T M (x-y), \qquad M \succeq 0,$3, with soft margins enforced by slack variables. Their differentiable objective

$d_M(x,y)^2=(x-y)^T M (x-y), \qquad M \succeq 0,$4

is optimized by alternating over $d_M(x,y)^2=(x-y)^T M (x-y), \qquad M \succeq 0,$5 and $d_M(x,y)^2=(x-y)^T M (x-y), \qquad M \succeq 0,$6, while feasibility is maintained through rank-one updates of the form

$d_M(x,y)^2=(x-y)^T M (x-y), \qquad M \succeq 0,$7

where $d_M(x,y)^2=(x-y)^T M (x-y), \qquad M \succeq 0,$8 is the leading eigenvector of the gradient matrix (Shen et al., 2010). The paper’s main methodological lesson is that the geometry of the PSD trace-one set can be exploited to avoid repeated full eigendecompositions.

Robust Mahalanobis metric learning under adversarial label noise takes a different route. Rather than optimizing a convex surrogate such as ITML or LMNN, the objective is to minimize the number of violated pairwise constraints,

$d_M(x,y)^2=(x-y)^T M (x-y), \qquad M \succeq 0,$9

where similar pairs must lie below threshold $M=L^T L$0 and dissimilar pairs above threshold $M=L^T L$1. The theoretical contribution is an LP-type reformulation whose combinatorial dimension is $M=L^T L$2, leading to a randomized approximation scheme for fixed ambient dimension and linear regularization such as $M=L^T L$3 (Ihara et al., 2019). This is still MAD-OPT, but of a combinatorial and approximation-algorithmic kind rather than a convex-relaxation kind.

A more expansive discriminative variant is "Active Metric Learning for Supervised Classification," which optimizes nearest-neighbor consistency by mixed-integer programming and allows a generalized quadratic-plus-linear distance

$M=L^T L$4

The paper is explicit that its main formulation does not require $M=L^T L$5, so the learned function is broader than a strict Mahalanobis metric; however, in the special case $M=L^T L$6 and $M=L^T L$7, it reduces to Mahalanobis-type learning. The model jointly optimizes the metric, neighborhood structure, and outlier indicators, which is significant because it removes reliance on pre-designated target neighbors or triplets (Kumaran et al., 2018).

4. Feature geometry, post-hoc scoring, and detector design

The reinterpretation of Mahalanobis OoD detection by Kamoi and Kobayashi is central to contemporary MAD-OPT thinking because it rejects a common explanation. Their analysis starts from the standard conditional score with shared covariance in penultimate feature space and introduces a marginal version

$M=L^T L$8

which discards class information altogether (Kamoi et al., 2020). The decisive empirical result is that low-variance principal directions, which are nearly useless for classification, can carry most of the anomaly signal because inverse-covariance weighting amplifies deviations along those directions. This suggests that Mahalanobis scoring works largely because it emphasizes anisotropy rather than because pretrained features truly obey a tied-covariance Gaussian discriminant model. A direct implication, explicitly marked as such in the paper, is that future Mahalanobis-based objectives should probably treat covariance geometry and low-variance subspaces as first-class design targets rather than mere proxies for classifier confidence.

A separate line of work in medical imaging shows that the layer at which Mahalanobis distance is computed is itself an optimization variable in practice. "On the use of Mahalanobis distance for out-of-distribution detection with neural networks for medical imaging" computes Mahalanobis scores after every module and finds that there is no one-size-fits-all optimal layer: the best depth changes with the type of OOD pattern, and last-hidden-layer Mahalanobis can perform poorly (Anthony et al., 2023). The paper proposes Multi-branch Mahalanobis (MBM),

$M=L^T L$9

which keeps several branch-specific detectors instead of enforcing a single universal layer or a single globally optimized fusion. This directly challenges the misconception that one Mahalanobis feature space can serve all OOD regimes equally well.

MahaVar extends the same post-hoc tradition by operating not only on the minimum classwise distance but on the variance of the full classwise distance vector: $$(\mathbf{x}-\mu)^\top \Sigma^{-1}(\mathbf{x}-\mu)\le \tau^2,$$0 Under relaxed Neural Collapse assumptions, ID samples tend to have a "one-small, many-large" classwise distance profile and therefore higher classwise variance than OOD samples (Kim et al., 14 May 2026). This is not presented as a training-time optimization recipe, but it suggests a richer MAD-OPT view in which one may optimize the entire classwise Mahalanobis distance profile rather than only the nearest-class term.

For distance-intensive loops, the bottleneck may be evaluation rather than metric learning itself. "Online Adaptive Mahalanobis Distance Estimation" provides sketching-based data structures for $$(\mathbf{x}-\mu)^\top \Sigma^{-1}(\mathbf{x}-\mu)\le \tau^2,$$1-approximate Mahalanobis distance queries under adaptive querying and row updates to a factor $$(\mathbf{x}-\mu)^\top \Sigma^{-1}(\mathbf{x}-\mu)\le \tau^2,$$2 with $$(\mathbf{x}-\mu)^\top \Sigma^{-1}(\mathbf{x}-\mu)\le \tau^2,$$3. Its QueryPair, QueryAll, and online update mechanisms are explicitly positioned as support for algorithms that repeatedly update a Mahalanobis metric and then need many downstream distance computations (Qin et al., 2023). This suggests that MAD-OPT has an algorithmic systems dimension in addition to its statistical one.

5. Domain-specific implementations

The paper that explicitly names MAD-OPT applies it to a 395 MW gas turbine system. All variables are min-max scaled to $$(\mathbf{x}-\mu)^\top \Sigma^{-1}(\mathbf{x}-\mu)\le \tau^2,$$4, separate ANN models are trained for power, thermal efficiency, and turbine heat rate, and optimization is carried out over the process variables under a power set-point condition and the Mahalanobis ellipsoid

$$(\mathbf{x}-\mu)^\top \Sigma^{-1}(\mathbf{x}-\mu)\le \tau^2,$$5

The scalar objective combines maximizing thermal efficiency and minimizing turbine heat rate in scaled space, while $$(\mathbf{x}-\mu)^\top \Sigma^{-1}(\mathbf{x}-\mu)\le \tau^2,$$6 is tuned empirically for in-range and extrapolative operating regimes (Ashraf et al., 11 Jul 2025). The methodological point is not only that the constraint improves realism, but that unconstrained ANN-driven optimization can produce numerically attractive yet operationally implausible solutions. In the reported 390 MW study, the unconstrained optimizer drove gas fuel flow to 28.56 lb/s and predicted THR and TE outside observed limits, whereas the constrained MAD-OPT solution remained inside historical operating ellipses and was robust under Monte Carlo perturbation (Ashraf et al., 11 Jul 2025).

In rotated object detection, Mahalanobis geometry is used as a regression loss rather than a feasibility constraint. "Rotated Object Detection via Scale-invariant Mahalanobis Distance in Aerial Images" defines the Mahalanobis Distance Loss

$$(\mathbf{x}-\mu)^\top \Sigma^{-1}(\mathbf{x}-\mu)\le \tau^2,$$7

with $$(\mathbf{x}-\mu)^\top \Sigma^{-1}(\mathbf{x}-\mu)\le \tau^2,$$8 vertices and covariance $$(\mathbf{x}-\mu)^\top \Sigma^{-1}(\mathbf{x}-\mu)\le \tau^2,$$9 computed from the four box corners (Wen et al., 2022). Because the covariance scales with object size, the loss is described as scale-invariant and more consistent with SkewIoU than $f(x)$0, $f(x)$1, or smooth $f(x)$2. Boundary discontinuity is handled by taking the minimum over the four cyclic permutations of the predicted vertices. The paper reports 76.16 mAP for MDL-p on DOTA-v1.0, compared with 73.98 mAP for smooth $f(x)$3, and explicitly states that MDL is more stable during training than smooth $f(x)$4 (Wen et al., 2022).

In multi-view crowd localization, Mahalanobis geometry enters the cost matrix of an entropic unbalanced optimal-transport loss. The proposed M-MVOT constructs, for each ground-truth point, a covariance

$f(x)$5

where the rotation $f(x)$6 is aligned with the closest camera’s view ray, $f(x)$7, and $f(x)$8. Transport cost is then

$f(x)$9

so errors along and across the view ray are penalized differently (Zhang et al., 2024). This is a highly explicit MAD-OPT construction: the optimization remains Sinkhorn-based OT, but the metric tensor inside the cost is camera-aware and anisotropic.

Speaker verification provides a ranking-based instance. pAUCMetric uses the pairwise score

$$M(x)=\max_c -\bigl(f(x)-\hat\mu_c\bigr)^\top \hat\Sigma^{-1}\bigl(f(x)-\hat\mu_c\bigr),$$0

and learns the PSD matrix $$M(x)=\max_c -\bigl(f(x)-\hat\mu_c\bigr)^\top \hat\Sigma^{-1}\bigl(f(x)-\hat\mu_c\bigr),$$1 by minimizing a hinge surrogate of partial AUC over the desired FPR interval $$M(x)=\max_c -\bigl(f(x)-\hat\mu_c\bigr)^\top \hat\Sigma^{-1}\bigl(f(x)-\hat\mu_c\bigr),$$2, together with $$M(x)=\max_c -\bigl(f(x)-\hat\mu_c\bigr)^\top \hat\Sigma^{-1}\bigl(f(x)-\hat\mu_c\bigr),$$3 regularization (Bai et al., 2019). The paper’s main claim is that the Mahalanobis parameterization makes the objective convex in $$M(x)=\max_c -\bigl(f(x)-\hat\mu_c\bigr)^\top \hat\Sigma^{-1}\bigl(f(x)-\hat\mu_c\bigr),$$4, so global optimum learning is feasible while directly targeting the operational part of the ROC curve.

6. Variants, limitations, and recurrent design issues

A major recurrent issue is covariance estimation under rank deficiency or small sample size. In time-series classification, covariance matrices are often low-rank, so the paper compares Moore-Penrose pseudoinverse, covariance shrinkage, and diagonal Mahalanobis approximations. Its recommendation is to learn one distance measure per class using either covariance shrinking or the diagonal approach, while noting that DTW remains superior in accuracy even though Mahalanobis measures are one to two orders of magnitude faster (Prekopcsák et al., 2010). This is a practical reminder that MAD-OPT gains can depend more on regularized covariance estimation and class-specific deployment rules than on sophisticated objective design alone.

In functional data, the same problem appears as an operator-theoretic one. "The Mahalanobis distance for functional data with applications to classification" replaces finite-dimensional inverse covariance by a truncated inverse square-root covariance operator,

$$M(x)=\max_c -\bigl(f(x)-\hat\mu_c\bigr)^\top \hat\Sigma^{-1}\bigl(f(x)-\hat\mu_c\bigr),$$5

and defines a functional Mahalanobis semi-distance as the Euclidean norm of the first $$M(x)=\max_c -\bigl(f(x)-\hat\mu_c\bigr)^\top \hat\Sigma^{-1}\bigl(f(x)-\hat\mu_c\bigr),$$6 whitened functional principal component scores (Joseph et al., 2013). The word "semi-distance" is deliberate: truncation means distinct functions can coincide in the retained subspace. For MAD-OPT, this is a reminder that covariance-aware geometry in infinite or very high dimension usually requires explicit regularization or truncation.

A more foundational unsupervised formulation appears in "Optimal Rescaling and the Mahalanobis Distance," which casts affine rescaling as cross-entropy minimization over Gaussian families. With a fixed center $$M(x)=\max_c -\bigl(f(x)-\hat\mu_c\bigr)^\top \hat\Sigma^{-1}\bigl(f(x)-\hat\mu_c\bigr),$$7, the optimal covariance has the closed form

$$M(x)=\max_c -\bigl(f(x)-\hat\mu_c\bigr)^\top \hat\Sigma^{-1}\bigl(f(x)-\hat\mu_c\bigr),$$8

equivalently the paper’s stated rank-one update formula, and the exact excess mismatch is

$$M(x)=\max_c -\bigl(f(x)-\hat\mu_c\bigr)^\top \hat\Sigma^{-1}\bigl(f(x)-\hat\mu_c\bigr),$$9

(Spurek et al., 2013). This provides a precise optimization foundation for Mahalanobis-type whitening and shows that even unsupervised MAD-OPT can be cast as likelihood or cross-entropy minimization under structural constraints.

Feature clustering offers another regularization strategy. "Mahalanonbis Distance Informed by Clustering" uses k-means on coordinates to define a subspace constraint for principal directions and then projects or optimizes PCA directions within that row-cluster subspace before constructing a global or local Mahalanobis matrix (Lahav et al., 2017). The empirical message is that coordinate-cluster regularization can improve principal-direction estimation and downstream distances when sample size is small relative to dimension. This suggests that structural priors on eigendirections may be as important as priors on eigenvalues.

Several controversies and misconceptions recur across the literature. One is the theory-practice mismatch in deep feature modeling: Mahalanobis OoD success does not validate the tied-covariance Gaussian assumption for penultimate features (Kamoi et al., 2020). Another is the assumption of a universal optimal layer or universal fusion for Mahalanobis OoD detection, which is contradicted in medical imaging (Anthony et al., 2023). A third is that not every quadratic optimization over pair distances is a strict Mahalanobis metric-learning problem; some generalized formulations relax the PSD requirement and therefore leave the classical metric setting (Kumaran et al., 2018). Finally, multiple papers show that covariance-aware methods can still lose to task-specific alternatives in some regimes: Euclidean distance can outperform Mahalanobis in certain metric-learning models for Omniglot vs EMNIST (Kamoi et al., 2020), and DTW can outperform Mahalanobis-based time-series classifiers despite much higher runtime (Prekopcsák et al., 2010).

Taken together, these results suggest that MAD-OPT is best understood not as a single closed recipe but as a design philosophy. The common principle is to make covariance geometry explicit—through a PSD matrix, a covariance operator, a classwise score vector, or an ellipsoidal feasible set—and then optimize the task-specific objective in that geometry. The main open design choices are where the geometry is imposed, how covariance is regularized, whether the metric is global or local, and whether the target phenomenon is discrimination, density conformity, robust feasibility, or anomaly sensitivity.