AdaMM: Multifaceted Methods in Machine Learning

• AdaMM is a multifaceted term in ML, encompassing adaptive momentum methods for optimization and adjoint matching techniques for generative model fine-tuning.
• In optimization, AdaMM builds on Adam/AMSGrad principles, extending to zeroth-order methods using function-query estimators for black-box scenarios.
• Beyond optimization, AdaMM is used in graph anomaly detection and brain tumor segmentation, each applying distinct architectures and evaluation metrics.

Searching arXiv for papers on “AdaMM” and closely related usages to ground the article in primary sources. AdaMM is a polysemous label in current machine-learning literature. In optimization, it denotes adaptive momentum methods and, more specifically, the zeroth-order adaptive momentum method ZO-AdaMM for black-box optimization. In generative-model fine-tuning, AdaMM-style methods denote adjoint-based matching methods for entropy-regularized reward optimization, including Adjoint Matching and its discrete extension DAM. The string also appears in unrelated acronyms, notably ADAMM for anomaly detection of attributed multi-graphs with metadata and AdaMM for missing-modality brain tumor segmentation. The name therefore does not identify a single canonical algorithm; it identifies several technically distinct research programs that share an emphasis on adaptive or matching-based mechanisms (Chen et al., 2019, So et al., 6 Feb 2026, Sotiropoulos et al., 2023, Zhu et al., 18 Sep 2025).

1. Terminological scope and disambiguation

Across the cited works, the same label is used in several non-equivalent senses. The optimization literature uses “AdaMM” as umbrella nomenclature for adaptive moment methods, whereas later papers reuse the string as a paper-specific acronym or as shorthand for adjoint-based matching.

Usage	Meaning in the cited work	Representative source
AdaMM	Adaptive momentum methods such as Adam, RMSProp, AMSGrad, Padam, and AdaFom	(Chen et al., 2019)
ZO-AdaMM	Zeroth-Order Adaptive Momentum Method for black-box optimization	(Chen et al., 2019)
AdaMM-style methods	Adjoint-based Matching Methods for entropy-regularized reward optimization	(So et al., 6 Feb 2026)
ADAMM	Anomaly Detection of Attributed Multi-graphs with Metadata	(Sotiropoulos et al., 2023)
AdaMM	Multi-modal brain tumor segmentation framework for missing modalities	(Zhu et al., 18 Sep 2025)

The black-box optimization survey places ZO-AdaMM among eight representative ML-enhanced BBO methods and describes it as “Zeroth-order Adam with adaptive moments,” which reinforces the optimization-side usage of the term (Kimiaei et al., 29 Sep 2025).

2. AdaMM as adaptive moment methodology

In the ZO-AdaMM line, “AdaMM” is used as an umbrella term for adaptive momentum methods. The cited formulation includes Adam, RMSProp, AMSGrad, Padam, and AdaFom, and treats AMSGrad as the canonical first-order representative. The common structure is a first-moment accumulator,

$m_t = \beta_{1,t} m_{t-1} + (1-\beta_{1,t}) g_t,$

a second-moment accumulator,

$$v_t = \beta_2 v_{t-1} + (1-\beta_2)(g_t \odot g_t),$$

an AMSGrad max-tracking step,

$\hat v_t = \max(\hat v_{t-1}, v_t),$

and a coordinate-wise preconditioned update using $\hat V_t = \mathrm{diag}(\hat v_t)$ (Chen et al., 2019).

A central conceptual contribution of this literature is the interpretation of adaptive updates through Mahalanobis geometry. In constrained settings, the update is written with projection

$$x_{t+1} = \Pi_{\mathcal{X}, \sqrt{\hat V_t}} \bigl(x_t - \alpha_t \hat V_t^{-1/2} m_t\bigr),$$

and the analysis argues that Euclidean projection combined with adaptive diagonal scaling can fail to recover a proper stationarity condition, whereas projection in the metric induced by $\sqrt{\hat V_t}$ aligns the geometry of the step with the geometry of the constraint handling (Chen et al., 2019).

The broader adaptive-moment design space also includes first-moment-only variants. RAME, for example, is a moment-based adaptive optimizer that does not track the second moment; instead, it updates

$x_t = x_{t-1} - \eta_t \frac{m_t}{|m_t|^q + \xi},$

with

$$m_t = \beta_t m_{t-1} + \alpha_t \nabla f(x_{t-1}),$$

thereby using a nonlinear function of the most recent first moment to produce per-parameter learning rates. Within the cited taxonomy, this places RAME near AdaMM-style optimizers while departing from Adam’s second-moment design principle (Zhang et al., 2019).

3. ZO-AdaMM and the black-box optimization lineage

ZO-AdaMM is the best-defined use of AdaMM in black-box optimization. It is introduced as a representative ML-enhanced BBO method and as the canonical example of importing Adam/AMSGrad-style adaptive moments into a zeroth-order regime where gradients are unavailable and only function values can be queried. The survey positions it as a zeroth-order, optimizer-inspired, ML-enhanced BBO method aimed at continuous, possibly noisy objectives, especially in high-dimensional settings where classical ZO-SGD-style schemes are unstable or tuning-sensitive (Kimiaei et al., 29 Sep 2025).

Algorithmically, ZO-AdaMM replaces the first-order gradient with a random-direction zeroth-order estimator. In the survey’s algorithm block, the estimator is

$$\hat g_t = \frac{d}{\mu}\bigl(f(x_t+\mu u)-f(x_t)\bigr)u, \qquad u \sim \mathrm{Unif}(\mathbb{S}^{d-1}),$$

followed by Adam/AMSGrad-style recursions

$$m_t = \beta_{1,t} m_{t-1} + (1-\beta_{1,t})\hat g_t, \qquad v_t = \beta_2 v_{t-1} + (1-\beta_2)(\hat g_t \odot \hat g_t),$$

$$v_t = \beta_2 v_{t-1} + (1-\beta_2)(g_t \odot g_t),$$0

The original paper emphasizes that this is a zeroth-order generalization of AdaMM in which adaptive learning rates, momentum, and AMSGrad max-tracking are preserved while the true gradient is replaced by a function-query estimator (Chen et al., 2019).

The theory summarized across the original paper and the later survey is qualitatively consistent. For both convex and nonconvex optimization, ZO-AdaMM incurs the expected zeroth-order dimension penalty: its convergence rate is described as roughly a factor of $$v_t = \beta_2 v_{t-1} + (1-\beta_2)(g_t \odot g_t),$$1 worse than that of the first-order AdaMM algorithm, and its query complexity matches first-order Adam/AMSGrad up to a dimension-dependent factor such as $$v_t = \beta_2 v_{t-1} + (1-\beta_2)(g_t \odot g_t),$$2 or $$v_t = \beta_2 v_{t-1} + (1-\beta_2)(g_t \odot g_t),$$3, regarded as the optimal order for ZO methods. The original analysis also isolates the role of Mahalanobis projection and gives a counterexample showing that Euclidean projection can leave an AdaMM-type method stuck at a non-stationary point in a constrained problem (Chen et al., 2019, Kimiaei et al., 29 Sep 2025).

Empirically, ZO-AdaMM was evaluated on black-box adversarial attacks, including per-image and universal attacks on ImageNet. The original paper reports that it converges much faster to a high-accuracy solution than six state-of-the-art ZO methods. In the universal attack experiment with $$v_t = \beta_2 v_{t-1} + (1-\beta_2)(g_t \odot g_t),$$4 iterations, ZO-AdaMM reached 84% attack success rate with final $$v_t = \beta_2 v_{t-1} + (1-\beta_2)(g_t \odot g_t),$$5, compared with 74% and 67.74 for ZO-NES, 78% and 49.92 for ZO-PSGD, and 79% and 47.36 for ZO-SMD (Chen et al., 2019).

Subsequent work refines this AdaMM lineage rather than replacing it. R-AdaZO keeps the same ZO gradient estimator and the same first-moment update as ZO-AdaMM, but changes the second moment from

$$v_t = \beta_2 v_{t-1} + (1-\beta_2)(g_t \odot g_t),$$6

to

$$v_t = \beta_2 v_{t-1} + (1-\beta_2)(g_t \odot g_t),$$7

The paper argues that ZO-AdaMM underutilizes the variance reduction effect of the first moment and therefore overestimates the second moment with noisy zeroth-order gradients. Its analysis compares a variance term of $$v_t = \beta_2 v_{t-1} + (1-\beta_2)(g_t \odot g_t),$$8 for ZO-AdaMM with $$v_t = \beta_2 v_{t-1} + (1-\beta_2)(g_t \odot g_t),$$9 for R-AdaZO, and its experiments report a 5.4× speedup over ZO-AdaMM in MNIST black-box attacks, together with approximate speedups of 4.29× on OPT-1.3B and 3.75× on OPT-13B in ZO LLM fine-tuning (Shu et al., 3 Feb 2025).

4. AdaMM-style adjoint matching for generative models

A distinct use of the term appears in the fine-tuning of generative models. Here, AdaMM-style methods are described as Adjoint-based Matching Methods for solving the entropy-regularized reward-optimization problem

$\hat v_t = \max(\hat v_{t-1}, v_t),$0

Under a memoryless base process, the optimal path law has the closed form

$\hat v_t = \max(\hat v_{t-1}, v_t),$1

This is the common objective underlying continuous Adjoint Matching (AM) and Discrete Adjoint Matching (DAM) (So et al., 6 Feb 2026).

In the continuous setting, AM works with SDE dynamics and a continuous adjoint whose terminal condition is $\hat v_t = \max(\hat v_{t-1}, v_t),$2. DAM transfers the same objective to discrete generative models characterized by Continuous-Time Markov Chains, including diffusion-based LLMs, where the state space is non-differentiable and the continuous adjoint PDE machinery is unavailable. DAM introduces a discrete adjoint derived statistically via Dynkin’s formula. For a CTMC with base rate $\hat v_t = \max(\hat v_{t-1}, v_t),$3, value function

$\hat v_t = \max(\hat v_{t-1}, v_t),$4

and optimal rate

$\hat v_t = \max(\hat v_{t-1}, v_t),$5

the discrete adjoint is constructed as an estimator of $\hat v_t = \max(\hat v_{t-1}, v_t),$6, allowing the optimal control to be learned by a matching objective based on a Bregman divergence (So et al., 6 Feb 2026).

The DAM paper further specializes this framework to masked diffusion CTMCs, where the base rate factorizes as

$\hat v_t = \max(\hat v_{t-1}, v_t),$7

and proves that the optimal rate preserves the same masked structure,

$\hat v_t = \max(\hat v_{t-1}, v_t),$8

This makes the method scalable to discrete diffusion LLMs because it permits parametrization of $\hat v_t = \max(\hat v_{t-1}, v_t),$9 rather than the full CTMC rate tensor. Experiments cover synthetic 2D CTMCs and reasoning tasks using LLaDA-8B-Instruct on GSM8K, MATH500, Countdown, and 4x4 Sudoku. The reported results state that DAM consistently improves over the base model and matches or surpasses D1, with particularly large gains on Countdown and Sudoku, where Sudoku accuracy is reported at approximately 89% for DAM versus approximately 24% for D1 at sequence length 128 (So et al., 6 Feb 2026).

5. Acronymic reuses in graph anomaly detection and medical imaging

The string also appears as unrelated acronyms in graph learning and medical imaging. ADAMM, expanded as Anomaly Detection of Attributed Multi-graphs with Metadata, addresses graph-level anomaly detection when each instance consists of a directed, node- and edge-attributed multi-graph together with graph-level metadata. Its architecture has five stages: multi-edge set representation via a DeepSet function, an edge-aware GIN for graph embedding, projection of graph and metadata embeddings into a shared space, joint fusion through an MLP, and a Membership Estimation Network that performs soft multi-centroid assignment. The anomaly score is the responsibility-weighted squared distance to learned centroids, and training uses an unsupervised objective combining this clustering term with entropy regularization on assignments and a centroid-diversity regularizer. On three accounting datasets and one mobility dataset, the paper reports average AUROC 0.787 for ADAMM versus 0.730 for the best baseline, and average AUPRC 0.443 versus approximately 0.208 for the next best baseline (Sotiropoulos et al., 2023).

A second unrelated reuse is AdaMM for missing-modality brain tumor segmentation. This method is a knowledge-distillation-centered framework for multi-modal MRI with incomplete modality sets on BraTS 2018 and BraTS 2024. The teacher is a full-modality 3D UNet; the student is a 3D UNet with three auxiliary components: the Graph-guided Adaptive Refinement Module (GARM), the Bi-Bottleneck Distillation Module (BBDM), and the Lesion-Presence-Guided Reliability Module (LGRM). Training covers all 15 modality combinations formed from T1, T1Gd/T1ce, T2, and FLAIR, with missing channels zero-filled. The paper reports that on BraTS 2024, averaged over all 15 combinations, AdaMM achieves 83.90% Dice for WT, 64.49% for TC, and 63.54% for ET, compared with 81.91%, 54.01%, and 52.94% for the best KD baseline MSTKDNet. On BraTS 2018, the corresponding average Dice values are 86.94% for WT, 80.81% for TC, and 63.75% for ET (Zhu et al., 18 Sep 2025).

6. Conceptual synthesis and open problems

A persistent misconception is that AdaMM names a single algorithmic family. The cited literature shows otherwise. In black-box optimization, AdaMM refers to adaptive moment methods and their zeroth-order extensions; in generative-model fine-tuning, AdaMM-style methods denote adjoint-based matching for entropy-regularized control; in graph learning and medical imaging, the same letters are paper-specific acronyms for unrelated architectures. The only safe encyclopedic treatment is therefore disambiguation rather than unification.

Within the optimization lineage, the main unresolved issues are those already isolated by the cited papers: the unavoidable dimension dependence of zeroth-order estimators, sensitivity to $\hat V_t = \mathrm{diag}(\hat v_t)$0, $\hat V_t = \mathrm{diag}(\hat v_t)$1, $\hat V_t = \mathrm{diag}(\hat v_t)$2, and $\hat V_t = \mathrm{diag}(\hat v_t)$3, the continuous-domain bias of current formulations, and the incomplete theory for fully momentum-driven constrained nonconvex ZO-AdaMM. The survey also notes that discrete and mixed-integer settings require additional adaptation, and that very high dimensions motivate block-wise or variance-reduced extensions (Kimiaei et al., 29 Sep 2025).

Within the adjoint-matching lineage, DAM makes clear that the statistical Dynkin-based derivation is broader than the original continuous adjoint formalism, but also leaves several explicit extensions open: general CTMCs beyond masked diffusion, hybrid discrete-continuous processes, improved bridge distributions, stronger variance control, and preference-based versions of DAM. The paper frames these as natural next steps in a unified adjoint framework for Markov-process-based generative models (So et al., 6 Feb 2026).

The acronymic reuses have their own task-specific limitations. ADAMM’s limitations include scaling to much larger graph databases, principled selection of the number of centroids $\hat V_t = \mathrm{diag}(\hat v_t)$4, and richer modeling of complex metadata such as per-trip sequences in mobility data (Sotiropoulos et al., 2023). The medical-imaging AdaMM depends on a full-modality teacher during training, increases architectural and training complexity through adapters, GARM, BBDM, and a discriminator, and is tuned to the four-sequence BraTS MRI protocol rather than being immediately task-agnostic (Zhu et al., 18 Sep 2025).

Taken together, these works suggest a broader naming pattern rather than a single doctrine: methods labeled AdaMM tend to intervene where standard information is missing or unreliable—missing gradients, missing modalities, or latent optimal-control quantities—but they do so through sharply different mathematics, architectures, and evaluation regimes.