FO-MAML: First-Order Meta-Learning

• FO-MAML is a meta-learning approach that approximates MAML by omitting the Hessian term, enabling rapid adaptation with minimal data.
• It simplifies the inner-outer loop computation while achieving competitive performance on few-shot benchmarks for various tasks.
• The method balances computational efficiency with effective generalization, positioning itself between full bilevel differentiation and heuristic initialization schemes.

First-Order Model-Agnostic Meta-Learning, usually abbreviated FO-MAML or FOMAML, is a gradient-based meta-learning method that inherits the task-adaptation objective of Model-Agnostic Meta-Learning (MAML) while removing the second-order differentiation required by the exact MAML meta-gradient. In the original MAML formulation, the goal is to train model parameters so that a small number of gradient steps, using only a small amount of data from a new task, produces good generalization on that task; the method is model-agnostic in the sense that it is compatible with any model trained with gradient descent and is applicable to classification, regression, and reinforcement learning (Finn et al., 2017). FOMAML is the first-order approximation to this construction obtained by dropping the Hessian-dependent term that appears when differentiating through the inner adaptation step (Nichol et al., 2018).

1. Origins and conceptual role

MAML formulates meta-learning as the search for an initialization that can be fine-tuned rapidly on a new task. The original method explicitly trains the parameters of a model so that a small number of gradient steps with limited training data from a new task yields good post-adaptation generalization, which the original paper characterizes as training the model to be easy to fine-tune (Finn et al., 2017). Within this framework, FOMAML is not a different meta-objective in its basic form, but an approximation to the MAML meta-gradient that removes second-order terms from the outer update (Nichol et al., 2018).

This first-order approximation was studied systematically in work that considered a broader family of first-order meta-learning algorithms, including Reptile as well as first-order MAML. That analysis treats the central meta-learning problem as follows: tasks are sampled from a distribution, each task induces a task-specific loss, and the objective is to learn a parameter vector that can be adapted in one or a few gradient steps so that the adapted parameter has low expected loss on the task (Nichol et al., 2018).

The practical appeal of FOMAML follows directly from its computational profile. Full MAML requires backpropagation through the inner-loop gradient step and therefore Hessian-vector terms; FOMAML keeps the same meta-learning template while using only first-order derivatives in the outer update (1908.10400). A plausible implication is that FOMAML occupies the operational midpoint between exact bilevel differentiation and simpler heuristic initialization-learning schemes.

2. Meta-objective and first-order approximation

In the one-step formulation used in the first-order analysis, a task $\tau \sim p(\tau)$ has loss $L_\tau(\theta)$, the inner adaptation step is

$$\theta'(\theta)=\theta-\alpha \nabla_\theta L_\tau(\theta),$$

and the MAML meta-objective is

$$J(\theta)=E_{\tau\sim p(\tau)}[\,L_\tau(\theta'(\theta))\,].$$

By the chain rule, the full one-step meta-gradient is

$$\nabla_\theta J(\theta) =E_{\tau}\big[(I-\alpha \nabla_\theta^2 L_\tau(\theta))\,\nabla_{\theta'}L_\tau(\theta')\big],$$

so the exact update depends on the Hessian of the task loss at the initialization (Nichol et al., 2018).

FOMAML is obtained by approximating

$$(I-\alpha \nabla_\theta^2 L_\tau(\theta)) \approx I.$$

This yields the first-order meta-gradient approximation

$$\nabla_\theta J(\theta)\approx E_{\tau}[\nabla_{\theta'}L_\tau(\theta')] =E_{\tau}[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))],$$

and therefore the outer update

$$\theta \leftarrow \theta-\beta\,E_{\tau}\big[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))\big].$$

The defining simplification is thus the omission of the Hessian term rather than any change to the adaptation operator itself (Nichol et al., 2018).

A closely related notation used in the convergence analysis writes, for task $i$,

$$F_i(w):=f_i\bigl(w-\alpha \nabla f_i(w)\bigr), \qquad F(w)=\mathbb E_{i\sim p}[F_i(w)].$$

In stochastic implementations, the inner and outer gradients are typically estimated on separate datasets, denoted $L_\tau(\theta)$0 and $L_\tau(\theta)$1, with the adapted point

$L_\tau(\theta)$2

and the FO-MAML outer direction

$L_\tau(\theta)$3

followed by

$L_\tau(\theta)$4

In this form, FO-MAML simply omits the factor $L_\tau(\theta)$5 from the exact MAML gradient (1908.10400).

3. Algorithmic structure and relation to neighboring first-order methods

A single FO-MAML meta-iteration consists of sampling a batch of tasks, performing one or a few inner-loop gradient steps for each task, evaluating the adapted parameters on held-out data from the same tasks, averaging the resulting first-order gradients, and taking one meta-step. In the notation above, this means computing a task-specific inner gradient, forming an adapted parameter $L_\tau(\theta)$6, and then using only $L_\tau(\theta)$7 in the outer update, with no backpropagation through the inner optimization map (1908.10400).

The first-order family analyzed alongside FOMAML also includes Reptile. Reptile samples one task, runs $L_\tau(\theta)$8 inner-loop steps of SGD or Adam on that task, and then moves the initialization toward the final task-trained weights: $L_\tau(\theta)$9 Unlike FO-MAML, Reptile does not explicitly split data into meta-train and meta-test partitions within the task (Nichol et al., 2018).

A Taylor-series analysis for $$\theta'(\theta)=\theta-\alpha \nabla_\theta L_\tau(\theta),$$0 inner steps identifies two leading-order effects. The first is AvgGrad, the gradient of the expected loss $$\theta'(\theta)=\theta-\alpha \nabla_\theta L_\tau(\theta),$$1. The second is AvgGradInner, a Hessian-vector contribution that, when subtracted, maximizes the inner product between gradients on different minibatches of the same task. In the reported expansion, MAML has coefficient approximately $$\theta'(\theta)=\theta-\alpha \nabla_\theta L_\tau(\theta),$$2 on this generalization term, whereas FOMAML and Reptile each have coefficient approximately $$\theta'(\theta)=\theta-\alpha \nabla_\theta L_\tau(\theta),$$3, though with different AvgGrad weights (Nichol et al., 2018). This suggests that the empirical proximity of FOMAML to full MAML in some regimes is linked to the fact that both retain the same leading-order meta-generalization mechanism, but with different weights.

The same first-order update can also be recovered from a different derivational route. Sign-MAML formulates MAML as a bilevel optimization problem and replaces the inner optimizer with signSGD,

$$\theta'(\theta)=\theta-\alpha \nabla_\theta L_\tau(\theta),$$4

using the fact that $$\theta'(\theta)=\theta-\alpha \nabla_\theta L_\tau(\theta),$$5 almost everywhere. In the MAML special case described in that work, this makes the outer derivative reduce to the same first-order meta-update as FO-MAML, but without assuming away second-order derivatives by fiat (Fan et al., 2021).

4. Empirical behavior on canonical few-shot benchmarks

The original MAML paper reports state-of-the-art performance on two few-shot image classification benchmarks, good results on few-shot regression, and accelerated fine-tuning for policy-gradient reinforcement learning with neural network policies (Finn et al., 2017). Subsequent analysis that explicitly compared first-order and second-order meta-learners found that first-order MAML performs very similarly to full MAML on some standard few-shot classification settings (Nichol et al., 2018).

The reported accuracies for the transductive versions of MAML and first-order MAML are summarized below (Nichol et al., 2018).

Setting	MAML + Transduction	1st-order MAML + Transduction
Mini-ImageNet 5-way 1-shot	$$\theta'(\theta)=\theta-\alpha \nabla_\theta L_\tau(\theta),$$6	$$\theta'(\theta)=\theta-\alpha \nabla_\theta L_\tau(\theta),$$7
Mini-ImageNet 5-way 5-shot	$$\theta'(\theta)=\theta-\alpha \nabla_\theta L_\tau(\theta),$$8	$$\theta'(\theta)=\theta-\alpha \nabla_\theta L_\tau(\theta),$$9
Omniglot 1-shot 5-way	$$J(\theta)=E_{\tau\sim p(\tau)}[\,L_\tau(\theta'(\theta))\,].$$0	$$J(\theta)=E_{\tau\sim p(\tau)}[\,L_\tau(\theta'(\theta))\,].$$1
Omniglot 5-shot 5-way	$$J(\theta)=E_{\tau\sim p(\tau)}[\,L_\tau(\theta'(\theta))\,].$$2	$$J(\theta)=E_{\tau\sim p(\tau)}[\,L_\tau(\theta'(\theta))\,].$$3
Omniglot 1-shot 20-way	$$J(\theta)=E_{\tau\sim p(\tau)}[\,L_\tau(\theta'(\theta))\,].$$4	$$J(\theta)=E_{\tau\sim p(\tau)}[\,L_\tau(\theta'(\theta))\,].$$5
Omniglot 5-shot 20-way	$$J(\theta)=E_{\tau\sim p(\tau)}[\,L_\tau(\theta'(\theta))\,].$$6	$$J(\theta)=E_{\tau\sim p(\tau)}[\,L_\tau(\theta'(\theta))\,].$$7

These results support two distinct observations. First, on Mini-ImageNet 5-way classification, first-order MAML and full MAML are nearly indistinguishable in the reported means. Second, the gap is larger on Omniglot 20-way 1-shot, where first-order MAML trails the transductive MAML baseline by several percentage points (Nichol et al., 2018). The same study also reports that Reptile is competitive, and that in the transductive Mini-ImageNet setting Reptile + Transduction reaches $$J(\theta)=E_{\tau\sim p(\tau)}[\,L_\tau(\theta'(\theta))\,].$$8 for 1-shot and $$J(\theta)=E_{\tau\sim p(\tau)}[\,L_\tau(\theta'(\theta))\,].$$9 for 5-shot (Nichol et al., 2018).

Later first-order descendants benchmarked directly against FO-MAML report that there remains room for algorithmic improvement without reintroducing second-order differentiation. On Fewshot-CIFAR100, Sign-MAML is reported at $$\nabla_\theta J(\theta) =E_{\tau}\big[(I-\alpha \nabla_\theta^2 L_\tau(\theta))\,\nabla_{\theta'}L_\tau(\theta')\big],$$0 versus FO-MAML at $$\nabla_\theta J(\theta) =E_{\tau}\big[(I-\alpha \nabla_\theta^2 L_\tau(\theta))\,\nabla_{\theta'}L_\tau(\theta')\big],$$1 for 5-way 1-shot, and $$\nabla_\theta J(\theta) =E_{\tau}\big[(I-\alpha \nabla_\theta^2 L_\tau(\theta))\,\nabla_{\theta'}L_\tau(\theta')\big],$$2 versus $$\nabla_\theta J(\theta) =E_{\tau}\big[(I-\alpha \nabla_\theta^2 L_\tau(\theta))\,\nabla_{\theta'}L_\tau(\theta')\big],$$3 for 5-way 5-shot, with essentially the same per-meta-step time as FO-MAML (Fan et al., 2021). This suggests that some of FO-MAML’s empirical limitations are not solely due to the absence of second-order computation, but also to the specific manner in which the first-order approximation is derived.

5. Theoretical analyses, limitations, and objective dependence

One line of analysis studies FO-MAML as an approximation to the original nonconvex MAML objective. Under assumptions that each $$\nabla_\theta J(\theta) =E_{\tau}\big[(I-\alpha \nabla_\theta^2 L_\tau(\theta))\,\nabla_{\theta'}L_\tau(\theta')\big],$$4 is $$\nabla_\theta J(\theta) =E_{\tau}\big[(I-\alpha \nabla_\theta^2 L_\tau(\theta))\,\nabla_{\theta'}L_\tau(\theta')\big],$$5-smooth and has $$\nabla_\theta J(\theta) =E_{\tau}\big[(I-\alpha \nabla_\theta^2 L_\tau(\theta))\,\nabla_{\theta'}L_\tau(\theta')\big],$$6-Lipschitz Hessian, with bounded stochastic variance, full MAML can find an $$\nabla_\theta J(\theta) =E_{\tau}\big[(I-\alpha \nabla_\theta^2 L_\tau(\theta))\,\nabla_{\theta'}L_\tau(\theta')\big],$$7-first-order stationary point after at most $$\nabla_\theta J(\theta) =E_{\tau}\big[(I-\alpha \nabla_\theta^2 L_\tau(\theta))\,\nabla_{\theta'}L_\tau(\theta')\big],$$8 iterations, but FO-MAML cannot achieve arbitrarily small stationarity accuracy because it ignores the factor $$\nabla_\theta J(\theta) =E_{\tau}\big[(I-\alpha \nabla_\theta^2 L_\tau(\theta))\,\nabla_{\theta'}L_\tau(\theta')\big],$$9 (1908.10400). In the reported result, FO-MAML stalls at a barrier of order $$(I-\alpha \nabla_\theta^2 L_\tau(\theta)) \approx I.$$0, and in a quadratic example it converges to a fixed point whose distance from the true meta-optimum is $$(I-\alpha \nabla_\theta^2 L_\tau(\theta)) \approx I.$$1, implying that it cannot compute an $$(I-\alpha \nabla_\theta^2 L_\tau(\theta)) \approx I.$$2-stationary point for $$(I-\alpha \nabla_\theta^2 L_\tau(\theta)) \approx I.$$3 (1908.10400).

A second line of work reframes first-order meta-learning through Moreau envelopes. There the meta-objective is

$$(I-\alpha \nabla_\theta^2 L_\tau(\theta)) \approx I.$$4

where

$$(I-\alpha \nabla_\theta^2 L_\tau(\theta)) \approx I.$$5

FO-MAML is viewed as performing $$(I-\alpha \nabla_\theta^2 L_\tau(\theta)) \approx I.$$6 inner gradient steps to approximate the proximal point $$(I-\alpha \nabla_\theta^2 L_\tau(\theta)) \approx I.$$7, followed by an outer step using $$(I-\alpha \nabla_\theta^2 L_\tau(\theta)) \approx I.$$8. Under $$(I-\alpha \nabla_\theta^2 L_\tau(\theta)) \approx I.$$9-smoothness, and $$\nabla_\theta J(\theta)\approx E_{\tau}[\nabla_{\theta'}L_\tau(\theta')] =E_{\tau}[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))],$$0-strong convexity in the convex case, that analysis proves convergence of FO-MAML to the vicinity of a solution of the Moreau-envelope objective, with linear convergence up to a neighborhood in the strongly convex setting and an $$\nabla_\theta J(\theta)\approx E_{\tau}[\nabla_{\theta'}L_\tau(\theta')] =E_{\tau}[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))],$$1 precision term plus a stationarity neighborhood in the nonconvex setting; the analysis explicitly does not require Hessian smoothness (Mishchenko et al., 2023).

These two theoretical pictures are not stated on the same objective. One treats FO-MAML as an inexact optimizer of the original MAML meta-objective, while the Moreau-envelope analysis treats it as a first-order procedure for a smoothed single-level reformulation (1908.10400). This suggests that apparently divergent guarantees in the literature are partly explained by different target objectives and different notions of approximation error (Mishchenko et al., 2023).

Another theoretical contribution studies the global optimality gap of stationary points attained by MAML-type procedures in supervised learning and reinforcement learning. In that framework, the suboptimality of an $$\nabla_\theta J(\theta)\approx E_{\tau}[\nabla_{\theta'}L_\tau(\theta')] =E_{\tau}[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))],$$2-stationary point is connected to the functional geometry of the inner-level objectives and to the representation power of the function approximator, including linear models and neural networks (Wang et al., 2020). In the supervised setting, the bound depends on $$\nabla_\theta J(\theta)\approx E_{\tau}[\nabla_{\theta'}L_\tau(\theta')] =E_{\tau}[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))],$$3, the average root-risk, and the approximation quality of a kernel-weighted feature class; in the reinforcement-learning setting, the bound depends on $$\nabla_\theta J(\theta)\approx E_{\tau}[\nabla_{\theta'}L_\tau(\theta')] =E_{\tau}[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))],$$4, reward and temperature parameters, bounded importance-weight ratios, and the approximation quality of a task-dependent target function by the policy feature class (Wang et al., 2020).

A further correction to common theoretical assumptions appears in later work on first-order meta-learning with convergence guarantees. That analysis states that the MAML objective does not satisfy the smoothness assumption used in previous works, and instead has a gradient-dependent smoothness constant of the form

$$\nabla_\theta J(\theta)\approx E_{\tau}[\nabla_{\theta'}L_\tau(\theta')] =E_{\tau}[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))],$$5

On that basis, normalized-gradient or clipped-gradient outer updates are argued to be better aligned with the geometry of the objective than plain gradient descent (Chayti et al., 2024).

6. Successor methods and the evolving interpretation of “first-order”

Several later algorithms are best understood as responses to specific weaknesses of FO-MAML rather than as simple rebrandings of it. Sign-MAML preserves a fully first-order computational pipeline by using signSGD in the inner loop, so that Hessian terms vanish because the Jacobian of the sign operator is zero almost everywhere, rather than because they are ignored in the outer differentiation (Fan et al., 2021). Its stated claim is therefore not merely lower cost, but a different theoretical grounding for the same class of first-order updates.

ES-MAML addresses a different bottleneck: the difficulty of estimating second derivatives in stochastic-policy reinforcement learning. It smooths the meta-objective with Gaussian perturbations and uses a zeroth-order gradient estimator,

$$\nabla_\theta J(\theta)\approx E_{\tau}[\nabla_{\theta'}L_\tau(\theta')] =E_{\tau}[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))],$$6

so that the adaptation operator $$\nabla_\theta J(\theta)\approx E_{\tau}[\nabla_{\theta'}L_\tau(\theta')] =E_{\tau}[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))],$$7 is treated as a black box and no derivative of $$\nabla_\theta J(\theta)\approx E_{\tau}[\nabla_{\theta'}L_\tau(\theta')] =E_{\tau}[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))],$$8 appears (Song et al., 2019). In the comparison reported there, FOMAML reduces cost but loses pre-adaptation credit assignment and can degrade performance, whereas ES-MAML can handle nonsmooth adaptation operators and deterministic policies (Song et al., 2019).

FO-B-MAML proposes a first-order alternative with vanishing approximation bias. It reformulates the inner problem as a strongly convex regularized objective, perturbs it by a parameter $$\nabla_\theta J(\theta)\approx E_{\tau}[\nabla_{\theta'}L_\tau(\theta')] =E_{\tau}[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))],$$9, and estimates the meta-gradient by finite differences between perturbed inner solutions. The reported forward estimator has bias $$\theta \leftarrow \theta-\beta\,E_{\tau}\big[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))\big].$$0 when $$\theta \leftarrow \theta-\beta\,E_{\tau}\big[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))\big].$$1, while FO-MAML and Reptile are described in the same work as having $$\theta \leftarrow \theta-\beta\,E_{\tau}\big[\nabla_\theta L_\tau(\theta-\alpha \nabla_\theta L_\tau(\theta))\big].$$2 bias in the synthetic comparison table (Chayti et al., 2024). The same paper also advocates normalized or clipped outer updates because of the gradient-dependent smoothness of the MAML objective (Chayti et al., 2024).

Taken together, these developments clarify that “first-order MAML” has two distinct meanings in the literature. In the narrow historical sense, it denotes the approximation to MAML obtained by dropping the Hessian term in the meta-gradient (Nichol et al., 2018). In the broader contemporary sense, it denotes a family of meta-learning procedures that preserve a first-order or Hessian-free computational profile while modifying either the inner optimizer, the target objective, or the outer update rule to control the bias introduced by the original FO-MAML approximation (Chayti et al., 2024).