Adjoint Tangent Structure

• Adjoint Tangent Structure is a framework that pairs a forward tangent model with a reverse adjoint model to capture first-order variations and sensitivities.
• It formalizes the duality in linearized approximations of nonlinear models, with applications in numerical weather prediction, CFD, automatic differentiation, and Lie-group backpropagation.
• This structure bridges numerical analysis, geometric representations, and categorical dualities, ensuring consistent and efficient sensitivity computation across diverse scientific domains.

Searching arXiv for the core paper and closely related uses of “adjoint tangent structure” across computational differentiation, tangent categories, and geometric settings. “Adjoint tangent structure” is not a single universally standardized term; rather, it names a recurring pattern in which a nonlinear object is accompanied by two linearized companions: a tangent construction that propagates perturbations forward, and an adjoint construction that propagates sensitivities backward. In numerical modeling, this is the mathematical and algorithmic “shadow” of a nonlinear code (0711.4444). In the TorchNWP framework for neural-network parameterizations inside numerical weather prediction systems, it is the formalization and implementation of tangent linear and adjoint models for neural-network components embedded in large Fortran models (Xiao et al., 17 Mar 2026). In categorical work, the phrase appears through tangent structures whose tangent bundle functor has an adjoint, yielding dual or opposite tangent categories (Leung, 2016, Ikonicoff et al., 2023, Lanfranchi, 24 Mar 2025, Ikonicoff, 22 Aug 2025). In Lie-theoretic and differential-geometric settings, it refers to tangent-bundle or cotangent-bundle structures tied to adjoint actions, adjoint orbits, or adjoint groups (Prado et al., 2022, Biswas et al., 2013). Across these contexts, the common invariant is a pair of mutually related linear operators or functors that encode first-order variation and reverse transport of dual data.

1. Computational meaning: tangent and adjoint as linearized companions of a nonlinear model

In numerical analysis and data assimilation, the basic object is a nonlinear mapping

$Y = F(X),$

or, for time-dependent models, a time-stepping scheme

$x_{k+1} = M_k(x_k,p).$

The tangent linear model describes forward propagation of perturbations, while the adjoint model describes backward propagation of sensitivities (0711.4444). In the tangent formulation, a perturbation $\delta x_k$ evolves according to

$$\delta x_{k+1} = \frac{\partial M_k}{\partial x_k}(x_k,p)\,\delta x_k + \frac{\partial M_k}{\partial p}(x_k,p)\,\delta p.$$

The adjoint propagates a dual variable $\lambda_k$ backward through the transposed Jacobian,

$$\lambda_k = \left(\frac{\partial M_k}{\partial x_k}(x_k,p)\right)^{T}\lambda_{k+1} + \frac{\partial J}{\partial x_k}(x_k,p)^T,$$

with terminal condition determined by the objective functional $J$ (0711.4444).

This forward–backward duality is also the basis of algorithmic differentiation. If a program $P$ computes $Y=F(X)$ through a composition of elementary instructions, then forward mode computes directional derivatives $\dot Y = F'(X)\dot X$, while reverse mode computes $x_{k+1} = M_k(x_k,p).$0 (0711.4444). In the language used for large CFD solvers, tangent mode realizes a Jacobian–vector product and adjoint mode realizes a vector–Jacobian product (Cardesa et al., 2019). A standard consistency identity is

$x_{k+1} = M_k(x_k,p).$1

which expresses the transposed relation between the two codes (Cardesa et al., 2019).

This computational meaning remains valid when the primal nonlinear problem is only partially converged. In that case, one may linearize the fixed-point iteration rather than the converged discrete PDE, obtaining tangent and adjoint systems about non-stationary points (Padway et al., 2021). The resulting sensitivities are approximate, and their accuracy depends on the convergence of the nonlinear system; the paper states that “in the limit of machine zero convergence of the nonlinear problem this approximation error vanishes” (Padway et al., 2021). A plausible implication is that the phrase “adjoint tangent structure” often denotes not merely a pair of formulas, but the entire discipline of keeping linearized and reverse-time models consistent with the discrete nonlinear computation.

2. Neural-network adjoint tangent structure in numerical weather prediction

In the TorchNWP work, adjoint tangent structure is the way tangent linear and adjoint models are formalized, implemented, and exposed for neural-network components embedded inside large Fortran numerical weather prediction systems (Xiao et al., 17 Mar 2026). The nonlinear neural-network parameterization is written as

$x_{k+1} = M_k(x_k,p).$2

with tangent linear model

$x_{k+1} = M_k(x_k,p).$3

where $x_{k+1} = M_k(x_k,p).$4 and $x_{k+1} = M_k(x_k,p).$5 denote input and output perturbations (Xiao et al., 17 Mar 2026). The adjoint is obtained from a scalar loss

$x_{k+1} = M_k(x_k,p).$6

and automatic differentiation computes

$x_{k+1} = M_k(x_k,p).$7

The paper states: “A scalar loss function $x_{k+1} = M_k(x_k,p).$8 is constructed. Using the automatic differentiation tool of LibTorch, the gradient of the scalar loss function $x_{k+1} = M_k(x_k,p).$9 with respect to the input physical quantities is calculated to obtain the adjoint model result” (Xiao et al., 17 Mar 2026).

TorchNWP builds these operators at the C/C++ level using LibTorch. Neural networks are trained in PyTorch, exported to .pt files, loaded by torch::jit::load, and called through standardized C/C++ interfaces that are then bound into Fortran through ISO_C_BINDING (Xiao et al., 17 Mar 2026). The forward interface is explicit: model_new initializes the model, model_forward performs inference, and model_delete frees resources (Xiao et al., 17 Mar 2026). The tangent and adjoint calculations are “encapsulated into standardized C/C++ interfaces, which are bound to the Fortran language through the ISO_C_Binding module” (Xiao et al., 17 Mar 2026).

A central design principle is structural shielding. The paper states that tangent linear and adjoint technologies based on neural networks are implemented “at the C++ layer to shield the impact of the internal structure of neural network models and provide standardized tangent linear and adjoint interfaces” (Xiao et al., 17 Mar 2026). In practice, the Fortran host model does not need to know the number of layers, activation functions, or convolution structure; the .pt model, LibTorch runtime, and automatic differentiation machinery provide the required derivatives.

This structure is motivated directly by four-dimensional variational data assimilation. The TorchNWP paper says that after coupling a neural-network model with a numerical model, “to adapt to the four-dimensional variational data assimilation system, it is necessary to construct tangent linear and adjoint models corresponding to the neural network model” (Xiao et al., 17 Mar 2026). The implementation is intended to simplify the construction of such systems in operational models including CMA-GFS and MCV, and the paper reports applications to deep learning based radiation and non-orographic gravity wave drag parameterizations (Xiao et al., 17 Mar 2026). For the CMA-GFS non-orographic gravity wave drag surrogate, the reported six-hour mean absolute errors relative to the original physical scheme are $\delta x_k$0 for 10 m wind speed, $\delta x_k$1 for 500 hPa wind speed, and $\delta x_k$2 for 500 hPa temperature; the coupled model is also reported to run a 10-day integration stably (Xiao et al., 17 Mar 2026).

3. Large-scale differentiated codes: checkpointing, MPI, and operational geophysical models

The classical geophysical interpretation of adjoint tangent structure is exemplified by OPA–TAPENADE and later by large AD-enabled atmosphere–ocean and CFD systems. In OPA, a FORTRAN95 ocean general circulation model, tangent and adjoint codes were generated mechanically with TAPENADE and validated by comparison with divided differences and by an identical twin experiment (0711.4444). The paper emphasizes that the differentiated code must mirror the modular structure of the original model, including modules, arrays, dynamic memory, and non-reentrant procedures (0711.4444). For array notation, TAPENADE differentiates expressions such as $\dot Y = F'(X)\dot X$1 into array-level adjoint code that also handles nondifferentiability at zero through WHERE constructs (0711.4444).

A defining issue in adjoint codes is memory. Reverse mode needs access to intermediate forward states, so naive “store-all” strategies become prohibitive for long integrations. OPA implements Griewank–Walther binomial checkpointing, for which the paper gives

$\delta x_k$3

as the number of time steps optimally handled with $\delta x_k$4 snapshots and $\delta x_k$5 permitted duplicates per time step (0711.4444). With $\delta x_k$6 snapshots, the paper reports inflection points at $\delta x_k$7 and $\delta x_k$8, matching theory (0711.4444). This makes checkpointing an integral part of the adjoint tangent structure rather than a mere implementation detail.

The same structural issues recur in parallel time-dependent PDE solvers. In JAGUAR, a massively parallel CFD code differentiated by TAPENADE, the two novelties are “1) the adjoint code is obtained by letting the AD tool Tapenade invert the complete layer of message passing interface (MPI) communications, and 2) the adjoint code integrates time-dependent, non-linear and dissipative (hence physically irreversible) PDEs with an explicit time integration loop running for ca. $\delta x_k$9 time steps” (Cardesa et al., 2019). The approach relies on the Adjoinable MPI library and on binomial checkpointing. The paper reports that for a viscous double shear layer case on 16 cores, a finite-difference sensitivity costs about $$\delta x_{k+1} = \frac{\partial M_k}{\partial x_k}(x_k,p)\,\delta x_k + \frac{\partial M_k}{\partial p}(x_k,p)\,\delta p.$$0 forward runs, tangent AD costs about $$\delta x_{k+1} = \frac{\partial M_k}{\partial x_k}(x_k,p)\,\delta x_k + \frac{\partial M_k}{\partial p}(x_k,p)\,\delta p.$$1, and adjoint AD about $$\delta x_{k+1} = \frac{\partial M_k}{\partial x_k}(x_k,p)\,\delta x_k + \frac{\partial M_k}{\partial p}(x_k,p)\,\delta p.$$2 relative to one primal run; tangent and adjoint agree to double precision, while finite differences differ in the last digits as expected (Cardesa et al., 2019).

MITgcm-AD v2 places the same ideas in a community ocean–atmosphere framework. There, the forward model, TLM, and adjoint are organized as

$$\delta x_{k+1} = \frac{\partial M_k}{\partial x_k}(x_k,p)\,\delta x_k + \frac{\partial M_k}{\partial p}(x_k,p)\,\delta p.$$3

with Tapenade-based source transformation, custom treatment of I/O and MPI exchanges, and binomial checkpointing integrated into the build system (Gaikwad et al., 2024). The motivation is practical: tangent-linear and adjoint models provide “gradient information which enables dynamics-based sensitivity and attribution studies, state and parameter estimation, and rigorous uncertainty quantification” (Gaikwad et al., 2024). In ECCO-style applications, they are essential for efficient large-scale data assimilation (Gaikwad et al., 2024).

A plausible synthesis is that in computational science the term “adjoint tangent structure” refers to a three-layer object: the linear algebra of Jacobians and transposes, the code-generation machinery that respects the full discrete algorithm, and the systems engineering needed to make reverse-time differentiation feasible at operational scale.

4. Manifold and Lie-theoretic uses: tangent spaces, adjoint actions, and bundle geometry

Outside computational differentiation, the phrase acquires a more geometric meaning. In tangent space backpropagation for $$\delta x_{k+1} = \frac{\partial M_k}{\partial x_k}(x_k,p)\,\delta x_k + \frac{\partial M_k}{\partial p}(x_k,p)\,\delta p.$$4, $$\delta x_{k+1} = \frac{\partial M_k}{\partial x_k}(x_k,p)\,\delta x_k + \frac{\partial M_k}{\partial p}(x_k,p)\,\delta p.$$5, and $$\delta x_{k+1} = \frac{\partial M_k}{\partial x_k}(x_k,p)\,\delta x_k + \frac{\partial M_k}{\partial p}(x_k,p)\,\delta p.$$6, the relevant structure is the Lie-group tangent space together with the adjoint representation $$\delta x_{k+1} = \frac{\partial M_k}{\partial x_k}(x_k,p)\,\delta x_k + \frac{\partial M_k}{\partial p}(x_k,p)\,\delta p.$$7, which moves tangent vectors between group elements and coordinate frames (Teed et al., 2021). For a group element $$\delta x_{k+1} = \frac{\partial M_k}{\partial x_k}(x_k,p)\,\delta x_k + \frac{\partial M_k}{\partial p}(x_k,p)\,\delta p.$$8,

$$\delta x_{k+1} = \frac{\partial M_k}{\partial x_k}(x_k,p)\,\delta x_k + \frac{\partial M_k}{\partial p}(x_k,p)\,\delta p.$$9

and it satisfies

$\lambda_k$0

(Teed et al., 2021). In composition $\lambda_k$1, the Jacobians in tangent coordinates are

$\lambda_k$2

so reverse-mode gradients on manifolds are transported by adjoint maps rather than by Euclidean embeddings (Teed et al., 2021). The paper describes this explicitly as “tangent space backpropagation,” and a plausible implication is that here “adjoint tangent structure” names the combination of tangent-space differentiation and adjoint frame transport.

In the geometric algebra paper on bivectors, the phrase refers to the commutator action of a bivector $\lambda_k$3 on vectors and multivectors. The adjoint operation is

$\lambda_k$4

and on vectors it is the grade-preserving map $\lambda_k$5 (Eelbode et al., 2024). The paper connects the eigenstructure of this adjoint action to the outer exponential $\lambda_k$6, an “outer tangent” $\lambda_k$7, and a factorized Cayley–Hamilton theorem (Eelbode et al., 2024). There, “adjoint tangent structure” designates a spectral and infinitesimal structure underlying spin-group transformations.

The phrase also arises in the geometry of adjoint orbits. For adjoint orbits of $\lambda_k$8, the paper studies geodesics by identifying the orbit $\lambda_k$9 with the cotangent bundle $$\lambda_k = \left(\frac{\partial M_k}{\partial x_k}(x_k,p)\right)^{T}\lambda_{k+1} + \frac{\partial J}{\partial x_k}(x_k,p)^T,$$0 of an $$\lambda_k = \left(\frac{\partial M_k}{\partial x_k}(x_k,p)\right)^{T}\lambda_{k+1} + \frac{\partial J}{\partial x_k}(x_k,p)^T,$$1-flag manifold, and then with $$\lambda_k = \left(\frac{\partial M_k}{\partial x_k}(x_k,p)\right)^{T}\lambda_{k+1} + \frac{\partial J}{\partial x_k}(x_k,p)^T,$$2 via an invariant metric (Prado et al., 2022). This yields an “adjoint tangent structure” in which the adjoint orbit of the non-compact group is reinterpreted as a tangent-bundle geometry over the compact adjoint orbit. Geodesics are then described using the Sasaki metric on the tangent bundle and the coupled equations

$$\lambda_k = \left(\frac{\partial M_k}{\partial x_k}(x_k,p)\right)^{T}\lambda_{k+1} + \frac{\partial J}{\partial x_k}(x_k,p)^T,$$3

(Prado et al., 2022).

A further geometric use appears in the wonderful compactification of an adjoint group. For a simple complex algebraic group $$\lambda_k = \left(\frac{\partial M_k}{\partial x_k}(x_k,p)\right)^{T}\lambda_{k+1} + \frac{\partial J}{\partial x_k}(x_k,p)^T,$$4 of adjoint type, the tangent bundle $$\lambda_k = \left(\frac{\partial M_k}{\partial x_k}(x_k,p)\right)^{T}\lambda_{k+1} + \frac{\partial J}{\partial x_k}(x_k,p)^T,$$5 of its wonderful compactification is a $$\lambda_k = \left(\frac{\partial M_k}{\partial x_k}(x_k,p)\right)^{T}\lambda_{k+1} + \frac{\partial J}{\partial x_k}(x_k,p)^T,$$6-equivariant bundle whose fiber at the identity realizes the adjoint representation on $$\lambda_k = \left(\frac{\partial M_k}{\partial x_k}(x_k,p)\right)^{T}\lambda_{k+1} + \frac{\partial J}{\partial x_k}(x_k,p)^T,$$7 (Biswas et al., 2013). Because this fiber representation is irreducible and because $$\lambda_k = \left(\frac{\partial M_k}{\partial x_k}(x_k,p)\right)^{T}\lambda_{k+1} + \frac{\partial J}{\partial x_k}(x_k,p)^T,$$8 is a $$\lambda_k = \left(\frac{\partial M_k}{\partial x_k}(x_k,p)\right)^{T}\lambda_{k+1} + \frac{\partial J}{\partial x_k}(x_k,p)^T,$$9-equivariant compactification, the paper proves that $J$0 is stable with respect to every polarization (Biswas et al., 2013). Here the “adjoint tangent structure” is representation-theoretic: the tangent fiber at the identity is literally the adjoint module.

5. Categorical adjoint tangent structures and universal tangent formalisms

In category theory, “adjoint tangent structure” becomes literal: it refers to tangent bundle functors that possess adjoints or dual tangent structures. The starting point is the Cockett–Cruttwell tangent structure, consisting of a tangent functor $J$1, a projection $J$2, a zero section $J$3, addition $J$4, vertical lift $J$5, and canonical flip $J$6, satisfying a family of axioms in $J$7 (Leung, 2016). The paper “The free tangent structure” shows that giving such a structure on $J$8 is equivalent to giving a strong monoidal functor

$J$9

preserving foundational pullbacks and a distinguished equaliser (Leung, 2016). This makes $P$0 the free or initial tangent structure.

The dualization theme becomes explicit in the work on $P$1-toposes. There, Lurie’s tangent bundle functor on presentable $P$2-categories yields a tangent structure on $P$3, while an adjoint functor $P$4 yields a geometric tangent structure on $P$5 itself (Ching, 2021). The paper states that Lurie’s tangent bundle functor $P$6 has a left adjoint $P$7, and that the opposite $P$8 underlies a tangent structure (Ching, 2021). For injective $P$9-toposes,

$Y=F(X)$0

so the adjoint tangent bundle is computed by applying the Goodwillie/Lurie tangent bundle to the $Y=F(X)$1-category of points (Ching, 2021). In this setting, “adjoint tangent structure” is an exact duality between algebraic and geometric tangent bundle functors.

A similar pattern appears for algebras over an operad. The category of $Y=F(X)$2-algebras has tangent bundle

$Y=F(X)$3

given by semi-direct product, while the opposite category has a tangent bundle

$Y=F(X)$4

built from Kähler differentials (Ikonicoff et al., 2023). These two functors are adjoint: $Y=F(X)$5 and both carry full tangent category structure (Ikonicoff et al., 2023). The divided power algebra paper later reproduces this pattern in a different algebraic context: the semidirect-product tangent structure on divided power algebras admits an adjoint tangent structure on the opposite category, again described by Kähler-differential-type data (Ikonicoff, 22 Aug 2025).

The formal theory of tangentads generalizes these ideas two-categorically. A tangentad is to a tangent category what a formal monad is to a monad; the paper constructs a 2-comonad on the 2-functor of tangentads and introduces adjunctable and representable tangentads (Lanfranchi, 24 Mar 2025). An adjunctable tangentad is one for which $Y=F(X)$6 has left adjoints $Y=F(X)$7, allowing mate constructions to define an op-tangent structure (Lanfranchi, 24 Mar 2025). A representable tangentad is one for which $Y=F(X)$8 is right adjoint to $Y=F(X)$9 for suitable infinitesimal objects $\dot Y = F'(X)\dot X$0 (Lanfranchi, 24 Mar 2025). These results suggest that, in formal category theory, an adjoint tangent structure is a tangent structure whose tangent powers themselves participate in a system of adjunctions.

6. Conceptual synthesis and recurring themes

Across the literature, “adjoint tangent structure” consistently combines three ingredients. The first is a tangent object: Jacobian action in numerical models, tangent spaces in Lie groups, tangent bundles in geometry, or tangent functors in category theory. The second is an adjoint object: transposed Jacobian, reverse-mode sensitivity, adjoint representation, or a left/right adjoint tangent functor. The third is a compatibility principle ensuring that the two constructions are not independent but dual.

Several recurring themes are visible.

Forward perturbations versus backward sensitivities: in numerical weather prediction, ocean modeling, and CFD, the tangent model propagates perturbations and the adjoint propagates sensitivities, typically for 4D-Var or optimization (Xiao et al., 17 Mar 2026, Gaikwad et al., 2024, Cardesa et al., 2019).

Black-box encapsulation versus structural exactness: TorchNWP shields the internal neural-network architecture while still exposing tangent and adjoint interfaces (Xiao et al., 17 Mar 2026). By contrast, OPA, MITgcm, and JAGUAR emphasize exact differentiation of the discrete code, including solvers, arrays, and MPI (0711.4444, Gaikwad et al., 2024, Cardesa et al., 2019). This suggests a continuum between encapsulated derivative services and full algorithmic transparency.

Memory, reversibility, and checkpointing: reverse-mode tangent structures are rarely purely formal; they require storage or recomputation policies. The OPA and JAGUAR papers make checkpointing a constitutive part of adjoint design (0711.4444, Cardesa et al., 2019).

Representation-theoretic and geometric duality: in adjoint groups, adjoint orbits, and Lie-algebra actions, the tangent structure at a point is controlled by an adjoint representation or an adjoint action (Biswas et al., 2013, Prado et al., 2022, Eelbode et al., 2024).

Universal and opposite constructions: categorical papers repeatedly realize tangent structure through free, dual, or opposite constructions, often with explicit adjunctions (Leung, 2016, Ching, 2021, Ikonicoff et al., 2023, Lanfranchi, 24 Mar 2025, Ikonicoff, 22 Aug 2025).

A common misconception is that “adjoint tangent structure” has a unique technical definition. The literature does not support that. The term is domain-relative. In computational science it usually means the pair of tangent-linear and adjoint models associated with a nonlinear code. In higher category theory it may mean a tangent structure whose tangent functor has an adjoint or whose opposite category inherits a dual tangent structure. In geometry it can denote tangent-bundle realizations attached to adjoint actions or adjoint orbits. What unifies these usages is not terminology but structure: a first-order forward object, a dual backward object, and a principled relation between them.