Linearizer Framework: Inducing Neural Linearity

• Linearizer Framework is a redefinition of linearity that induces vector space structures via invertible neural networks to enable exact linear mappings.
• It allows the use of classical linear algebra tools such as SVD, pseudoinverse, and orthogonal projections on neural network transformations.
• Practical applications include one-step diffusion sampling, idempotent generative models, and modular style transfer through compositional linear combinations.

The Linearizer Framework defines a class of neural architectures and induced coordinate systems in which conventionally nonlinear mappings are rendered exactly linear by construction. By composing a linear operator with invertible neural networks, the framework endows the domain and codomain with novel vector space structures whose algebraic operations are determined via the invertible maps. Relative to these non-standard vector spaces, the overall function admits the full suite of linear algebraic tools—SVD, pseudo-inverse, orthogonal projection, operator composition, and more—even though it may appear highly nonlinear in the standard coordinates. This enables principled decomposition and manipulation of neural mappings, compositionality across network modules, and direct design of properties such as idempotency and modular transfer between tasks.

1. Definition and Construction of Linearizer

The Linearizer Framework commences by redefining the concept of linearity. For an invertible neural network $g:V\to\mathbb{R}^n$, the framework defines induced addition and scalar multiplication on $V$ by pulling back the standard (Euclidean) operations: $$v_1 \oplus_{g} v_2 = g^{-1}(g(v_1) + g(v_2)),\qquad a \odot_{g} v = g^{-1}(a \cdot g(v)).$$ Let $g_x:\mathcal{X}\to\mathbb{R}^n$ and $g_y:\mathcal{Y}\to\mathbb{R}^m$ be invertible neural networks (with suitable regularity and bijectivity). Given a linear operator $A:\mathbb{R}^n\to\mathbb{R}^m$, the framework constructs a Linearizer mapping

$f(x) = g_y^{-1}(A\, g_x(x)),$

where $$f:(\mathcal{X}, \oplus_{g_x}, \odot_{g_x}) \to (\mathcal{Y}, \oplus_{g_y}, \odot_{g_y})$$ is exactly linear.

In this setting, $f$ is nonlinear in the standard algebraic structure but linear with respect to the operations induced by $g_x$ and $g_y$. Thus, the functional structure decomposes into three interpretable parts:

• $g_x$: nonlinear encoding into a "latent" space,
• $A$: linear transformation in latent coordinates,
• $g_y^{-1}$: nonlinear decoding back to the target space.

2. Linear Algebra Operations and Induced Hilbert Structures

Once linearity is transferred into the induced vector spaces, all standard linear algebra operations apply:

• Singular Value Decomposition (SVD): The SVD of $f$ is given by pulling back the singular vectors of $A$:

$$A = U\Sigma V^\top \implies \text{singular vectors of } f:~\tilde{u}_i = g_y^{-1}(u_i),~\tilde{v}_i = g_x^{-1}(v_i).$$

• Moore–Penrose Pseudoinverse: The framework defines the adjoint and the pseudoinverse as

$f^\dagger(y) = g_x^{-1}(A^\dagger g_y(y)).$

This enables the exact inversion of $f$ on the image space and analytic design of invertible or partially invertible neural architectures.

• Inner Product and Orthogonal Projections: The inner product in induced coordinates is $$\langle v_1, v_2 \rangle_{g_x} = \langle g_x(v_1), g_x(v_2)\rangle$$, rendering $(\mathcal{X}, \oplus_{g_x}, \odot_{g_x})$ and $(\mathcal{Y}, \oplus_{g_y}, \odot_{g_y})$ genuine Hilbert spaces. This enables direct projection, eigenvalue analysis, and spectral manipulations.

Thus, the Linearizer emulates the full analytic tractability of linear operators in a highly expressive, learned coordinate system.

3. Compositionality and Composition of Linearizers

A critical property of the Linearizer Framework is closure under composition. Let

$$f_1(x) = g_y^{-1}(A_1 g_x(x)), \qquad f_2(y) = g_z^{-1}(A_2 g_y(y)).$$

Then

$f_2 \circ f_1(x) = g_z^{-1}(A_2 A_1 g_x(x)),$

which is again a Linearizer from $(\mathcal{X}, g_x)$ to $(\mathcal{Z}, g_z)$. This closure property permits deep stacking, modular design, and algebraic tractability of network compositions, since each layer can be analyzed via its core linear operator. In particular, the composition reduces to multiplication of the corresponding $A$ matrices in the induced latent spaces.

This compositionality allows, for example, efficient "collapsing" of iterative update steps (as in diffusion models) into a single linear transformation, as demonstrated in the paper.

4. Practical Applications

A. One-Step Diffusion Model Sampling

In diffusion and flow-based generative models, inference typically requires multiple iterative updates; in the Linearizer setting, each update in latent space is linear: $g(x_{t+1}) = (I + \Delta t\, A_t) g(x_t).$ By composing the updates, a sequence of hundreds of steps reduces to a single mapping $x_1 = g^{-1}(B\, g(x_0))$, where $B$ is the product of $(I + \Delta t\, A_t)$ over steps. This leads to the possibility of exact one-step sampling through a learned invertible coordinate system.

B. Idempotent Generative Models

Enforcing $f(f(x)) = f(x)$ (idempotency) corresponds to requiring $A^2 = A$ in the latent space. This yields neural models that are globally projective, providing generative mappings with precise properties (e.g., style transfer via projection onto a subspace).

C. Modular Style Transfer

The framework enables modular style injection via parameterizing each "style" as its own $A_{\text{style}}$, while holding $g_x$, $g_y$ fixed. Mixed or interpolated styles are simply linear combinations of $A_{\text{style}}$ (i.e., $A = \lambda_1 A_1 + \lambda_2 A_2$), without retraining or entanglement with the main invertible architectures.

5. Theoretical and Methodological Implications

The Linearizer Framework demonstrates that nonlinearity is relative to the choice of underlying algebraic operations and coordinate systems. By making the underlying vector space structures explicit and learned, the framework enables:

• Analytic expressivity: Nonlinear mappings can be described exactly as linear with respect to induced operations.
• Full utilization of linear operator theory: SVD, projections, pseudoinverse, dynamics, and more are all accessible.
• The potential for stability guarantees and interpretable neural network design, since classical results lift to the induced spaces.
• Potential integration with Koopman operator learning and normalizing flows, suggesting a possible unification of operator learning paradigms.

6. Future Directions

Suggested research avenues include:

• Scaling the Linearizer Framework to large-scale, high-resolution generative models where analytic tractability could allow rapid sampling and inversion.
• Exploring continuous-time models using matrix exponentials in latent coordinates, which could model dynamical systems directly.
• Investigating the space of functions representable by Linearizers, their universal approximation properties under constrained $g_x$, $g_y$, and $A$, and the theoretical links to invertible architectures and operator-theoretic frameworks.
• Developing modular, compositional architectures for tasks such as transfer learning, domain adaptation, and style transfer, taking advantage of the analytic decomposability.

7. Key Formulas

Below is a summary table of central mathematical aspects:

Concept	Formula	Context
Linearizer Map	$f(x) = g_y^{-1}(A g_x(x))$	Core construction
Induced Addition/Scaling	$v_1 \oplus_g v_2 = g^{-1}(g(v_1) + g(v_2))$, $a \odot_g v = g^{-1}(a g(v))$	Induced vector space
SVD of $f$	$\tilde{u}_i = g_y^{-1}(u_i)$, $\tilde{v}_i = g_x^{-1}(v_i)$	Pullback of $A$'s decomposition
Pseudoinverse	$f^\dagger(y) = g_x^{-1}(A^\dagger g_y(y))$	Exact inversion
Idempotency	$f(f(x)) = f(x) \iff A^2 = A$	Projective models
Composition	$f_2(f_1(x)) = g_z^{-1}(A_2 A_1 g_x(x))$	Stacking Linearizers

By unifying expressive invertible networks and classical linear operators via coordinate induction, the Linearizer Framework opens a new paradigm in the design and analysis of neural architectures, balancing nonlinear flexibility with the algebraic tractability of linear systems.