Glow: Invertible Flow-Based Model

• Glow is an invertible flow-based generative model that maps images to a latent space using a tractable change-of-variables framework for exact density evaluation.
• It utilizes a three-part flow step—ActNorm, invertible 1x1 convolutions, and affine coupling layers—in a multi-scale architecture to balance memory usage and capture image structure.
• Glow enables efficient training and real-time high-resolution image synthesis while providing a semantically smooth latent space for controllable image manipulations.

Glow usually denotes the flow-based generative model introduced in "Glow: Generative Flow with Invertible 1x1 Convolutions" (Kingma et al., 2018). It defines an invertible mapping $f_1 \circ f_2 \circ \cdots \circ f_K$ from data $x \in \mathbb{R}^{h \times w \times c}$ to latent $z$, making exact density evaluation, exact latent-variable inference, and parallelizable training and synthesis simultaneously tractable. Within the class of flow-based generative models, Glow is characterized by a simple three-part flow step—ActNorm, an invertible $1\times1$ convolution, and an affine coupling layer—combined with a multi-scale architecture and implementation choices aimed at stabilizing very deep, high-resolution models (Kingma et al., 2018).

1. Formal definition and probabilistic structure

Glow models the data distribution by an invertible transformation from image space to a latent space with a simple tractable prior, such as $\mathcal{N}(0,I)$. Exact log-density follows from the change-of-variables formula:

$$\log p_X(x) = \log p_Z(z) + \sum_{i=1}^K \log \left| \det \frac{\partial f_i(h_{i-1})}{\partial h_{i-1}} \right|.$$

Here $h_0=x$, the intermediate states are produced by the sequence of invertible transformations, and $z$ is the final latent representation (Kingma et al., 2018).

This formulation makes the model analytically different from approximate-likelihood or implicit generators: density evaluation is exact, latent inference is exact because the mapping is invertible, and synthesis is obtained by sampling from the prior and applying the inverse transformation. The architecture is therefore organized so that every constituent operation has an inexpensive inverse and a tractable log-Jacobian determinant.

2. Invertible building blocks of a Glow step

Each Glow flow step interleaves three inexpensive invertible operations (Kingma et al., 2018).

ActNorm. Glow applies a per-channel scale $s \in \mathbb{R}^c$ and bias $b \in \mathbb{R}^c$:

$x \in \mathbb{R}^{h \times w \times c}$0

The inverse is $x \in \mathbb{R}^{h \times w \times c}$1, and the log-Jacobian is

$x \in \mathbb{R}^{h \times w \times c}$2

Unlike batch-norm, ActNorm is initialized on a single minibatch to make each channel zero-mean and unit-variance, after which $x \in \mathbb{R}^{h \times w \times c}$3 and $x \in \mathbb{R}^{h \times w \times c}$4 are trained as free parameters. This is used to stabilize very deep flows even with one-image batches per GPU.

Invertible $x \in \mathbb{R}^{h \times w \times c}$5 convolution. Glow replaces fixed channel permutations with a learned dense channel-mixing transform:

$x \in \mathbb{R}^{h \times w \times c}$6

where $x \in \mathbb{R}^{h \times w \times c}$7. The inverse is $x \in \mathbb{R}^{h \times w \times c}$8, and the log-Jacobian is

$x \in \mathbb{R}^{h \times w \times c}$9

Because computing $z$0 is $z$1, Glow optionally parameterizes $z$2 by an LU decomposition,

$z$3

reducing the per-forward-pass cost from $z$4 to $z$5.

Affine coupling layer. The channels are split as $z$6. A small convolutional network computes $z$7, and the transformation is

$z$8

Inversion is trivial,

$z$9

and the Jacobian contribution is $1\times1$0. Glow uses zero initialization on the last convolution in this network so that each coupling layer starts as the identity, easing optimization of very deep models.

3. Channel mixing, hierarchy, and multi-scale factorization

The invertible $1\times1$1 convolution is the architectural feature that most sharply distinguishes Glow from earlier fixed-permutation flow designs. Learning a dense $1\times1$2, initialized to a random rotation, substantially increases mixing across channels compared to fixed or random permutations, and this yields faster convergence and better final likelihoods (Kingma et al., 2018).

Glow also introduces a multi-scale hierarchy. It stacks $1\times1$3 flow steps per level and has $1\times1$4 levels. After each level, half of the channels are “squeezed” out and factored into the prior, while the remainder are passed to the next level. This splitting reduces memory growth and encourages hierarchical latents at multiple spatial resolutions. The resulting latent representation is therefore distributed across scales rather than concentrated in a single terminal tensor.

The reported qualitative behavior is consistent with this design. Deeper models, specifically $1\times1$5 versus $1\times1$6, capture longer-range coherence. This suggests that Glow’s hierarchy is not only a memory-management device but also a mechanism for progressively modeling coarse-to-fine image structure.

4. Optimization and implementation regime

Glow is trained by maximum likelihood on dequantized data of the form $1\times1$7 using Adam with learning rate $1\times1$8, $1\times1$9, and $\mathcal{N}(0,I)$0 (Kingma et al., 2018). Typical minibatch sizes are $\mathcal{N}(0,I)$1 for CIFAR-10, $\mathcal{N}(0,I)$2 for ImageNet32, and as low as $\mathcal{N}(0,I)$3 image per GPU at $\mathcal{N}(0,I)$4 resolution.

Several engineering choices are integral to the training recipe. ActNorm replaces batch-dependent normalization and is explicitly intended to support extremely small effective batch sizes. LU factorization of $\mathcal{N}(0,I)$5 makes determinant evaluation cheap. Zero initialization in the coupling networks makes each layer initially close to the identity map. Gradient checkpointing is used for stability and memory efficiency at high resolution. No additional regularization, including weight decay or dropout, is needed.

For sample quality, Glow uses a lower-bit representation, specifically $\mathcal{N}(0,I)$6-bit, together with sampling at reduced temperature $\mathcal{N}(0,I)$7, written as $\mathcal{N}(0,I)$8. Sampling at reduced temperature, for example $\mathcal{N}(0,I)$9, is reported to provide the best balance between diversity and sharpness.

5. Quantitative performance

Glow is reported to set a new state of the art among non-autoregressive flows on several natural-image benchmarks trained with $$\log p_X(x) = \log p_Z(z) + \sum_{i=1}^K \log \left| \det \frac{\partial f_i(h_{i-1})}{\partial h_{i-1}} \right|.$$0-bit data (Kingma et al., 2018).

Dataset	Glow bpd	Comparison
CIFAR-10	3.35	RealNVP 3.49
ImageNet 32×32	4.09	4.28
ImageNet 64×64	3.81	3.98
LSUN Bedroom 64×64	2.38	2.72
LSUN Tower 64×64	2.46	2.81
LSUN Church 64×64	2.67	3.08

With $$\log p_X(x) = \log p_Z(z) + \sum_{i=1}^K \log \left| \det \frac{\partial f_i(h_{i-1})}{\partial h_{i-1}} \right|.$$1-bit dequantization, Glow further achieves $$\log p_X(x) = \log p_Z(z) + \sum_{i=1}^K \log \left| \det \frac{\partial f_i(h_{i-1})}{\partial h_{i-1}} \right|.$$2 bpd on CIFAR-10, $$\log p_X(x) = \log p_Z(z) + \sum_{i=1}^K \log \left| \det \frac{\partial f_i(h_{i-1})}{\partial h_{i-1}} \right|.$$3 bpd on ImageNet $$\log p_X(x) = \log p_Z(z) + \sum_{i=1}^K \log \left| \det \frac{\partial f_i(h_{i-1})}{\partial h_{i-1}} \right|.$$4, $$\log p_X(x) = \log p_Z(z) + \sum_{i=1}^K \log \left| \det \frac{\partial f_i(h_{i-1})}{\partial h_{i-1}} \right|.$$5 on ImageNet $$\log p_X(x) = \log p_Z(z) + \sum_{i=1}^K \log \left| \det \frac{\partial f_i(h_{i-1})}{\partial h_{i-1}} \right|.$$6, and $$\log p_X(x) = \log p_Z(z) + \sum_{i=1}^K \log \left| \det \frac{\partial f_i(h_{i-1})}{\partial h_{i-1}} \right|.$$7 on CelebA-HQ $$\log p_X(x) = \log p_Z(z) + \sum_{i=1}^K \log \left| \det \frac{\partial f_i(h_{i-1})}{\partial h_{i-1}} \right|.$$8 (Kingma et al., 2018).

The paper also emphasizes throughput and resolution. Glow produces high-fidelity samples at up to $$\log p_X(x) = \log p_Z(z) + \sum_{i=1}^K \log \left| \det \frac{\partial f_i(h_{i-1})}{\partial h_{i-1}} \right|.$$9 resolution in real time, reported as $h_0=x$0 s/image on a Titan 1080 Ti. A plausible implication is that the model’s architectural tractability was not merely theoretical; it translated into practical synthesis at resolutions that were significant for flow models at the time.

6. Latent-space geometry, image manipulation, and name reuse

Glow is presented not only as a likelihood model but also as a generative model with a smooth, semantically meaningful latent space (Kingma et al., 2018). Linear interpolation between latent codes of real images produces gradual face morphing without collapse. Post-hoc editing is performed by traversing learned “attribute vectors,” computed as the difference between the mean latent code for positives and negatives of a label. This enables manipulation of smiles, hair color, gender, and age without retraining the generative flow.

The same invertible backbone also motivated later conditional extensions. "Full-Glow: Fully conditional Glow for more realistic image generation" (Sorkhei et al., 2020) uses a paired source-and-target Glow architecture in which conditioning information modulates actnorm, invertible $h_0=x$1 convolution, and affine coupling at every step, indicating that Glow’s invertible design could be repurposed for label-to-image generation.

The name itself is polysemous in the arXiv literature. Distinct works use GLOW or GLoW for unrelated topics, including "Glow: Graph Lowering Compiler Techniques for Neural Networks" (Rotem et al., 2018), "GLOW : Global Weighted Self-Attention Network for Web Search" (Shan et al., 2020), "Gamma Ray Glow Observations at 20-km Altitude" (Østgaard et al., 2021), "GLoW: novel methods for wave-optics phenomena in gravitational lensing" (Villarrubia-Rojo et al., 2024), "GLOW: A Unified Particle Flow Transformer" (Kobylianskii et al., 27 Aug 2025), and "GLOW: Graph-Language Co-Reasoning for Agentic Workflow Performance Prediction" (Guan et al., 11 Dec 2025). In machine learning, however, the unqualified term “Glow” most often refers to the invertible generative flow model of Kingma and Dhariwal (Kingma et al., 2018).

7. Position within flow-based generative modeling

Glow occupies a specific position within flow-based generative modeling: it preserves exact likelihood and exact invertibility while introducing a learned channel-mixing operator, a stable normalization scheme for deep flows, zero-initialized affine couplings, and a multi-scale factorization strategy (Kingma et al., 2018). The model’s reported behavior links these design choices to both quantitative and qualitative outcomes: better log-likelihoods than prior non-autoregressive flows, efficient high-resolution synthesis, and latent manipulations that remain semantically coherent.

Taken together, Glow’s contribution is less a single isolated mechanism than a coordinated recipe: ActNorm, invertible $h_0=x$2 convolutions, zero-initialized affine couplings, multi-scale splitting, LU parameterization of $h_0=x$3, and reduced-batch normalization. Within the evidence reported for the model, this combination yields a fast, expressive, and fully invertible generative model that advances both density estimation and controllable image synthesis (Kingma et al., 2018).