Full-Glow: Conditional Flow for Scene Synthesis

• Full-Glow is a conditional normalizing flow architecture that generates photo-realistic street scenes from semantic segmentation maps.
• It employs hierarchical conditioning by integrating corresponding source features at every transformation step to improve semantic fidelity and detail.
• Benchmark results demonstrate superior segmentation metrics, with a mean pixel accuracy of 73.50%, supporting its utility in synthetic data generation for autonomous driving.

Searching arXiv for Full-Glow and related conditional Glow papers. Full-Glow is a fully conditional Glow-based architecture for semantic label map $\rightarrow$ photo translation, designed to generate plausible and realistic images of novel street scenes from a semantic segmentation map indicating scene layout. It was introduced as a flow-based alternative for controllable synthetic data generation in settings such as autonomous driving, where large amounts of labeled visual data are required. The central claim is that conditioning the target Glow hierarchy at every operation on corresponding source-domain features yields images that are more similar to real images of the same scenes and objects, as measured by downstream semantic segmentation performance with a pretrained PSPNet (Sorkhei et al., 2020).

1. Problem setting and conceptual position

Full-Glow addresses conditional image generation in the setting where the source domain is a semantic representation, such as a segmentation mask, and the target domain is a photorealistic RGB image. The intended use case is the generation of synthetic street-scene imagery with explicit control over scene layout and ground-truth labeling, so that the generated images can augment real datasets for visual semantic segmentation or object recognition (Sorkhei et al., 2020).

The model is presented in relation to prior conditional flow architectures. In the formulation summarized for Full-Glow, Glow models are invertible deep generative models composed of a sequence of bijective transformations. Prior conditional variants include C-Glow, where sublayers in standard Glow become conditional on the raw conditioning image; Dual-Glow, which jointly models condition and output and links latent variables via neural networks; and C-Flow, where coupling layers in one flow are conditioned on intermediate features in the other. Full-Glow distinguishes itself by making the conditioning hierarchical and ubiquitous: every operation in the target Glow is conditioned on source Glow features at the corresponding level, rather than only on raw input or a more limited set of latent interactions (Sorkhei et al., 2020).

A common simplification is to describe Full-Glow as merely a conditional Glow model. That description is incomplete. The specific architectural claim is stronger: the conditioning applies to actnorm, $1\times1$ convolution, and coupling layers throughout the target hierarchy, with dedicated conditioning networks generating the relevant parameters from source representations (Sorkhei et al., 2020).

2. Architecture and conditioning pathway

The architecture consists of a source Glow and a target Glow. The source Glow processes the conditioning input, such as a semantic segmentation map, and exposes intermediate representations at successive levels of the flow. The target Glow models the conditional distribution of the RGB image, and every operation within it is conditioned on the output of the corresponding source Glow step (Sorkhei et al., 2020).

This conditioning is implemented through dedicated conditioning networks (CNs). For each operation in the target Glow stack, a CN receives the source Glow representation at the same level and produces the parameters required by the target operation. This applies to conditional actnorm, conditional $1\times1$ convolution, and conditional coupling layers. The summarized description emphasizes that side information, such as boundary maps derived from instance labels, can be concatenated to the features used by the conditioning networks, making the design extensible to auxiliary spatial cues (Sorkhei et al., 2020).

The architectural motivation is to preserve semantic structure across scales. Conditioning only on the raw input makes it harder to inject level-specific scene information into deep layers. By contrast, hierarchical conditioning allows the target model to access source features at every resolution and block. This suggests that the model is intended not only to preserve coarse layout but also to stabilize local object boundaries, textures, and instance structure under a fully invertible generative process.

3. Mathematical formulation

The general flow model is written as a chain of invertible mappings:

$$x \xleftrightarrow{f_1} h_1 \xleftrightarrow{f_2} h_2 \ldots \xleftrightarrow{f_K} z .$$

Its density is

$$\log p_x(x) = \log p_z(z) + \sum_i \log\left|\det \left(\frac{dh_i}{dh_{i-1}}\right)\right| .$$

Full-Glow jointly models a pair of images, with $x_a$ denoting the source image and $x_b$ the target image. The reported training objective is

$$\frac{1}{N} \left[ -\sum_{n=1}^N \lambda \log p_{\theta}(x_a^{(n)}) - \sum_{n=1}^N \log p_{\phi}(x_b^{(n)} \mid x_a^{(n)}) \right] .$$

Here, $p_{\theta}(x_a)$ is an unconditional model of the source domain, $p_{\phi}(x_b \mid x_a)$ is the conditional model of the target domain, and $1\times1$0 tunes the relative focus on source and target modeling (Sorkhei et al., 2020).

The conditional operations are parameterized explicitly. For conditional actnorm,

$1\times1$1

For conditional $1\times1$2 convolution, the kernel factors from the LU decomposition are predicted as

$1\times1$3

For the conditional coupling layer, the target features are split and transformed using both target and source representations:

$1\times1$4

All concatenation is along the channel dimension. The reported design implication is that conditioning is not confined to latent priors or top-level blocks; it directly modulates the invertible transformations themselves (Sorkhei et al., 2020).

4. Sampling, controllability, and side information

To generate a target image $1\times1$5 conditioned on a source image $1\times1$6, the source image is encoded through the source Glow, a latent code $1\times1$7 is sampled from the base distribution, and the target image is decoded by the fully conditional target Glow while the conditioning networks consume the corresponding source features at each step (Sorkhei et al., 2020).

The sampling temperature is an explicit control variable. Lowering the sampling temperature, by scaling the latent variance, increases image fidelity at the cost of diversity, and the reported optimal classification performance is obtained at $1\times1$8 (Sorkhei et al., 2020).

Side information can be injected at each level through concatenation. The specific example described is the use of boundary maps derived from instance labels. In the summarized experiments, adding boundary maps and using deeper or wider architectures yields further gains, including up to $1\times1$9 pixel accuracy and $1\times1$0 mean class IoU in the deeper configuration (Sorkhei et al., 2020).

These design elements make Full-Glow a controllable generative model in a stronger sense than layout-only conditioning. The conditioning signal can include semantically meaningful auxiliary structure, and the sampling temperature provides a direct mechanism for moving along the fidelity-diversity trade-off.

5. Benchmark protocol and reported results

The experiments are reported on the Cityscapes dataset at $1\times1$1 and $1\times1$2 resolutions. The principal quantitative protocol follows earlier semantic image synthesis work: a pretrained PSPNet, trained on real Cityscapes images, is applied to the generated RGB images, and its predicted segmentation is compared with the ground-truth segmentation used as input condition. The rationale is that if the synthetic images are realistic and semantically faithful, a segmentation network trained on real imagery should recover the intended mask accurately (Sorkhei et al., 2020).

The reported metrics are mean pixel accuracy, mean class accuracy, mean class Intersection over Union, and conditional bits per dimension (BPD). The summarized benchmark values are:

Model	Cond. BPD	Mean pixel acc.
C-Glow v1	2.568	35.02 $1\times1$3 0.56
C-Glow v2	2.363	52.33 $1\times1$4 0.46
Dual-Glow	2.585	71.44 $1\times1$5 0.03
pix2pix	--	60.56 $1\times1$6 0.11
Full-Glow	2.345	73.50 $1\times1$7 0.13
Ground-truth	--	95.97

The full set of reported segmentation metrics further shows that Full-Glow attains mean class accuracy of 29.13 $1\times1$8 0.39 and mean class IoU of 23.86 $1\times1$9 0.30, compared with 23.91 $$x \xleftrightarrow{f_1} h_1 \xleftrightarrow{f_2} h_2 \ldots \xleftrightarrow{f_K} z .$$0 0.19 and 18.96 $$x \xleftrightarrow{f_1} h_1 \xleftrightarrow{f_2} h_2 \ldots \xleftrightarrow{f_K} z .$$1 0.17 for Dual-Glow, and 22.64 $$x \xleftrightarrow{f_1} h_1 \xleftrightarrow{f_2} h_2 \ldots \xleftrightarrow{f_K} z .$$2 0.21 and 16.42 $$x \xleftrightarrow{f_1} h_1 \xleftrightarrow{f_2} h_2 \ldots \xleftrightarrow{f_K} z .$$3 0.06 for pix2pix (Sorkhei et al., 2020).

The lower conditional BPD for Full-Glow is reported as evidence of a better fit to the conditional data distribution. The segmentation-based evaluation is also used to support the practical claim that the generated images are suitable as training data for semantic segmentation or object recognition systems (Sorkhei et al., 2020).

6. Significance, interpretation, and boundaries of the claim

The central significance attributed to Full-Glow is not merely photorealistic synthesis, but semantic fidelity under explicit structural control. The model’s images are reported to be, to a higher degree than those of competing models, similar to real images of the same kinds of scenes and objects. That is the basis for its proposed use in generative data augmentation for safety-critical vision tasks such as autonomous driving (Sorkhei et al., 2020).

A second significance claim concerns diversity. Because Full-Glow is a flow-based model covering a conditional distribution rather than an adversarial model optimized primarily for realism, the summarized interpretation is that it can provide diverse samples while avoiding GAN mode collapse. This suggests an advantage when generating varied but label-faithful training data, although the concrete quantitative evidence presented in the summary is the PSPNet-based segmentation benchmark rather than a dedicated diversity metric.

The boundaries of the evidence are also clear from the reported protocol. The main quantitative assessment is downstream semantic segmentation performance on synthesized images, not a direct human-perception study or a broad battery of generative metrics. A plausible implication is that Full-Glow’s strongest demonstrated property is semantic consistency with the conditioning signal under a real-image-trained parser, rather than an exhaustive characterization of visual realism in all senses.

Within that scope, Full-Glow occupies a specific position in conditional generative modeling: a fully conditional normalizing-flow architecture in which hierarchical source features modulate every target transformation, enabling controllable scene synthesis from semantic layouts and supporting the use of synthetic imagery as structured training data (Sorkhei et al., 2020).