---
title: Gaussian-Image Representation (GIR)
url: https://www.emergentmind.com/topics/gaussian-image-representation-gir
type: topic
---

# Gaussian-Image Representation (GIR)

Gaussian-Image Representation (GIR) denotes a continuous, explicit image model in which an image is represented as the superposition of a finite set of anisotropic 2D Gaussian primitives (“splats”). In the formulation introduced by "GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splatting" [2403.08551], GIR replaces large MLPs or feature-grid implicit neural representations (INRs) with analytic Gaussians and a permutation-invariant summation renderer, yielding fast fitting, extremely high rendering throughput, low GPU-memory footprints, and competitive rate–distortion performance when used as a codec.

## 1. Definition and terminological scope

The original motivation for GIR arose from the observation that INRs can offer high visual quality and fast rendering speeds with 10-1000 FPS, assuming sufficient GPU resources are available, but that this requirement often hinders their use on low-end devices with limited memory. GaussianImage therefore proposed image representation and compression by 2D Gaussian Splatting, with an explicit, analytic representation rather than a large neural decoder [2403.08551].

Within the 2D-image literature, GIR refers to representing an image as a sum of Gaussian primitives in image-plane coordinates, with continuous reconstruction and GPU-oriented splatting. Closely related work preserves the same core idea while changing optimization, initialization, editability, or downstream use. "MiraGe: Editable 2D Images using Gaussian Splatting" extends GaussianImage toward editable 2D images; "Large Images are Gaussians: High-Quality Large Image Representation with Levels of 2D Gaussian Splatting" adapts the formulation to very large images; "Fast 2DGS: Efficient Image Representation with Deep Gaussian Prior" and "Instant GaussianImage: A Generalizable and Self-Adaptive Image Representation via 2D Gaussian Splatting" focus on feed-forward or near-feed-forward initialization; and "Contour Information Aware 2D Gaussian Splatting for Image Representation" adds region-constrained rasterization for boundary preservation [2410.01521].

The acronym is not unique across arXiv. "GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization" uses GIR to denote 3D Gaussian Inverse Rendering rather than 2D Gaussian-Image Representation, and LongSplat defines a Gaussian-Image Representation that encodes 3D Gaussian parameters into a structured, image-like 2D format for online 3D Gaussian fusion and compression [2312.05133]. This suggests that the term is partly overloaded, even though the GaussianImage family has established a specific meaning centered on explicit 2D image modeling [2507.16144].

## 2. Mathematical formulation

In GaussianImage, each primitive $G_i$ is a 2D Gaussian defined by a center $\mu_i \in \mathbb{R}^2$, a positive-definite covariance $\Sigma_i \in \mathbb{R}^{2\times2}$, and a color weight $c_i \in \mathbb{R}^3$, with opacity merged into the color weight. The primitive and full-image reconstruction are

$$
G_i(x) = c_i \cdot \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right],
$$

and

$$
G(x) = \sum_{i=1}^N G_i(x)
= \sum_{i=1}^N c_i \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right].
$$

To guarantee $\Sigma_i \succ 0$ throughout training, GIR optimizes a factorized form of the covariance. Two decompositions are described as common: Cholesky factorization,
$$
\Sigma_i = L_i L_i^\top,\qquad
L_i =
\begin{bmatrix}
l_{i,1} & 0\\
l_{i,2} & l_{i,3}
\end{bmatrix},
$$
and rotation-scaling factorization,
$$
\Sigma_i = (R_i S_i)(R_i S_i)^\top.
$$
The GaussianImage paper reports that the Cholesky form is more robust under quantization, so each Gaussian is parameterized by $(\mu_{i,x},\mu_{i,y})$, $(l_{i,1},l_{i,2},l_{i,3})$, and color weight $(c_{i,R},c_{i,G},c_{i,B})$, for a total of 8 learnable scalars per primitive. During training, $\mu_i$ is unconstrained, or in $(-1,1)$ via $\tanh$, and $c_i \in \mathbb{R}^3$ is free [2403.08551].

Later variants often separate amplitude or opacity from color, while preserving the same superposition structure. MiraGe represents an image $I:\mathbb{R}^2\to\mathbb{R}^3$ by $N$ weighted 2D Gaussian blobs with mean $\mu_i$, covariance $\Sigma_i$, weight $w_i \in \mathbb{R}_+$, and color $c_i \in \mathbb{R}^3$, whereas Fast-2DGS uses mean, covariance, amplitude $\alpha_i$, and color vector $c_i$ [2410.01521]. This variation does not alter the basic ontology of GIR: a continuous image is reconstructed from anisotropic Gaussian basis functions [2512.12774].

## 3. Summation rendering and fitting

A defining distinction of GaussianImage is its rendering rule. Traditional 3D Gaussian Splatting uses $\alpha$-blending with depth sorting,
$$
C_i = \sum c_n \alpha_n T_n,\qquad
T_n=\prod_{m<n}(1-\alpha_m).
$$
GIR observes that for a single view there is no depth ambiguity, so Gaussians need not be sorted and $\alpha$-accumulation can be absorbed into a single weight. For pixel $p$, with
$$
\sigma_{i,p} = \tfrac12 (x_p-\mu_i)^\top \Sigma_i^{-1}(x_p-\mu_i),
$$
the rendered color is
$$
C_p = \sum_{i=1}^N c_i\,\exp(-\sigma_{i,p}).
$$
The resulting renderer has no sorting and no per-primitive $T_n$ term [2403.08551].

On the GPU, GaussianImage builds a tile-based index of which Gaussians influence which pixel blocks, using a confidence-interval radius, and then launches one CUDA thread per pixel that loops over only the local Gaussians. Since each pixel sums $O(K)$ terms and $K$ is independent of $N$ once the image is large, the rendering throughput is $O(HW\cdot K)$ with negligible overhead in $N$, achieving 1500–2000 FPS on an NVIDIA V100 regardless of $N$ up to hundreds of thousands [2403.08551].

Fitting is performed by minimizing an $L_2$ reconstruction loss,
$$
L_{\text{rec}} = \|I_{\text{orig}} - I_{\text{recon}}\|^2,
$$
over, for example, 50 000 steps with the Adan optimizer, using an initial learning rate of $1\mathrm{e}{-3}$ halved at 20 k and 40 k. No split/clone density control is needed in the original formulation, since the entire image plane is occupied. Gaussians are initialized with random $\mu_i$, small isotropic $\Sigma_i$, and random $c_i$ [2403.08551].

This renderer established the order-invariant, explicit alternative that later papers either preserve or modify. Fast-2DGS, for example, keeps a tile-based rasterization but replaces pure summation with a weighted average over the top-$K$ Gaussians covering each pixel, while Dehaze-GaussianImage reintroduces a volume-rendering-style compositing because it couples Gaussian rendering to an atmospheric scattering model [2512.12774].

## 4. Compression and codec formulation

GaussianImage does not stop at overfitting; it explicitly constructs an image codec. After overfitting, GIR applies a two-step pipeline consisting of quantization-aware fine-tuning and entropy encoding. In the first stage, $\mu_i$ is quantized to 16-bit float; the Cholesky parameters $l_{i,j}$ are quantized by $b$-bit asymmetric learned quantization via LSQ+,
$$
\hat l_{i,j} = \mathrm{round}\!\left(\mathrm{clamp}\!\left(\frac{l_{i,j}-\beta_j}{\gamma_j},0,2^b-1\right)\right),
$$
with reconstruction
$$
l_{i,j} \approx \hat l_{i,j}\cdot \gamma_j + \beta_j;
$$
and the color vectors are quantized by a cascade $M=2$ stage residual vector quantization with codebook size $B=8$, minimizing a commitment loss [2403.08551].

In the second stage, Asymmetric Numeral Systems (ANS) codes the quantized Cholesky parameters and the RVQ indices. The codec also allows optional partial bits-back coding to amortize the $N!$ symmetry of unordered Gaussians, yielding an additional rate saving of $\log((N-K)!)-\log(N-K)$ bits once $K$ initial items are coded. Rate–distortion is controlled by adjusting $b$, $B$, $M$, and the quantization schedule, and can be written as minimizing
$$
R + \lambda D,
$$
where $R$ is the ANS-coded bitstream length and $D$ is the MSE reconstruction [2403.08551].

The reported behavior is that GIR matches COIN and COIN++ in PSNR versus bpp, and that the bits-back variant even slightly outperforms them. The codec also facilitates decoding speeds of approximately 2000 FPS, and the paper reports preliminary proof of concept that partial bits-back coding surpasses COIN and COIN++ in performance [2403.08551].

## 5. Benchmarks, operating regimes, and scaling behavior

The original GaussianImage paper reports a representation snapshot on Kodak and codec results at low bitrate. The benchmarked values below summarize the canonical operating regime of the method [2403.08551].

| Setting | GaussianImage | Comparator |
|---|---|---|
| Image representation snapshot, $N\approx 30\,000$ | PSNR 44.08 dB; MS-SSIM 0.9985; training 106 s; memory 419 MiB; rendering 2092 FPS | I-NGP: 43.88 dB; 490 s; 1525 MiB; 1297 FPS |
| Codec, $\text{bpp}\approx 0.32$ | PSNR $\approx 25.66$ dB; MS-SSIM 0.915; decoding speed $\approx 942$ FPS on V100 | JPEG2000 27.28 dB; COIN 25.80 dB; COIN MS-SSIM 0.890; JPEG $\approx 615$ FPS on CPU |

These numbers substantiate the specific claim that GaussianImage attains approximately 5$\times$ faster fitting time, at least 3$\times$ lower GPU memory usage, and faster rendering than I-NGP while remaining competitive in final reconstruction quality [2403.08551].

Subsequent work has explored different operating regimes rather than a single benchmark point. "Large Images are Gaussians: High-Quality Large Image Representation with Levels of 2D Gaussian Splatting" addresses fitting large images with a large number of Gaussian points by combining a variant of representation and optimization strategy with a Level-of-Gaussian approach for coarse low-frequency initialization and fine high-frequency detail recovery. On STimage 9K with $N=3.5\times10^7$, the paper reports GaussianImage at PSNR $=29.86$ dB, memory $=20.47$ GB, FPS $=19.9$, and LIG at PSNR $=37.47$ dB, memory $=16.67$ GB, FPS $=20.2$ [2502.09039].

This broadens GIR from the original tens-of-thousands-of-Gaussians regime to datasets using tens of millions of Gaussians, showing that the same primitive-based representation can be adapted to very large images when the optimization strategy is modified [2502.09039].

## 6. Extensions, applications, and limitations

A major extension of GIR concerns editability. "MiraGe: Editable 2D Images using Gaussian Splatting" lifts each 2D Gaussian into 3D as a flat Gaussian on the $X$–$Z$ plane at $y=0$, represents it through the GaMeS mapping as a 3-vertex triangle, and uses mirror-camera augmentation so that one virtual camera sees the forward image and the other sees its horizontal mirror. The representation then supports affine triangle edits, realistic image modifications, and integration with Taichi_elements MPM in 2D or Blender’s rigid-body or soft-body engine in 3D [2410.01521].

Another line of work addresses initialization and convergence. "EigenGS Representation: From Eigenspace to Gaussian Image Space" learns a shared pool of Gaussian shapes in the eigenimage domain and initializes new images by closed-form PCA projection; on FFHQ (512×512) with $N=20\,000$ and PCA-dimension $k=300$, it reports PSNR $=28.0$ dB at iteration 0 and 36.4 dB at 100 iterations, whereas GaussianImage reports 21.8 dB at 100 iterations [2503.07446]. "Instant GaussianImage: A Generalizable and Self-Adaptive Image Representation via 2D Gaussian Splatting" replaces random initialization with a feed-forward network, introduces a self-adaptive mechanism for choosing the number of Gaussians, and reports that GaussianImage requires $\sim 20$ s to reach PSNR $\approx 40$ dB while Instant-GI reaches PSNR $\approx 40$ dB in $\sim 2$ s on DIV2K ×2 upsampling [2506.23479]. "Fast 2DGS: Efficient Image Representation with Deep Gaussian Prior" further introduces Deep Gaussian Prior and an attribute regression network, reports an init PSNR of 28.1 dB at $K=50k$, reaches 40 dB PSNR within 2 seconds of fine-tuning, final $\approx 43.1$ dB at 10 s, and renders at $>1000$ FPS [2512.12774].

Specialized variants adapt GIR to particular image-formation or structural priors. "Contour Information Aware 2D Gaussian Splatting for Image Representation" constrains each Gaussian to a specific segmentation region during rasterization, refreshes region IDs during a warm-up phase, and reports gains of +0.2–0.4 dB PSNR on the full image and +0.6–1.0 dB PSNR in edge bands versus baseline 2DGS on synthetic color charts [2512.23255]. "Dehaze-GaussianImage: Zero-Shot Dehazing via Efficient 2D Gaussian Splatting Representation" embeds the atmospheric scattering model into the Gaussian parameter space, introduces a reconstruction-decoupling zero-shot learning strategy, and drives Gaussian primitives to adaptively split, clone, and prune during optimization; it also states that an HD image $1920\times1080$ can be represented with $N\approx2000$ primitives, approximately $0.3\%$ of pixel count [2606.16163].

The representation has also migrated into downstream vision and tokenization. "GViT: Representing Images as Gaussians for Visual Recognition" encodes each image as a few hundred Gaussians optimized jointly with a ViT classifier and reaches a 76.9% top-1 accuracy on Imagenet-1k using a ViT-B architecture [2506.23532]. "GPSToken: Gaussian Parameterized Spatially-adaptive Tokenization for Image Representation and Generation" parameterizes non-uniform image tokens as 2D Gaussians with texture features and reports rFID and FID scores of 0.65 and 1.50 on image reconstruction and generation tasks using 128 tokens [2509.01109].

The original GaussianImage paper identifies two limitations. First, current quantization is generic, and specialized Gaussian-tailored entropy models could close the gap to VAE codecs. Second, encoding speed, at $\sim243$ s per 768×512 image, remains higher than classical transform codecs. The same paper lists possible extensions: spatially varying covariance priors or multi-scale (“hierarchical”) Gaussians, learned context models for covariance and color coefficients, integration with semantic layout or text guidance for editing, and direct downstream coding for machine vision tasks [2403.08551].

Taken together, these developments show that GIR has evolved from a codec-oriented, explicit alternative to INRs into a broader family of Gaussian-based continuous image models. The unifying principle remains stable: image content is represented not by a monolithic neural field, but by a finite set of anisotropic Gaussian primitives whose geometry, appearance, and rendering rules can be optimized, quantized, edited, or repurposed for compression, interaction, and downstream vision tasks.

Source: https://www.emergentmind.com/topics/gaussian-image-representation-gir