Gaussian-Image Representation (GIR)

• Gaussian-Image Representation (GIR) is a continuous model that represents images as a finite set of anisotropic 2D Gaussian primitives, replacing large neural decoders.
• It uses a summation rendering rule and efficient tile-based GPU indexing to achieve high FPS, low memory footprints, and competitive rate–distortion performance.
• Extensions of GIR include editable image representations, fast feed-forward initialization, and specialized codecs for enhanced compression and downstream vision tasks.

Gaussian-Image Representation (GIR) denotes a continuous, explicit image model in which an image is represented as the superposition of a finite set of anisotropic 2D Gaussian primitives (“splats”). In the formulation introduced by "GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splatting" (Zhang et al., 2024), GIR replaces large MLPs or feature-grid implicit neural representations (INRs) with analytic Gaussians and a permutation-invariant summation renderer, yielding fast fitting, extremely high rendering throughput, low GPU-memory footprints, and competitive rate–distortion performance when used as a codec.

1. Definition and terminological scope

The original motivation for GIR arose from the observation that INRs can offer high visual quality and fast rendering speeds with 10-1000 FPS, assuming sufficient GPU resources are available, but that this requirement often hinders their use on low-end devices with limited memory. GaussianImage therefore proposed image representation and compression by 2D Gaussian Splatting, with an explicit, analytic representation rather than a large neural decoder (Zhang et al., 2024).

Within the 2D-image literature, GIR refers to representing an image as a sum of Gaussian primitives in image-plane coordinates, with continuous reconstruction and GPU-oriented splatting. Closely related work preserves the same core idea while changing optimization, initialization, editability, or downstream use. "MiraGe: Editable 2D Images using Gaussian Splatting" extends GaussianImage toward editable 2D images; "Large Images are Gaussians: High-Quality Large Image Representation with Levels of 2D Gaussian Splatting" adapts the formulation to very large images; "Fast 2DGS: Efficient Image Representation with Deep Gaussian Prior" and "Instant GaussianImage: A Generalizable and Self-Adaptive Image Representation via 2D Gaussian Splatting" focus on feed-forward or near-feed-forward initialization; and "Contour Information Aware 2D Gaussian Splatting for Image Representation" adds region-constrained rasterization for boundary preservation (Waczyńska et al., 2024).

The acronym is not unique across arXiv. "GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization" uses GIR to denote 3D Gaussian Inverse Rendering rather than 2D Gaussian-Image Representation, and LongSplat defines a Gaussian-Image Representation that encodes 3D Gaussian parameters into a structured, image-like 2D format for online 3D Gaussian fusion and compression (Shi et al., 2023). This suggests that the term is partly overloaded, even though the GaussianImage family has established a specific meaning centered on explicit 2D image modeling (Huang et al., 22 Jul 2025).

2. Mathematical formulation

In GaussianImage, each primitive $G_i$ is a 2D Gaussian defined by a center $\mu_i \in \mathbb{R}^2$, a positive-definite covariance $\Sigma_i \in \mathbb{R}^{2\times2}$, and a color weight $c_i \in \mathbb{R}^3$, with opacity merged into the color weight. The primitive and full-image reconstruction are

$$G_i(x) = c_i \cdot \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right],$$

and

$$G(x) = \sum_{i=1}^N G_i(x) = \sum_{i=1}^N c_i \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right].$$

To guarantee $\Sigma_i \succ 0$ throughout training, GIR optimizes a factorized form of the covariance. Two decompositions are described as common: Cholesky factorization,

$$\Sigma_i = L_i L_i^\top,\qquad L_i = \begin{bmatrix} l_{i,1} & 0\ l_{i,2} & l_{i,3} \end{bmatrix},$$

and rotation-scaling factorization,

$\Sigma_i = (R_i S_i)(R_i S_i)^\top.$

The GaussianImage paper reports that the Cholesky form is more robust under quantization, so each Gaussian is parameterized by $(\mu_{i,x},\mu_{i,y})$, $\mu_i \in \mathbb{R}^2$0, and color weight $\mu_i \in \mathbb{R}^2$1, for a total of 8 learnable scalars per primitive. During training, $\mu_i \in \mathbb{R}^2$2 is unconstrained, or in $\mu_i \in \mathbb{R}^2$3 via $\mu_i \in \mathbb{R}^2$4, and $\mu_i \in \mathbb{R}^2$5 is free (Zhang et al., 2024).

Later variants often separate amplitude or opacity from color, while preserving the same superposition structure. MiraGe represents an image $\mu_i \in \mathbb{R}^2$6 by $\mu_i \in \mathbb{R}^2$7 weighted 2D Gaussian blobs with mean $\mu_i \in \mathbb{R}^2$8, covariance $\mu_i \in \mathbb{R}^2$9, weight $\Sigma_i \in \mathbb{R}^{2\times2}$0, and color $\Sigma_i \in \mathbb{R}^{2\times2}$1, whereas Fast-2DGS uses mean, covariance, amplitude $\Sigma_i \in \mathbb{R}^{2\times2}$2, and color vector $\Sigma_i \in \mathbb{R}^{2\times2}$3 (Waczyńska et al., 2024). This variation does not alter the basic ontology of GIR: a continuous image is reconstructed from anisotropic Gaussian basis functions (Wang et al., 14 Dec 2025).

3. Summation rendering and fitting

A defining distinction of GaussianImage is its rendering rule. Traditional 3D Gaussian Splatting uses $\Sigma_i \in \mathbb{R}^{2\times2}$4-blending with depth sorting,

$\Sigma_i \in \mathbb{R}^{2\times2}$5

GIR observes that for a single view there is no depth ambiguity, so Gaussians need not be sorted and $\Sigma_i \in \mathbb{R}^{2\times2}$6-accumulation can be absorbed into a single weight. For pixel $\Sigma_i \in \mathbb{R}^{2\times2}$7, with

$\Sigma_i \in \mathbb{R}^{2\times2}$8

the rendered color is

$\Sigma_i \in \mathbb{R}^{2\times2}$9

The resulting renderer has no sorting and no per-primitive $c_i \in \mathbb{R}^3$0 term (Zhang et al., 2024).

On the GPU, GaussianImage builds a tile-based index of which Gaussians influence which pixel blocks, using a confidence-interval radius, and then launches one CUDA thread per pixel that loops over only the local Gaussians. Since each pixel sums $c_i \in \mathbb{R}^3$1 terms and $c_i \in \mathbb{R}^3$2 is independent of $c_i \in \mathbb{R}^3$3 once the image is large, the rendering throughput is $c_i \in \mathbb{R}^3$4 with negligible overhead in $c_i \in \mathbb{R}^3$5, achieving 1500–2000 FPS on an NVIDIA V100 regardless of $c_i \in \mathbb{R}^3$6 up to hundreds of thousands (Zhang et al., 2024).

Fitting is performed by minimizing an $c_i \in \mathbb{R}^3$7 reconstruction loss,

$c_i \in \mathbb{R}^3$8

over, for example, 50 000 steps with the Adan optimizer, using an initial learning rate of $c_i \in \mathbb{R}^3$9 halved at 20 k and 40 k. No split/clone density control is needed in the original formulation, since the entire image plane is occupied. Gaussians are initialized with random $$G_i(x) = c_i \cdot \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right],$$0, small isotropic $$G_i(x) = c_i \cdot \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right],$$1, and random $$G_i(x) = c_i \cdot \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right],$$2 (Zhang et al., 2024).

This renderer established the order-invariant, explicit alternative that later papers either preserve or modify. Fast-2DGS, for example, keeps a tile-based rasterization but replaces pure summation with a weighted average over the top-$$G_i(x) = c_i \cdot \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right],$$3 Gaussians covering each pixel, while Dehaze-GaussianImage reintroduces a volume-rendering-style compositing because it couples Gaussian rendering to an atmospheric scattering model (Wang et al., 14 Dec 2025).

4. Compression and codec formulation

GaussianImage does not stop at overfitting; it explicitly constructs an image codec. After overfitting, GIR applies a two-step pipeline consisting of quantization-aware fine-tuning and entropy encoding. In the first stage, $$G_i(x) = c_i \cdot \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right],$$4 is quantized to 16-bit float; the Cholesky parameters $$G_i(x) = c_i \cdot \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right],$$5 are quantized by $$G_i(x) = c_i \cdot \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right],$$6-bit asymmetric learned quantization via LSQ+,

$$G_i(x) = c_i \cdot \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right],$$7

with reconstruction

$$G_i(x) = c_i \cdot \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right],$$8

and the color vectors are quantized by a cascade $$G_i(x) = c_i \cdot \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1} (x-\mu_i)\right],$$9 stage residual vector quantization with codebook size $$G(x) = \sum_{i=1}^N G_i(x) = \sum_{i=1}^N c_i \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right].$$0, minimizing a commitment loss (Zhang et al., 2024).

In the second stage, Asymmetric Numeral Systems (ANS) codes the quantized Cholesky parameters and the RVQ indices. The codec also allows optional partial bits-back coding to amortize the $$G(x) = \sum_{i=1}^N G_i(x) = \sum_{i=1}^N c_i \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right].$$1 symmetry of unordered Gaussians, yielding an additional rate saving of $$G(x) = \sum_{i=1}^N G_i(x) = \sum_{i=1}^N c_i \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right].$$2 bits once $$G(x) = \sum_{i=1}^N G_i(x) = \sum_{i=1}^N c_i \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right].$$3 initial items are coded. Rate–distortion is controlled by adjusting $$G(x) = \sum_{i=1}^N G_i(x) = \sum_{i=1}^N c_i \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right].$$4, $$G(x) = \sum_{i=1}^N G_i(x) = \sum_{i=1}^N c_i \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right].$$5, $$G(x) = \sum_{i=1}^N G_i(x) = \sum_{i=1}^N c_i \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right].$$6, and the quantization schedule, and can be written as minimizing

$$G(x) = \sum_{i=1}^N G_i(x) = \sum_{i=1}^N c_i \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right].$$7

where $$G(x) = \sum_{i=1}^N G_i(x) = \sum_{i=1}^N c_i \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right].$$8 is the ANS-coded bitstream length and $$G(x) = \sum_{i=1}^N G_i(x) = \sum_{i=1}^N c_i \exp\!\left[-\tfrac12 (x-\mu_i)^\top \Sigma_i^{-1}(x-\mu_i)\right].$$9 is the MSE reconstruction (Zhang et al., 2024).

The reported behavior is that GIR matches COIN and COIN++ in PSNR versus bpp, and that the bits-back variant even slightly outperforms them. The codec also facilitates decoding speeds of approximately 2000 FPS, and the paper reports preliminary proof of concept that partial bits-back coding surpasses COIN and COIN++ in performance (Zhang et al., 2024).

5. Benchmarks, operating regimes, and scaling behavior

The original GaussianImage paper reports a representation snapshot on Kodak and codec results at low bitrate. The benchmarked values below summarize the canonical operating regime of the method (Zhang et al., 2024).

Setting	GaussianImage	Comparator
Image representation snapshot, $\Sigma_i \succ 0$0	PSNR 44.08 dB; MS-SSIM 0.9985; training 106 s; memory 419 MiB; rendering 2092 FPS	I-NGP: 43.88 dB; 490 s; 1525 MiB; 1297 FPS
Codec, $\Sigma_i \succ 0$1	PSNR $\Sigma_i \succ 0$2 dB; MS-SSIM 0.915; decoding speed $\Sigma_i \succ 0$3 FPS on V100	JPEG2000 27.28 dB; COIN 25.80 dB; COIN MS-SSIM 0.890; JPEG $\Sigma_i \succ 0$4 FPS on CPU

These numbers substantiate the specific claim that GaussianImage attains approximately 5$\Sigma_i \succ 0$5 faster fitting time, at least 3$\Sigma_i \succ 0$6 lower GPU memory usage, and faster rendering than I-NGP while remaining competitive in final reconstruction quality (Zhang et al., 2024).

Subsequent work has explored different operating regimes rather than a single benchmark point. "Large Images are Gaussians: High-Quality Large Image Representation with Levels of 2D Gaussian Splatting" addresses fitting large images with a large number of Gaussian points by combining a variant of representation and optimization strategy with a Level-of-Gaussian approach for coarse low-frequency initialization and fine high-frequency detail recovery. On STimage 9K with $\Sigma_i \succ 0$7, the paper reports GaussianImage at PSNR $\Sigma_i \succ 0$8 dB, memory $\Sigma_i \succ 0$9 GB, FPS $$\Sigma_i = L_i L_i^\top,\qquad L_i = \begin{bmatrix} l_{i,1} & 0\ l_{i,2} & l_{i,3} \end{bmatrix},$$0, and LIG at PSNR $$\Sigma_i = L_i L_i^\top,\qquad L_i = \begin{bmatrix} l_{i,1} & 0\ l_{i,2} & l_{i,3} \end{bmatrix},$$1 dB, memory $$\Sigma_i = L_i L_i^\top,\qquad L_i = \begin{bmatrix} l_{i,1} & 0\ l_{i,2} & l_{i,3} \end{bmatrix},$$2 GB, FPS $$\Sigma_i = L_i L_i^\top,\qquad L_i = \begin{bmatrix} l_{i,1} & 0\ l_{i,2} & l_{i,3} \end{bmatrix},$$3 (Zhu et al., 13 Feb 2025).

This broadens GIR from the original tens-of-thousands-of-Gaussians regime to datasets using tens of millions of Gaussians, showing that the same primitive-based representation can be adapted to very large images when the optimization strategy is modified (Zhu et al., 13 Feb 2025).

6. Extensions, applications, and limitations

A major extension of GIR concerns editability. "MiraGe: Editable 2D Images using Gaussian Splatting" lifts each 2D Gaussian into 3D as a flat Gaussian on the $$\Sigma_i = L_i L_i^\top,\qquad L_i = \begin{bmatrix} l_{i,1} & 0\ l_{i,2} & l_{i,3} \end{bmatrix},$$4–$$\Sigma_i = L_i L_i^\top,\qquad L_i = \begin{bmatrix} l_{i,1} & 0\ l_{i,2} & l_{i,3} \end{bmatrix},$$5 plane at $$\Sigma_i = L_i L_i^\top,\qquad L_i = \begin{bmatrix} l_{i,1} & 0\ l_{i,2} & l_{i,3} \end{bmatrix},$$6, represents it through the GaMeS mapping as a 3-vertex triangle, and uses mirror-camera augmentation so that one virtual camera sees the forward image and the other sees its horizontal mirror. The representation then supports affine triangle edits, realistic image modifications, and integration with Taichi_elements MPM in 2D or Blender’s rigid-body or soft-body engine in 3D (Waczyńska et al., 2024).

Another line of work addresses initialization and convergence. "EigenGS Representation: From Eigenspace to Gaussian Image Space" learns a shared pool of Gaussian shapes in the eigenimage domain and initializes new images by closed-form PCA projection; on FFHQ (512×512) with $$\Sigma_i = L_i L_i^\top,\qquad L_i = \begin{bmatrix} l_{i,1} & 0\ l_{i,2} & l_{i,3} \end{bmatrix},$$7 and PCA-dimension $$\Sigma_i = L_i L_i^\top,\qquad L_i = \begin{bmatrix} l_{i,1} & 0\ l_{i,2} & l_{i,3} \end{bmatrix},$$8, it reports PSNR $$\Sigma_i = L_i L_i^\top,\qquad L_i = \begin{bmatrix} l_{i,1} & 0\ l_{i,2} & l_{i,3} \end{bmatrix},$$9 dB at iteration 0 and 36.4 dB at 100 iterations, whereas GaussianImage reports 21.8 dB at 100 iterations (Tai et al., 10 Mar 2025). "Instant GaussianImage: A Generalizable and Self-Adaptive Image Representation via 2D Gaussian Splatting" replaces random initialization with a feed-forward network, introduces a self-adaptive mechanism for choosing the number of Gaussians, and reports that GaussianImage requires $\Sigma_i = (R_i S_i)(R_i S_i)^\top.$0 s to reach PSNR $\Sigma_i = (R_i S_i)(R_i S_i)^\top.$1 dB while Instant-GI reaches PSNR $\Sigma_i = (R_i S_i)(R_i S_i)^\top.$2 dB in $\Sigma_i = (R_i S_i)(R_i S_i)^\top.$3 s on DIV2K ×2 upsampling (Zeng et al., 30 Jun 2025). "Fast 2DGS: Efficient Image Representation with Deep Gaussian Prior" further introduces Deep Gaussian Prior and an attribute regression network, reports an init PSNR of 28.1 dB at $\Sigma_i = (R_i S_i)(R_i S_i)^\top.$4, reaches 40 dB PSNR within 2 seconds of fine-tuning, final $\Sigma_i = (R_i S_i)(R_i S_i)^\top.$5 dB at 10 s, and renders at $\Sigma_i = (R_i S_i)(R_i S_i)^\top.$6 FPS (Wang et al., 14 Dec 2025).

Specialized variants adapt GIR to particular image-formation or structural priors. "Contour Information Aware 2D Gaussian Splatting for Image Representation" constrains each Gaussian to a specific segmentation region during rasterization, refreshes region IDs during a warm-up phase, and reports gains of +0.2–0.4 dB PSNR on the full image and +0.6–1.0 dB PSNR in edge bands versus baseline 2DGS on synthetic color charts (Takabe et al., 29 Dec 2025). "Dehaze-GaussianImage: Zero-Shot Dehazing via Efficient 2D Gaussian Splatting Representation" embeds the atmospheric scattering model into the Gaussian parameter space, introduces a reconstruction-decoupling zero-shot learning strategy, and drives Gaussian primitives to adaptively split, clone, and prune during optimization; it also states that an HD image $\Sigma_i = (R_i S_i)(R_i S_i)^\top.$7 can be represented with $\Sigma_i = (R_i S_i)(R_i S_i)^\top.$8 primitives, approximately $\Sigma_i = (R_i S_i)(R_i S_i)^\top.$9 of pixel count (Chen et al., 15 Jun 2026).

The representation has also migrated into downstream vision and tokenization. "GViT: Representing Images as Gaussians for Visual Recognition" encodes each image as a few hundred Gaussians optimized jointly with a ViT classifier and reaches a 76.9% top-1 accuracy on Imagenet-1k using a ViT-B architecture (Hernandez et al., 30 Jun 2025). "GPSToken: Gaussian Parameterized Spatially-adaptive Tokenization for Image Representation and Generation" parameterizes non-uniform image tokens as 2D Gaussians with texture features and reports rFID and FID scores of 0.65 and 1.50 on image reconstruction and generation tasks using 128 tokens (Zhang et al., 1 Sep 2025).

The original GaussianImage paper identifies two limitations. First, current quantization is generic, and specialized Gaussian-tailored entropy models could close the gap to VAE codecs. Second, encoding speed, at $(\mu_{i,x},\mu_{i,y})$0 s per 768×512 image, remains higher than classical transform codecs. The same paper lists possible extensions: spatially varying covariance priors or multi-scale (“hierarchical”) Gaussians, learned context models for covariance and color coefficients, integration with semantic layout or text guidance for editing, and direct downstream coding for machine vision tasks (Zhang et al., 2024).

Taken together, these developments show that GIR has evolved from a codec-oriented, explicit alternative to INRs into a broader family of Gaussian-based continuous image models. The unifying principle remains stable: image content is represented not by a monolithic neural field, but by a finite set of anisotropic Gaussian primitives whose geometry, appearance, and rendering rules can be optimized, quantized, edited, or repurposed for compression, interaction, and downstream vision tasks.