---
title: 'Ideogram 4.0: Dual-Branch Diffusion Model'
url: https://www.emergentmind.com/topics/ideogram-4-0
type: topic
---

# Ideogram 4.0: Dual-Branch Diffusion Model

Searching arXiv for the specified Ideogram 4.0 papers to ground the article in current research.
I’ll look up the arXiv records for the cited Ideogram 4.0 quantization and kernel papers.
Ideogram 4.0 is a **9.3B flow-matching diffusion transformer (DiT)** for text-to-image generation whose deployment profile, as documented in recent systems work, is defined as much by its inference substrate as by its backbone architecture. In the available arXiv literature, it is described as a **single-stream 34-layer backbone** shipped in **two separate-weight copies** for **classifier-free guidance (CFG)**, with one conditional branch and one unconditional branch, and conditioned by a **Qwen3-VL-8B** text encoder. The model has become a case study in hardware-aware post-training quantization, especially on **consumer Ampere GPUs** such as the RTX 3090, where the interaction among FP8, NF4, INT8 W8A8, and fused native-INT8 kernels materially changes the practical operating point [2606.12280][2606.14598].

## 1. Architectural definition and computational footprint

Ideogram 4.0 is characterized in the cited work as a large DiT whose backbone is duplicated for CFG. Because the model is shipped as **two separate-weight copies** of the same **single-stream 34-layer backbone**, quantization and deployment decisions apply twice over: once to the conditional branch and once to the unconditional branch. The model is further conditioned by a **Qwen3-VL-8B encoder**, while the **Qwen3-VL encoder and the VAE are left at published precision** during quantization experiments. Quantization therefore concentrates on the duplicated DiT backbone rather than on the full end-to-end stack [2606.12280].

This duplication has direct systems consequences. The available description states that quantization must cover **211 linear layers × 2 branches**, which makes weight and activation formats unusually consequential for consumer-GPU deployment. A plausible implication is that Ideogram 4.0 is not merely large in parameter count, but structurally dominated by repeated linear operators whose execution characteristics strongly determine end-to-end latency.

The later kernel study makes that operator structure more concrete by identifying **five actual DiT GEMM shapes** used by the model’s linear layers: **qkv \((13824, 4608)\)**, **attn-out \((4608, 4608)\)**, **ffn-up \((12288, 4608)\)**, **ffn-down \((4608, 12288)\)**, and **llm-proj \((4608, 53248)\)**. Because Ideogram 4.0 exposes only these five distinct GEMM shapes, per-shape autotuning is described as tractable and operationally important rather than incidental [2606.14598].

## 2. Consumer-Ampere deployment regime

The deployment literature on Ideogram 4.0 is explicitly centered on **RTX 3090 (24 GB, Ampere)**. That hardware target matters because **Ampere has no FP8 tensor cores**, even though a published Ideogram 4.0 checkpoint includes an FP8 variant. On Ampere, the FP8 checkpoint therefore **dequantizes each linear to bf16 every forward**. The same studies contrast three practical formats on this hardware: the **FP8 published variant**, the **NF4 published variant**, and **INT8 W8A8**. FP8 is usable but incurs dequantization-to-bf16 at each linear; NF4 is smaller on disk but exhibits quality loss, especially in text; INT8 W8A8 is positioned as the format most aligned with Ampere’s native integer tensor-core capabilities, but only if the implementation actually uses those capabilities [2606.12280].

A central misconception addressed in the native-INT8 paper is that an “INT8” checkpoint necessarily performs native INT8 compute. For Ideogram 4.0’s prior deployed INT8 path, that was not the case. The reported forward path was: **quantize activations and weights to INT8, immediately dequantize both back to bf16, then run `bf16 F.linear`**. In other words, the model used INT8 as a storage or intermediate format, but the actual multiply-accumulate remained bf16. The cited diagnosis is that the production INT8 forward **“never engage[d] the GPU’s INT8 tensor cores”** [2606.14598].

This distinction is critical for interpreting measured latencies. The quantization recipe and the compute kernel are separate design axes. The research record on Ideogram 4.0 repeatedly shows that post-training quantization alone does not determine performance; hardware-native execution does.

## 3. Post-training quantization recipes

The primary INT8 recipe reported for Ideogram 4.0 is a **W8A8** post-training quantization scheme with **per-channel (per-output-row) INT8 weights**, **per-token dynamic INT8 activations**, **SmoothQuant**, and **mixed-precision protection** for a small fragile layer set retained in **bf16** [2606.12280].

SmoothQuant is described as outlier migration with **\(a = 0.5\)**, and the paper states: for a weight \(W\) and per-channel activation scale \(s\), a smoothing vector
\[
A_j = s^\alpha / \max_i |W_{ij}|
\]
rescales activations down and weights up before quantization. The dynamic activation path is motivated by the observation that DiT activations are **timestep-dependent**, so **per-token dynamic scaling** avoids reliance on a static calibration table.

The most distinctive methodological element is the fragility analysis used to decide which layers should remain in bf16. A profiling pass on **8 prompts** from a calibration set records **max-abs**, **standard deviation**, and **kurtosis** over the denoising trajectory for every linear layer in the conditional DiT. Layers are then ranked by **mean kurtosis × max-abs**. The ranking is reported as stable across time, with **Spearman 0.930 between early- and late-step halves**. The most fragile layers are stated to be overwhelmingly the **FFN down-projections** (`feed_forward.w2`), and the single most fragile layer is reported as about **10×** the runner-up. The selected protection set is **top-17 most fragile layers**, about **8% of the 211 linears**, corresponding to about **1.5 GB bf16 overhead**, and includes the high-fragility FFN down-projections and the **time-embedding output** [2606.12280].

The same study also evaluates **GGUF weight-only quantization**. Two reported formats are **Q8_0**, described as **8.5 bits/weight**, and **Q4_K**, described as **4.5 bits/weight** and explicitly placed in the **NF4 size class**. Q4_K is reported to have essentially the same on-disk size as NF4—**10.44 GB vs ~10.4 GB**—while outperforming NF4 on the reported quality metrics, leading the authors to label it the **Pareto winner on the quality-memory frontier**. Q8_0 is described as **quality-neutral** on a **50-prompt subset** [2606.12280].

## 4. Native INT8 compute and the fused GEMM kernel

The second Ideogram 4.0 paper addresses the gap between an INT8 quantized model and actual **native INT8 compute**. Its proposed remedy is a **single fused Triton INT8 GEMM** inserted into Ideogram 4.0’s linear layers in place of the dequantize-to-bf16 path. The kernel performs **`int8 × int8 -> int32`** accumulation on **Ampere `mma.s8` tensor cores**, applies **per-token activation dequantization**, **per-channel weight dequantization**, and **bias add** in the epilogue, and emits dequantized output in one pass [2606.14598].

The operational structure is summarized by the paper as
\[
C_{int32} = A_{int8} \times B_{int8}
\]
followed by epilogue dequantization and bias:
\[
Y = \operatorname{dequant}\!\left(C_{int32}; s_{token}, s_{channel}\right) + b.
\]
The important design constraint is that the activation scale is **per-token dynamic**. Because that scale is not a single static constant, it cannot simply be folded offline into the weights; it must be applied after integer accumulation. This is why the epilogue is the load-bearing part of the design rather than a cosmetic fusion.

The implementation is reported as **bit-exact against `torch._int_mm`** for the integer accumulation, while the dequantized output matches the reference at **cosine similarity 1.0** with **no NaNs**. The kernel is **autotuned over 36 configurations**, keyed per \((M, N, K)\), across the five actual DiT GEMM shapes. The paper is explicit that autotuning is not optional: without it, the fused kernel would be only about **\(1.4\times\) to \(2.9\times\)** faster than bf16, and for the **llm-proj** shape it could even be **slower** than bf16 at **\(0.64\times\)**. With autotuning, every shape improves into the **\(2.8\times\) to \(4.2\times\)** range [2606.14598].

Per-GEMM speedups on an RTX 3090 are reported as **2.79–3.46×** for qkv, **2.86–3.76×** for attn-out, **2.78–4.18×** for ffn-up, **2.94–3.51×** for ffn-down, and **2.95–3.17×** for llm-proj. The same kernel is also reported as **4–8× faster than the unfused int8 path** consisting of `torch._int_mm` followed by separate dequantization, directly quantifying the cost of the prior fake-quant detour.

## 5. Quality, OCR, and reference fidelity

The main quantitative quality study uses three disjoint prompt sets built from PartiPrompts: a **calibration set** of **\(n=128\)**, a **quality benchmark** of **\(n=200\)** that is category-stratified and contains **no text prompts**, and a **text-rendering benchmark** of **\(n=100\)**, of which **63 carry machine-checkable OCR targets**. All variants use the same generation settings: **seed 1000**, **48 steps**, **1024×1024**, and **V4_QUALITY_48 preset**. Reported metrics include **PickScore**, **CLIPScore**, **OCR exact-match**, **Normalized edit distance (NED)** via EasyOCR, **PSNR**, **SSIM**, **LPIPS**, plus latency and peak VRAM. Statistical comparisons on the 200-prompt benchmark are based on **paired per-prompt deltas** with a **10,000-sample bootstrap 95% CI**; the paper also notes that it does **not** apply multiple-comparison correction [2606.12280].

On the 200-prompt benchmark, **INT8 vs FP8** yields a **Pick difference of \(-0.013\)** with **95% CI \([-0.12, +0.09]\)** and a **CLIP difference of \(-0.125\)** with **95% CI \([-0.73, +0.47]\)**. Both intervals include zero, so INT8 is reported as **statistically indistinguishable from FP8** at that sample size. Against NF4, INT8 improves by **\(+0.46\)** Pick with **95% CI \([+0.33, +0.59]\)** and **\(+1.90\)** CLIP with **95% CI \([+1.21, +2.64]\)**, both excluding zero. This is the basis for the paper’s claim that INT8 **holds the FP8 quality ceiling** while **beating NF4 robustly**, especially on CLIP [2606.12280].

The OCR analysis is emphasized because text legibility is described as the most quantization-fragile aspect of this model class. On the **63 OCR-target prompts**, reported **OCR NED** is **0.704** for INT8, **0.715** for FP8, and **0.760** for NF4. Exact-match accuracy is near zero for all variants, with FP8 reported at **0%**, and the authors therefore treat NED as the informative ranking metric. The paper states that the largest per-prompt LPIPS deviations occur on **text prompts**, that **fine secondary lettering breaks first**, and that in the showcased examples INT8 is the only variant that reliably reproduces the FP8 lettering: **“OPEN 24 HOURS”** is matched exactly by INT8, and **“Batch Norm”** is reproduced correctly by INT8 while NF4 and Q4_K garble it [2606.12280].

Reference-fidelity metrics against the FP8 image at the same seed further refine the picture. Over **300 prompts**, INT8 records **PSNR 21.42**, **SSIM 0.722**, and **LPIPS 0.306**; NF4 records **PSNR 21.91**, **SSIM 0.705**, and **LPIPS 0.296**. The paper’s interpretation is that INT8 and NF4 occupy a similar perceptual-distance band, but that **pixel closeness to FP8 is not the same as quality**: INT8 may drift comparably in pixel space while preserving standalone quality and OCR legibility better than NF4. This distinction is central to the evaluation logic.

The ablation results attribute most of the quality recovery to selective protection. On a **50-prompt quality slice**, **naive W8A8 \((a=0,\ \text{no protection})\)** yields **CLIP 14.30**; **+ SmoothQuant only \((a=0.5)\)** yields **16.56**; **+ Protection only (top-17)** yields **16.81**; and **+ Both (final recipe)** yields **17.54**. A protection ladder reports **16.01** at **\(N=8\)** and **17.42** at **\(N=17\)**. The study concludes that **protection is the dominant lever**, SmoothQuant helps but less, the effects are **sub-additive**, and there is a **sharp knee** between protecting **8** and **17** layers [2606.12280].

## 6. End-to-end latency and hardware-specific deployment map

Without a fused kernel, the reported single-GPU-compute latencies at **1024×1024** are **172.9 s/image** for **FP8 (deq.)**, **164.5 s/image** for **NF4**, **184–185 s/image** for **INT8 (ours)**, and **203.3 s/image** for **Q4_K (ours)**. In that regime, INT8 functions primarily as a **quality-preserving compute substrate** rather than a speed optimization. The paper is explicit that **an INT8 speedup on Ampere requires a fused INT8 kernel**; otherwise, pseudo-quantized INT8 remains slower than FP8 and NF4 on this hardware [2606.12280].

With the fused Triton INT8 GEMM, the measured picture changes. At **768px**, the fused INT8 build runs in **97.79 s/image** versus **107.06 s/image** for the fake-quant path, corresponding to about **1.095×** or about **9–10%** speedup. The paper states that the gap between approximately **3.5×** per-GEMM speedup and approximately **9.5%** end-to-end speedup is consistent with **Amdahl’s law**, because the DiT linear GEMMs are only about **12% of the forward pass**. On the same **4-prompt point-estimate subset**, **PickScore** is **21.22 vs 20.14** and **CLIPScore** is **24.35 vs 21.76**, so the fused kernel is reported as causing no measurable quality loss on those point estimates [2606.14598].

At **1024px**, the fused INT8 build generates an image in **156.49 s/image** on a **single RTX 3090**, with **peak VRAM 23.40 GB** and **4/4** successful image generations. The paper compares this with **FP8: 172.9 s/image, requires 2 GPUs**; **NF4: 164.5 s/image, single GPU**; and **prior INT8 fake-quant: 184–185 s/image, requires 2 GPUs**. On those reported numbers, fused INT8 beats FP8 by about **9.5% / ~16 s** and NF4 by about **4.9% / ~8 s**, while making **1024px single-GPU feasible** on a 3090. The authors nevertheless qualify the NF4 margin as being within unquantified run-to-run variance and note that the 1024px comparison mixes a single-GPU fused result with some prior two-GPU baselines, so it is not perfectly apples-to-apples [2606.14598].

The final deployment conclusion is strongly hardware-specific. The fused INT8 kernel is beneficial on **consumer Ampere** because those cards provide native INT8 tensor cores but relatively weak native bf16/FP8 alternatives. On **A100** and **B200**, the same kernel is reported to lose badly to those cards’ native bf16/FP8 paths: at **1024px**, fused INT8 is **1.38× slower** on **A100** and **3.49× slower** on **B200**. The resulting guidance is explicit: use the fused INT8 kernel on **consumer Ampere**, and use native **bf16/FP8** on **datacenter Ampere or Blackwell** [2606.14598].

Taken together, these results define Ideogram 4.0 not only as a large text-to-image DiT, but as a model whose practical identity depends on the alignment between quantization format and true hardware execution. The research record supports three distinct operating points: **INT8 W8A8 with selective bf16 protection** for preserving the FP8 quality ceiling, **Q4_K** for the best reported quality-memory trade-off at the NF4 size class, and **fused native INT8 GEMM** as the mechanism that turns INT8 from the slowest measured variant into the fastest one on consumer Ampere.

Source: https://www.emergentmind.com/topics/ideogram-4-0