Ideogram 4.0: Dual-Branch Diffusion Model

• Ideogram 4.0 is a 9.3B diffusion transformer featuring a dual-weight, single-stream 34-layer backbone designed for classifier-free guidance.
• It employs hardware-aware post-training quantization across FP8, NF4, and INT8 formats with selective bf16 protection for robust image and OCR quality.
• A fused native INT8 GEMM kernel, autotuned for five distinct GEMM shapes, significantly enhances inference speed on consumer Ampere GPUs.

Searching arXiv for the specified Ideogram 4.0 papers to ground the article in current research. I’ll look up the arXiv records for the cited Ideogram 4.0 quantization and kernel papers. Ideogram 4.0 is a 9.3B flow-matching diffusion transformer (DiT) for text-to-image generation whose deployment profile, as documented in recent systems work, is defined as much by its inference substrate as by its backbone architecture. In the available arXiv literature, it is described as a single-stream 34-layer backbone shipped in two separate-weight copies for classifier-free guidance (CFG), with one conditional branch and one unconditional branch, and conditioned by a Qwen3-VL-8B text encoder. The model has become a case study in hardware-aware post-training quantization, especially on consumer Ampere GPUs such as the RTX 3090, where the interaction among FP8, NF4, INT8 W8A8, and fused native-INT8 kernels materially changes the practical operating point (Gandhi et al., 10 Jun 2026, Asaria et al., 12 Jun 2026).

1. Architectural definition and computational footprint

Ideogram 4.0 is characterized in the cited work as a large DiT whose backbone is duplicated for CFG. Because the model is shipped as two separate-weight copies of the same single-stream 34-layer backbone, quantization and deployment decisions apply twice over: once to the conditional branch and once to the unconditional branch. The model is further conditioned by a Qwen3-VL-8B encoder, while the Qwen3-VL encoder and the VAE are left at published precision during quantization experiments. Quantization therefore concentrates on the duplicated DiT backbone rather than on the full end-to-end stack (Gandhi et al., 10 Jun 2026).

This duplication has direct systems consequences. The available description states that quantization must cover 211 linear layers × 2 branches, which makes weight and activation formats unusually consequential for consumer-GPU deployment. A plausible implication is that Ideogram 4.0 is not merely large in parameter count, but structurally dominated by repeated linear operators whose execution characteristics strongly determine end-to-end latency.

The later kernel study makes that operator structure more concrete by identifying five actual DiT GEMM shapes used by the model’s linear layers: qkv $(13824, 4608)$, attn-out $(4608, 4608)$, ffn-up $(12288, 4608)$, ffn-down $(4608, 12288)$, and LLM-proj $(4608, 53248)$. Because Ideogram 4.0 exposes only these five distinct GEMM shapes, per-shape autotuning is described as tractable and operationally important rather than incidental (Asaria et al., 12 Jun 2026).

2. Consumer-Ampere deployment regime

The deployment literature on Ideogram 4.0 is explicitly centered on RTX 3090 (24 GB, Ampere). That hardware target matters because Ampere has no FP8 tensor cores, even though a published Ideogram 4.0 checkpoint includes an FP8 variant. On Ampere, the FP8 checkpoint therefore dequantizes each linear to bf16 every forward. The same studies contrast three practical formats on this hardware: the FP8 published variant, the NF4 published variant, and INT8 W8A8. FP8 is usable but incurs dequantization-to-bf16 at each linear; NF4 is smaller on disk but exhibits quality loss, especially in text; INT8 W8A8 is positioned as the format most aligned with Ampere’s native integer tensor-core capabilities, but only if the implementation actually uses those capabilities (Gandhi et al., 10 Jun 2026).

A central misconception addressed in the native-INT8 paper is that an “INT8” checkpoint necessarily performs native INT8 compute. For Ideogram 4.0’s prior deployed INT8 path, that was not the case. The reported forward path was: quantize activations and weights to INT8, immediately dequantize both back to bf16, then run bf16 F.linear. In other words, the model used INT8 as a storage or intermediate format, but the actual multiply-accumulate remained bf16. The cited diagnosis is that the production INT8 forward “never engage[d] the GPU’s INT8 tensor cores” (Asaria et al., 12 Jun 2026).

This distinction is critical for interpreting measured latencies. The quantization recipe and the compute kernel are separate design axes. The research record on Ideogram 4.0 repeatedly shows that post-training quantization alone does not determine performance; hardware-native execution does.

3. Post-training quantization recipes

The primary INT8 recipe reported for Ideogram 4.0 is a W8A8 post-training quantization scheme with per-channel (per-output-row) INT8 weights, per-token dynamic INT8 activations, SmoothQuant, and mixed-precision protection for a small fragile layer set retained in bf16 (Gandhi et al., 10 Jun 2026).

SmoothQuant is described as outlier migration with $a = 0.5$, and the paper states: for a weight $W$ and per-channel activation scale $s$, a smoothing vector

$A_j = s^\alpha / \max_i |W_{ij}|$

rescales activations down and weights up before quantization. The dynamic activation path is motivated by the observation that DiT activations are timestep-dependent, so per-token dynamic scaling avoids reliance on a static calibration table.

The most distinctive methodological element is the fragility analysis used to decide which layers should remain in bf16. A profiling pass on 8 prompts from a calibration set records max-abs, standard deviation, and kurtosis over the denoising trajectory for every linear layer in the conditional DiT. Layers are then ranked by mean kurtosis × max-abs. The ranking is reported as stable across time, with Spearman 0.930 between early- and late-step halves. The most fragile layers are stated to be overwhelmingly the FFN down-projections (feed_forward.w2), and the single most fragile layer is reported as about 10× the runner-up. The selected protection set is top-17 most fragile layers, about 8% of the 211 linears, corresponding to about 1.5 GB bf16 overhead, and includes the high-fragility FFN down-projections and the time-embedding output (Gandhi et al., 10 Jun 2026).

The same study also evaluates GGUF weight-only quantization. Two reported formats are Q8_0, described as 8.5 bits/weight, and Q4_K, described as 4.5 bits/weight and explicitly placed in the NF4 size class. Q4_K is reported to have essentially the same on-disk size as NF4—10.44 GB vs ~10.4 GB—while outperforming NF4 on the reported quality metrics, leading the authors to label it the Pareto winner on the quality-memory frontier. Q8_0 is described as quality-neutral on a 50-prompt subset (Gandhi et al., 10 Jun 2026).

4. Native INT8 compute and the fused GEMM kernel

The second Ideogram 4.0 paper addresses the gap between an INT8 quantized model and actual native INT8 compute. Its proposed remedy is a single fused Triton INT8 GEMM inserted into Ideogram 4.0’s linear layers in place of the dequantize-to-bf16 path. The kernel performs int8 × int8 -> int32 accumulation on Ampere mma.s8 tensor cores, applies per-token activation dequantization, per-channel weight dequantization, and bias add in the epilogue, and emits dequantized output in one pass (Asaria et al., 12 Jun 2026).

The operational structure is summarized by the paper as

$C_{int32} = A_{int8} \times B_{int8}$

followed by epilogue dequantization and bias: $(4608, 4608)$0 The important design constraint is that the activation scale is per-token dynamic. Because that scale is not a single static constant, it cannot simply be folded offline into the weights; it must be applied after integer accumulation. This is why the epilogue is the load-bearing part of the design rather than a cosmetic fusion.

The implementation is reported as bit-exact against torch._int_mm for the integer accumulation, while the dequantized output matches the reference at cosine similarity 1.0 with no NaNs. The kernel is autotuned over 36 configurations, keyed per $(4608, 4608)$1, across the five actual DiT GEMM shapes. The paper is explicit that autotuning is not optional: without it, the fused kernel would be only about $(4608, 4608)$2 to $(4608, 4608)$3 faster than bf16, and for the LLM-proj shape it could even be slower than bf16 at $(4608, 4608)$4. With autotuning, every shape improves into the $(4608, 4608)$5 to $(4608, 4608)$6 range (Asaria et al., 12 Jun 2026).

Per-GEMM speedups on an RTX 3090 are reported as 2.79–3.46× for qkv, 2.86–3.76× for attn-out, 2.78–4.18× for ffn-up, 2.94–3.51× for ffn-down, and 2.95–3.17× for LLM-proj. The same kernel is also reported as 4–8× faster than the unfused int8 path consisting of torch._int_mm followed by separate dequantization, directly quantifying the cost of the prior fake-quant detour.

5. Quality, OCR, and reference fidelity

The main quantitative quality study uses three disjoint prompt sets built from PartiPrompts: a calibration set of $(4608, 4608)$7, a quality benchmark of $(4608, 4608)$8 that is category-stratified and contains no text prompts, and a text-rendering benchmark of $(4608, 4608)$9, of which 63 carry machine-checkable OCR targets. All variants use the same generation settings: seed 1000, 48 steps, 1024×1024, and V4_QUALITY_48 preset. Reported metrics include PickScore, CLIPScore, OCR exact-match, Normalized edit distance (NED) via EasyOCR, PSNR, SSIM, LPIPS, plus latency and peak VRAM. Statistical comparisons on the 200-prompt benchmark are based on paired per-prompt deltas with a 10,000-sample bootstrap 95% CI; the paper also notes that it does not apply multiple-comparison correction (Gandhi et al., 10 Jun 2026).

On the 200-prompt benchmark, INT8 vs FP8 yields a Pick difference of $(12288, 4608)$0 with 95% CI $(12288, 4608)$1 and a CLIP difference of $(12288, 4608)$2 with 95% CI $(12288, 4608)$3. Both intervals include zero, so INT8 is reported as statistically indistinguishable from FP8 at that sample size. Against NF4, INT8 improves by $(12288, 4608)$4 Pick with 95% CI $(12288, 4608)$5 and $(12288, 4608)$6 CLIP with 95% CI $(12288, 4608)$7, both excluding zero. This is the basis for the paper’s claim that INT8 holds the FP8 quality ceiling while beating NF4 robustly, especially on CLIP (Gandhi et al., 10 Jun 2026).

The OCR analysis is emphasized because text legibility is described as the most quantization-fragile aspect of this model class. On the 63 OCR-target prompts, reported OCR NED is 0.704 for INT8, 0.715 for FP8, and 0.760 for NF4. Exact-match accuracy is near zero for all variants, with FP8 reported at 0%, and the authors therefore treat NED as the informative ranking metric. The paper states that the largest per-prompt LPIPS deviations occur on text prompts, that fine secondary lettering breaks first, and that in the showcased examples INT8 is the only variant that reliably reproduces the FP8 lettering: “OPEN 24 HOURS” is matched exactly by INT8, and “Batch Norm” is reproduced correctly by INT8 while NF4 and Q4_K garble it (Gandhi et al., 10 Jun 2026).

Reference-fidelity metrics against the FP8 image at the same seed further refine the picture. Over 300 prompts, INT8 records PSNR 21.42, SSIM 0.722, and LPIPS 0.306; NF4 records PSNR 21.91, SSIM 0.705, and LPIPS 0.296. The paper’s interpretation is that INT8 and NF4 occupy a similar perceptual-distance band, but that pixel closeness to FP8 is not the same as quality: INT8 may drift comparably in pixel space while preserving standalone quality and OCR legibility better than NF4. This distinction is central to the evaluation logic.

The ablation results attribute most of the quality recovery to selective protection. On a 50-prompt quality slice, naive W8A8 $(12288, 4608)$8 yields CLIP 14.30; + SmoothQuant only $(12288, 4608)$9 yields 16.56; + Protection only (top-17) yields 16.81; and + Both (final recipe) yields 17.54. A protection ladder reports 16.01 at $(4608, 12288)$0 and 17.42 at $(4608, 12288)$1. The study concludes that protection is the dominant lever, SmoothQuant helps but less, the effects are sub-additive, and there is a sharp knee between protecting 8 and 17 layers (Gandhi et al., 10 Jun 2026).

6. End-to-end latency and hardware-specific deployment map

Without a fused kernel, the reported single-GPU-compute latencies at 1024×1024 are 172.9 s/image for FP8 (deq.), 164.5 s/image for NF4, 184–185 s/image for INT8 (ours), and 203.3 s/image for Q4_K (ours). In that regime, INT8 functions primarily as a quality-preserving compute substrate rather than a speed optimization. The paper is explicit that an INT8 speedup on Ampere requires a fused INT8 kernel; otherwise, pseudo-quantized INT8 remains slower than FP8 and NF4 on this hardware (Gandhi et al., 10 Jun 2026).

With the fused Triton INT8 GEMM, the measured picture changes. At 768px, the fused INT8 build runs in 97.79 s/image versus 107.06 s/image for the fake-quant path, corresponding to about 1.095× or about 9–10% speedup. The paper states that the gap between approximately 3.5× per-GEMM speedup and approximately 9.5% end-to-end speedup is consistent with Amdahl’s law, because the DiT linear GEMMs are only about 12% of the forward pass. On the same 4-prompt point-estimate subset, PickScore is 21.22 vs 20.14 and CLIPScore is 24.35 vs 21.76, so the fused kernel is reported as causing no measurable quality loss on those point estimates (Asaria et al., 12 Jun 2026).

At 1024px, the fused INT8 build generates an image in 156.49 s/image on a single RTX 3090, with peak VRAM 23.40 GB and 4/4 successful image generations. The paper compares this with FP8: 172.9 s/image, requires 2 GPUs; NF4: 164.5 s/image, single GPU; and prior INT8 fake-quant: 184–185 s/image, requires 2 GPUs. On those reported numbers, fused INT8 beats FP8 by about 9.5% / ~16 s and NF4 by about 4.9% / ~8 s, while making 1024px single-GPU feasible on a 3090. The authors nevertheless qualify the NF4 margin as being within unquantified run-to-run variance and note that the 1024px comparison mixes a single-GPU fused result with some prior two-GPU baselines, so it is not perfectly apples-to-apples (Asaria et al., 12 Jun 2026).

The final deployment conclusion is strongly hardware-specific. The fused INT8 kernel is beneficial on consumer Ampere because those cards provide native INT8 tensor cores but relatively weak native bf16/FP8 alternatives. On A100 and B200, the same kernel is reported to lose badly to those cards’ native bf16/FP8 paths: at 1024px, fused INT8 is 1.38× slower on A100 and 3.49× slower on B200. The resulting guidance is explicit: use the fused INT8 kernel on consumer Ampere, and use native bf16/FP8 on datacenter Ampere or Blackwell (Asaria et al., 12 Jun 2026).

Taken together, these results define Ideogram 4.0 not only as a large text-to-image DiT, but as a model whose practical identity depends on the alignment between quantization format and true hardware execution. The research record supports three distinct operating points: INT8 W8A8 with selective bf16 protection for preserving the FP8 quality ceiling, Q4_K for the best reported quality-memory trade-off at the NF4 size class, and fused native INT8 GEMM as the mechanism that turns INT8 from the slowest measured variant into the fastest one on consumer Ampere.