Neural Texture Compression Architectures

• Neural texture compression architectures are methods that combine hardware-compatible block compression with neural decoding to efficiently store and render complex, multi-channel textures.
• They leverage differentiable training pipelines that emulate hardware decompression steps, enabling real-time random-access rendering with superior perceptual quality.
• Empirical benchmarks show significant storage reductions (14–25×) and sub-millisecond decoding costs, facilitating seamless integration with GPU pipelines.

Neural texture compression architectures represent an intersection of hardware-compatible block compression and neural decoding, targeting efficient storage and real-time random-access rendering of complex, multichannel texture sets. These methods leverage differentiable training pipelines that mirror hardware decompression, enabling deployment within conventional graphics engines while achieving higher compression ratios and better perceptual quality relative to traditional schemes.

1. Feature Organization, Block Layout, and Hardware Compatibility

State-of-the-art neural texture compression frameworks structure learned feature maps as block-wise, GPU-compatible compressed textures. For example, neural material architectures use collections of feature textures $T_e^i$ conforming to BC6H (block-compressed high-dynamic-range) or BC1 (low-dynamic-range) formats (Weinreich et al., 2023, Laurent et al., 6 Jun 2025). Each texture is stored as a mipmapped pyramid, with each level's spatial extent $w_i \times h_i$ and typically three channels to match hardware encoding (e.g., RGB).

Each $4\times4$ texel block encodes:

• Endpoint vectors $e_1,e_2$ or dual pairs $(e_1,e_2,e_3,e_4)$,
• A block partition index $k$,
• Per-texel interpolation weights $x_j$ (3-bits for BC6H, 2-bits for BC1).

All data is natively formatted for direct on-GPU use. At runtime, hardware texture samplers support random (u,v)-coordinate sampling with built-in bilinear and trilinear filtering over block-mip levels, yielding continuous multi-scale feature decoding (Weinreich et al., 2023).

Other methods store learned latent codes as regular 2D or 3D grid tensors, quantized to low bit-width, often without further entropy coding. Random-access is enabled through constant-time lookup and local interpolation in each grid, eliminating the need for serial decompression (Vaidyanathan et al., 2023, Farhadzadeh et al., 2024).

2. Differentiable Block Compression Emulation in Training

To facilitate end-to-end optimization while ensuring compatibility with block-compression hardware, these architectures perform “fake” or simulated decompression in the learning loop. The emulation mirrors integer arithmetic, partition logic, and quantization steps of native BC hardware. For BC6H (Weinreich et al., 2023):

• Endpoint quantization is emulated as

$\tilde e_i = \frac{2^{16}e_i + 2^{15}}{2^b-1}a,$

where $a$ is a mode-dependent scaling factor.

• Per-texel decoding uses linear expressions over $P_k$ (partition mask) and $w_i \times h_i$0 (indices).
• The decode path terminates with an emulated half-float bit-cast, matching hardware truncation for $w_i \times h_i$1.

All such steps are constructed to be (piecewise) differentiable, supporting backpropagation into all decompression-relevant latent variables: block endpoints, partition indices, and per-texel interpolants.

Neural Texture Block Compression (NTBC) emulates the entire BC1/BC4 packing path in the forward training pass, using a straight-through estimator to allow gradients through the discrete index selection ($w_i \times h_i$2 over palette distances) (Fujieda et al., 2024).

3. Decoder Network Architectures

Once block-compressed latent features are decoded, all major approaches utilize lightweight neural decoders, typically single-hidden-layer or shallow multi-layer MLPs. Notable configurations include:

• Concatenation of feature vectors from parallel block-compressed levels ($w_i \times h_i$3 input for four 3-channel BC6H feature textures), followed by a $w_i \times h_i$4 MLP with ReLU (Weinreich et al., 2023), where $w_i \times h_i$5 is the number of reconstructed texture channels.
• Small MLPs with four residual (linear+skip) blocks for random-access multi-channel decompression. Inputs include local latent grid samples and positional encodings (Farhadzadeh et al., 2024).
• Cooperative vector decoding: groups of pixels within a tile execute a single matrix-vector operation in parallel via hardware matrix-multiply units, maximizing throughput (Laurent et al., 6 Jun 2025).
• Decoder MLP sizes are intentionally minimized (<100 KB weights), typically mapping directly to in-shader matrix-vector instructions in GPU pipelines.

4. Rate-Distortion Trade-offs, Continuous Decoding, and Filtering

Continuous, multiscale decoding is achieved by training with random (u,v,scale) samples and leveraging hardware filtering (bilinear/trilinear) on block-compressed latent textures (Weinreich et al., 2023, Laurent et al., 6 Jun 2025). As a result:

• A single texture fetch per level, filtered at the hardware level, suffices to provide anti-aliased, smoothly varying feature vectors across spatial position and scale.
• No additional in-shader pre-filtering or multi-sample filtering is required, significantly reducing per-pixel compute and bandwidth (Weinreich et al., 2023).

Rate-distortion metrics report that block-compressed neural architectures outperform both BC and ASTC in filtered PSNR (e.g., achieving 31.8 dB vs. 28.5 dB for BC1/BC5 at similar storage, with substantially lower VRAM footprints) (Weinreich et al., 2023). Decompression overhead is kept below 1 ms even for full 4K assets with anisotropic filtering (Laurent et al., 6 Jun 2025).

5. Architectural Variants and Practical Integration

Multiple architectural strategies coexist:

• Neural feature maps as standard BCx-compressed GPU resources (BC6H, BC1), tightly coupled with minimal MLP decoders (Weinreich et al., 2023, Laurent et al., 6 Jun 2025).
• Learned BC-compatible block mappings (NTBC) that train small per-material neural encoders to output directly BC-compliant packed blocks, enabling fully hardware-compatible runtime decompression with no shader or pipeline modification (Fujieda et al., 2024).
• Architectures that store latent grids at reduced bit-width, with small decoders for each material, optimized for random-access decoding on demand (Vaidyanathan et al., 2023, Farhadzadeh et al., 2024).

Table: Comparison of Select Neural Texture Compression Architectures (format, decoder, random access, filtering)

Architecture	Compression Format	Decoder MLP Size	Random-Access	Hardware Filtering
(Weinreich et al., 2023)	BC6H block, 4x4, mip-pyramid	~34 KB	Yes	Bilinear/trilinear, GPU
(Laurent et al., 6 Jun 2025)	BC1, tile-based, cooperative	~25 KB	Yes	Bilinear/trilinear/anis.
(Fujieda et al., 2024)	BC1/BC4 via NTBC, on-disk	hundreds of KB	Yes	Native BCx paths
(Vaidyanathan et al., 2023)	Quantized 2D feature grids	~27 KB	Yes	None, MLP reconstructs
(Farhadzadeh et al., 2024)	Quantized grids + MLP	~100 KB	Yes	None, MLP reconstructs

6. Performance Characteristics and Benchmark Results

Empirical results across architectures highlight the strengths of neural block-compressed approaches:

• Storage reduction: Block-compressed neural features yield 14–25× compression ratios over raw multi-channel texture sets, and up to 70% storage reduction compared to fixed-rate BC1/BC4 (Laurent et al., 6 Jun 2025, Fujieda et al., 2024).
• Decoding cost: Full-frame decode (4K, multi-channel, anisotropic filter) consistently achieves $w_i \times h_i$61 ms cost on modern GPUs, when run with in-shader matrix-vector code or via hardware-accelerated cooperative vectors (Weinreich et al., 2023, Laurent et al., 6 Jun 2025).
• PSNR: Quality metrics around 41–46 dB for large materials, exceeding BC/ASTC for comparable bitrates (Laurent et al., 6 Jun 2025, Vaidyanathan et al., 2023, Wu et al., 14 Apr 2026).
• No runtime shader modification: Neural block-based architectures (e.g., NTBC) preserve the fully native block decompression pipeline, ensuring integration with legacy engines (Fujieda et al., 2024).

7. Methodological Best Practices and Future Directions

Best practices established across recent work include:

• Training feature maps directly into the target block-compressed format, simulating hardware decompression during optimization to avoid post-hoc quantization artifacts (Weinreich et al., 2023, Wu et al., 14 Apr 2026).
• Keeping runtime decoders shallow (preferably 1–2 layer MLP) to minimize per-pixel computation and facilitate direct mapping to GPU SIMD or matrix-multiply hardware (Weinreich et al., 2023, Laurent et al., 6 Jun 2025).
• Storing multi-mip/scale features to maximize decoder flexibility and spatial/scale smoothness (Weinreich et al., 2023).
• Using quantization-aware training (QAT) for all latent and grid parameters, with final model weights and feature grids stored in low-bit (4–8 bit) form for efficient on-disk and memory representation (Farhadzadeh et al., 2024, Fujieda et al., 2024).
• Enabling fully random-access decoding to support virtual texturing, temporal lightmaps, and on-the-fly asset streaming in dynamic rendering contexts (Wu et al., 14 Apr 2026).

These advances jointly enable real-time, block-compressed neural materials and textures that integrate seamlessly with GPU hardware pipelines, facilitating high-quality photorealistic rendering at low memory costs without custom hardware or expensive compute overhead (Weinreich et al., 2023, Laurent et al., 6 Jun 2025, Fujieda et al., 2024, Vaidyanathan et al., 2023, Farhadzadeh et al., 2024, Wu et al., 14 Apr 2026).