Granular Codebook: Designs and Applications

• Granular codebook is a discretized representation where high-resolution code vectors capture subtle semantic, spatial, or angular distinctions in data.
• It leverages techniques like GMM clustering, neural network refinement, and vector quantization to optimize performance in tasks such as beamforming and spectral efficiency.
• Its design enhances applications in MIMO, RIS configurations, and discrete representation learning while addressing challenges like codebook collapse and scalability.

A granular codebook is a discretized representational structure composed of high-resolution, often high-cardinality code vectors, each corresponding to a compact, semantically or physically meaningful “granule” in the underlying feature, data, or signal space. Granular codebooks are now central to diverse applications in communication systems, neural generative modeling, and representation learning, with their fundamental property being the ability to partition a rich, high-dimensional input space into a large set of finely distinguished, informative atoms. The notion of granularity is quantitatively and qualitatively reflected in the codebook’s capacity to capture local variations, subtle semantic details, or minor geometric/angular distinctions—annulling the crudeness sometimes imposed by legacy coarse quantization schemes.

1. Construction and Purpose of Granular Codebooks

In communication theory and machine learning, codebook construction replaces continuous, high-entropy latent or channel spaces with a finite set of vectors optimized for specific tasks.

• GMM-Based Granular Codebooks: In frequency division duplex (FDD) precoding, a granular codebook is constructed by fitting a K-component Gaussian Mixture Model (GMM) to training channel data $\mathcal{H} = \{ h_m = \mathrm{vec}(H_m) \}$, partitioning the space into clusters (regions of similar channel statistics) via the “responsibility” function:

$$p(k|h) = \frac{p(k)\, \mathcal{N}_\mathbb{C}(h; \mu_k, C_k)}{\sum_{i=1}^{K} p(i)\, \mathcal{N}_\mathbb{C}(h; \mu_i, C_i)}$$

(Turan et al., 2022). Each codebook entry $Q_k$ is tuned to maximize spectral efficiency within its “granule.” This approach generalizes to multi-user and time-evolving contexts.

• Neural/ML-Based Beam Granular Codebooks: In 5G/6G MIMO beam management, neural networks refine traditional DFT or virtual-channel codebooks to fine grid “granular” beams, enabling adaptation to site-dependent multipath or spatial cluster characteristics (Dreifuerst et al., 2023). In this context, granularity refers to the angular or spatial resolution of the codebook, yielding improved RSRP or multiuser spectral efficiency.
• Vector Quantization and Discrete Representation Learning: In VQ-based models, a granular codebook consists of a large number (e.g., thousands) of code vectors in a latent space. Techniques such as CVQ-VAE explicitly maintain high codebook utilization by dynamically reinitializing underused entries (“dead” codevectors), ensuring that the codebook covers the feature distribution at a fine granularity (Zheng et al., 2023). In discrete VAE variants, the granularity of code assignment is further diversified via evidential deep learning mechanisms (Baykal et al., 2023).
• Semantic and Cross-Modal Granular Codebooks: For multi-modal learning, e.g., language-guided VQ (Liang et al., 23 May 2024) and text-augmented VQ (Liang et al., 3 Mar 2025), the granularity is defined by the ability to align codebook atoms not only with image features but with semantic or compositional linguistic representations at word, phrase, and sentence levels. Multi-hierarchical and sampling-based alignment strategies support this semantic granularity.

2. Mathematical Formulations and Optimization

Granular codebook design leverages optimization frameworks—clustering, quantization, or alignment—to maximize task-relevant metrics.

<table> <thead> <tr><th>Domain</th><th>Key Formula</th><th>Optimization Target</th></tr> </thead> <tbody> <tr> <td>FDD Precoding (Turan et al., 2022)</td> <td> $$Q_k = \arg\max_{Q \succeq 0} \frac{1}{|\mathcal{V}_k|} \sum_{h \in \mathcal{V}_k} r(H, Q)$$<br>

$\text{subject to } \operatorname{tr}(Q) \leq \rho$

</td> <td> Maximize average spectral efficiency per GMM partition. </td> </tr> <tr> <td>RIS Codebooks (Yang et al., 2023)</td> <td> $\Phi = \Psi \circ \Gamma(\vec{v})$<br>

$$\Gamma(\vec{v}) = [\sin(\pi C \cdot \vec{v}), \cos(\pi C \cdot \vec{v})]$$

</td> <td> Map spatial coordinates $\vec{v}$ to optimal beam phases with Fourier-encoded INRs. </td> </tr> <tr> <td>Text-Aligned Granular VQ (Liang et al., 3 Mar 2025)</td> <td>

$$\mathcal{L} = \mathcal{L}_{\mathrm{vq}} + \alpha\mathcal{L}_{t_{w}} + \beta\mathcal{L}_{t_{p}} + \gamma\mathcal{L}_{t_{s}}$$

</td> <td> Multi-level semantic alignment: word, phrase, sentence (granularity in codebook semantics). </td> </tr> <tr> <td>Audio Latent Granular Synthesis (Tokui et al., 25 Jul 2025)</td> <td>

$$P(\text{select } B_i) = \frac{\exp(-D_{\mathrm{cos}}(A, B_i)/\tau)}{\sum_j \exp(-D_{\mathrm{cos}}(A, B_j)/\tau)}$$

</td> <td> Probabilistic matching for latent grain resynthesis. </td> </tr> </tbody> </table>

Codebook granularity is reflected in these frameworks by the fine partitioning (or encoding) of input spaces, ensuring that codebook atoms densely and distinctively cover all relevant feature, angular, or semantic “granules.”

3. Granular Codebooks in Communication Systems

Granular codebooks have transformed classic communication system design, especially for high-dimensional MIMO, semantic communications, RIS configuration, and ultra-low bandwidth applications:

• Pilot-Efficient Feedback and Robustness: By clustering channel realizations into granular regions using GMMs, feedback in FDD precoding is reduced to an index, obviating the need for pilot-intensive channel estimation at the mobile terminal (Turan et al., 2022). In the online phase, the mobile terminal selects the index via responsibility evaluation on received pilot signals, thus exploiting codebook granularity for low-overhead and high-robustness adaptation.
• Beamspace and RIS Operation: Granular codebooks in beamspace (MIMO) (Dreifuerst et al., 2023) or RIS phase patterns (Yang et al., 2023) allow beamforming/reflecting directions to be defined at exceedingly fine angular resolutions, optimizing for site-specific scattering or device positioning.
• Digital Modulation Compatibility: Multilevel granular codebooks such as MOC (multi-head octonary codebook) balance granularity with index compactness to align with digital signal constellations, enabling high-quality semantic communication at reduced channel bandwidth ratios (Zhou et al., 2 Jan 2024).
• Composite Semantic Control: Condition codebooks for generative semantic communication are used for compactly encoding salient or task-oriented object features, drastically reducing the bitrate while maintaining semantic fidelity—even outperforming traditional JSCC and DeepJSCC in extreme compression regimes (Ye et al., 7 Apr 2025).

4. Granular Codebooks in Discrete Representation Learning

In neural architectures, granular codebooks are fundamental to the expressive capacity and interpretability of models employing discrete representational bottlenecks:

• Codebook Collapse and Granular Utilization: In classical VQ-VAE frameworks, under-utilization (“collapse”) curtails granularity; only a small set of codevectors is used. Online updating mechanisms (e.g., anchor-based “CVQ-VAE” (Zheng et al., 2023)) address this by reinitializing dead entries based on feature distribution statistics, achieving high codebook perplexity and balanced granular representation.
• Evidential Bayesian Granularity: Hierarchical Bayesian coding with Dirichlet priors (as in EdVAE) (Baykal et al., 2023) increases codebook entropy, forcing more even—and hence granular—distribution of usage, as supported by perplexity and entropy metrics.
• Sparse/Discrete Interpretability: Placing a sparse quantization bottleneck at each layer, as in codebook Transformers, enables the mapping of hidden states to distinct, interpretable codes, each corresponding to disentangled concepts or behavioral control switches (Tamkin et al., 2023).
• Cross-Modal and Semantic-Granular Codebooks: Recent work in LG-VQ (Liang et al., 23 May 2024) and TA-VQ (Liang et al., 3 Mar 2025) extends granularity into multi-modal and semantic dimensions, employing textual priors, multi-granularity alignment (at word/phrase/sentence levels), and sampling-based optimal transport losses to ground each code in fine-grained linguistic or compositional meaning.

5. Applications and Extensions

Granular codebooks are advantageous and often essential in applications requiring high discriminative resolution, adaptive control, or task-specific compactness:

• Massive MIMO and 5G/6G: Enable pilot-efficient, feedback-minimal precoding and beam management even under highly scattering environments (Turan et al., 2022, Dreifuerst et al., 2023).
• RIS-Aided Wireless: Support 3D spatial targeting and fast adaptation without channel model dependency (Yang et al., 2023).
• Semantic and Ultra-Low Bitrate Communication: Allow generative or task-driven image transmission under severe bandwidth constraints, using codebook indices as compact semantic keys (Zhou et al., 2 Jan 2024, Ye et al., 7 Apr 2025).
• Creative Audio and Latent Synthesis: Facilitate fast, training-free, timbre-adaptive audio transformations by leveraging granular latent codebooks as described in neural audio codecs (Tokui et al., 25 Jul 2025).
• Model Interpretability and Control: Offer units of analysis for understanding, attribution, and controlled intervention in deep models by associating codes with precise semantics or behaviors (Tamkin et al., 2023).

6. Performance, Metrics, and Empirical Outcomes

Key performance indicators confirm the effectiveness of granular codebooks:

• Spectral Efficiency and Robustness: In FDD MIMO, GMM-based granular codebooks yield higher probabilities of achieving near-optimal normalized spectral efficiency (nSE) in scenarios with sparse pilot symbols, outperforming conventional Lloyd clustering (Turan et al., 2022).
• Beamforming Signal Quality: Neural granular codebooks achieve 2–3 dB higher RSRP for initial access in 5G NR and support adaptation to new environments with minimal fine-tuning (Dreifuerst et al., 2023).
• Expressivity and Semantic Alignment: Text-aligned granular codebooks in LG-VQ, TA-VQ exhibit improved FID scores and multi-modal task accuracy, with ablations highlighting gains from multi-granular alignment (Liang et al., 23 May 2024, Liang et al., 3 Mar 2025).
• Compression Efficiency: Denoising Diffusion Codebook Models (DDCM) attain state-of-the-art perceptual image compression with codebooks orders of magnitude smaller than the latent space (Ohayon et al., 3 Feb 2025).
• Low-Rate Semantic Fidelity: Condition codebooks in ULBSC deliver superior saliency fidelity and visual quality in generative semantic communication with far less overhead than prior diffuse or pixel-wise schemes (Ye et al., 7 Apr 2025).
• Codebook Perplexity/Usage: Enhanced granular codebooks maintain high perplexity and uniform usage, correlating directly to improved reconstruction accuracy and generative diversity (Zheng et al., 2023, Baykal et al., 2023).

7. Limitations and Future Research Directions

Despite their practical advantages, granular codebooks present open research questions regarding optimality, computational scalability, and theoretical guarantees:

• Scalability to Extreme Cardinality: As granularity increases, memory and search complexity grow, prompting the need for efficient indexation and search (e.g., PQ, product quantization or efficient neural lookup).
• Adaptive/Online Granularity: Dynamic adjustment of codebook size or structure may be required in non-stationary environments or for continual learning applications.
• Semantic Interpretability: While alignment strategies have improved codebook semantics, full interpretability, especially in arbitrary or highly non-stationary domains, remains a challenge.
• Interplay with Feedback/Control: The feedback channel’s capacity may ultimately limit the practical use of highly granular codebooks, especially in wireless or real-time settings.
• Theoretical Characterization: Deeper analysis is warranted to understand why discrete, granular codebooks retain expressiveness and foster diversity, particularly in DDCM and semantic communication frameworks (Ohayon et al., 3 Feb 2025).

Granular codebooks represent an overview of statistical learning, signal theory, and discrete optimization, with broad and expanding impact across communications, machine learning, and creative AI domains.