Jointly Optimized Codebook Construction (JOCC)

• JOCC is a co-design principle that jointly optimizes codebooks and associated modules to enhance quantization, inference, and transmission performance.
• It leverages discrete representations via learnable codebooks with objectives such as mutual information maximization, entropy regularization, and channel robustness.
• JOCC applications show improvements in metrics like PSNR, BER, and spectral efficiency across diverse areas including semantic communications, beamforming, and model compression.

Jointly Optimized Codebook Construction (JOCC) denotes a family of formulations in which a codebook is optimized together with the mappings, inference modules, or transmission mechanisms that use it. In the cited literature, the term spans codebook-enabled quantization for digital semantic communications, site-specific limited-feedback beamforming, memory-footprint compression through jointly learnable codebooks and mappings, second-order representation learning with codebook-conditioned factorization, code-domain NOMA autoencoders, Bayes-decoded error-correction code design, and joint BS/RIS beam codebooks for near-field beam training (Wang et al., 8 Oct 2025, Zhao et al., 16 Apr 2026, Yvinec et al., 2023, Jacob et al., 2019, Han et al., 2021, Wu, 2018, Zhang et al., 26 Aug 2025). The common feature is that the codebook is not fixed a priori: it is part of a system-level optimization objective such as mutual information, reconstruction fidelity, CSI-capture efficiency, BER, Bayes risk, or beam-pattern matching.

1. Terminological scope and definitional core

In current arXiv usage, JOCC is not tied to a single canonical architecture. The 2025 semantic-communications paper studies a “theoretically-grounded codebook” through joint optimization of quantization efficiency, transmission efficiency, and robust performance; the 2026 beamforming paper uses JOCC for the coupled design of a probing codebook and a subspace-inference network; the 2023 compression paper presents the same general idea under the name “jointly learnable codebooks and mappings” (JLCM); and the 2019 retrieval paper presents a related formulation as “Joint Optimization of Codebook and Factorization” (JCF) (Wang et al., 8 Oct 2025, Zhao et al., 16 Apr 2026, Yvinec et al., 2023, Jacob et al., 2019). This suggests that the acronym is best understood as a co-design principle rather than a standardized single algorithm.

The term “codebook” also changes meaning with application. In vector-quantized semantic communication it is a discrete latent vocabulary or constellation lookup; in model compression it is a set of quantization centroids assigned to weights; in beamforming it is a probing beam set or a set of BS/RIS beam codewords; and in error-correction coding it is a mapping from source symbols to binary codewords (Huang et al., 3 Mar 2026, Zhang et al., 26 Aug 2025, Wu, 2018).

Setting	Jointly optimized objects	Representative objective
Digital semantic communications	codebook, encoder/decoder, channel-aware loss	$$L_{\mathrm{total}} = L_{\mathrm{task}} + L_Q + \lambda L_{\mathrm{dist}}$$
Site-specific limited-feedback beamforming	probing codebook $B$, inference network $\Psi_\Theta$	$$\{B^*,\Theta^*\} = \arg\max_{B,\Theta} E_{h\sim p_s}[\eta_p(h;B,\Psi_\Theta)]$$
Model compression	multiple codebooks and index-score tensor $I$	$$\mathcal L(C,I)=\|\widetilde f(\tilde X)-f(X)\|_2^2+\mathcal L_1+\lambda\mathcal L_2$$
XL-RIS beam training	BS codeword $w_t$, RIS phases $\bm\phi$, phase-adjust $\psi$	$$\min \|\bm\phi^H\widetilde{\mathbf H}(w_t)-(p\odot\psi)^T\|_2^2$$

2. Shared mathematical structure

A recurrent JOCC pattern is discrete representation induced by nearest-neighbor partitions or softened assignments. In the semantic-communications derivation, a learnable codebook $B$0 defines Voronoi cells

$B$1

and the quantizer $B$2 is

$B$3

The same paper treats the continuous semantic feature $B$4 and discrete index $B$5, derives

$B$6

and uses the empirical entropy

$B$7

to regularize codeword utilization through

$B$8

or an equivalent cross-entropy form (Wang et al., 8 Oct 2025).

Other JOCC formulations retain the same discrete core but modify the assignment mechanism. In compression, one learns an index-score tensor $B$9, defines soft assignments

$\Psi_\Theta$0

and constructs

$\Psi_\Theta$1

The per-layer objective combines a distillation loss, a weight-reconstruction term, and a one-hot regularizer,

$\Psi_\Theta$2

A custom gradient update replaces the standard $\Psi_\Theta$3 with

$\Psi_\Theta$4

to enforce a proximal search of codebooks and mappings (Yvinec et al., 2023).

In second-order representation learning, soft assignments to a codebook $\Psi_\Theta$5 are

$\Psi_\Theta$6

and the codebook is integrated directly into bilinear pooling and low-rank factorization. In the compact JCF form,

$\Psi_\Theta$7

with end-to-end optimization under the N-pair loss plus weight decay (Jacob et al., 2019).

A further generalization appears in ESC-MVQ, where JOCC means jointly training multiple codebooks $\Psi_\Theta$8 and trainable bit-flip probabilities $\Psi_\Theta$9 under a parallel-BSC model,

$$\{B^*,\Theta^*\} = \arg\max_{B,\Theta} E_{h\sim p_s}[\eta_p(h;B,\Psi_\Theta)]$$0

together with an end-to-end reconstruction loss and codebook regularization (Shin et al., 16 Apr 2025).

3. Semantic communications and channel-aware JOCC

Digital semantic communication is the area in which JOCC is most explicitly tied to quantization, mutual information, and channel robustness. The 2025 theoretically grounded formulation establishes a formal equivalence between semantic synonymy and Voronoi-based many-to-one quantization, derives the mutual information objective $$\{B^*,\Theta^*\} = \arg\max_{B,\Theta} E_{h\sim p_s}[\eta_p(h;B,\Psi_\Theta)]$$1, introduces entropy-regularized end-to-end codebook training, and models channel-induced semantic distortion under bit-flip errors through

$$\{B^*,\Theta^*\} = \arg\max_{B,\Theta} E_{h\sim p_s}[\eta_p(h;B,\Psi_\Theta)]$$2

leading to

$$\{B^*,\Theta^*\} = \arg\max_{B,\Theta} E_{h\sim p_s}[\eta_p(h;B,\Psi_\Theta)]$$3

Its full joint objective is

$$\{B^*,\Theta^*\} = \arg\max_{B,\Theta} E_{h\sim p_s}[\eta_p(h;B,\Psi_\Theta)]$$4

implemented with a VQ-VAE backbone, straight-through estimation, Adam, learning rate $$\{B^*,\Theta^*\} = \arg\max_{B,\Theta} E_{h\sim p_s}[\eta_p(h;B,\Psi_\Theta)]$$5, batch size $$\{B^*,\Theta^*\} = \arg\max_{B,\Theta} E_{h\sim p_s}[\eta_p(h;B,\Psi_\Theta)]$$6, $$\{B^*,\Theta^*\} = \arg\max_{B,\Theta} E_{h\sim p_s}[\eta_p(h;B,\Psi_\Theta)]$$7 for the entropy term, $$\{B^*,\Theta^*\} = \arg\max_{B,\Theta} E_{h\sim p_s}[\eta_p(h;B,\Psi_\Theta)]$$8 for the channel loss, 64-QAM modulation, Rayleigh fading, and codebook size fixed at $$\{B^*,\Theta^*\} = \arg\max_{B,\Theta} E_{h\sim p_s}[\eta_p(h;B,\Psi_\Theta)]$$9 for the reported results. On image reconstruction tasks at SNR $I$0 dB, the reported improvement is $I$1 in PSNR and $I$2 in LPIPS compared to existing codebook designs; ablations report that “+Index Entropy” recovers balanced $I$3 and improves PSNR at high SNR, while “+Channel-Aware” sharply reduces LPIPS at low SNR by minimizing semantic drift under bit flips (Wang et al., 8 Oct 2025).

A distinct two-stage JOCC-style semantic pipeline appears in the Transformer-based generative system. Stage 1 jointly trains semantic encoder $I$4, codebook $I$5, and decoder $I$6 with a VQ-VAE-style loss that combines $I$7, $I$8, and $I$9:

$$\mathcal L(C,I)=\|\widetilde f(\tilde X)-f(X)\|_2^2+\mathcal L_1+\lambda\mathcal L_2$$0

The codebook has size $$\mathcal L(C,I)=\|\widetilde f(\tilde X)-f(X)\|_2^2+\mathcal L_1+\lambda\mathcal L_2$$1 and dimension $$\mathcal L(C,I)=\|\widetilde f(\tilde X)-f(X)\|_2^2+\mathcal L_1+\lambda\mathcal L_2$$2. Stage 2 freezes encoder, decoder, and codebook, then trains a nine-block Transformer encoder with $$\mathcal L(C,I)=\|\widetilde f(\tilde X)-f(X)\|_2^2+\mathcal L_1+\lambda\mathcal L_2$$3, eight heads, feed-forward dimension $$\mathcal L(C,I)=\|\widetilde f(\tilde X)-f(X)\|_2^2+\mathcal L_1+\lambda\mathcal L_2$$4, and learned $$\mathcal L(C,I)=\|\widetilde f(\tilde X)-f(X)\|_2^2+\mathcal L_1+\lambda\mathcal L_2$$5D positional embeddings to recover the correct codebook indices from noisy latent maps. On FFHQ-test, the reported averages are: at $$\mathcal L(C,I)=\|\widetilde f(\tilde X)-f(X)\|_2^2+\mathcal L_1+\lambda\mathcal L_2$$6 dB, JOCC yields PSNR $$\mathcal L(C,I)=\|\widetilde f(\tilde X)-f(X)\|_2^2+\mathcal L_1+\lambda\mathcal L_2$$7, SSIM $$\mathcal L(C,I)=\|\widetilde f(\tilde X)-f(X)\|_2^2+\mathcal L_1+\lambda\mathcal L_2$$8, and LPIPS $$\mathcal L(C,I)=\|\widetilde f(\tilde X)-f(X)\|_2^2+\mathcal L_1+\lambda\mathcal L_2$$9; at $w_t$0 dB, JOCC yields PSNR $w_t$1, SSIM $w_t$2, and LPIPS $w_t$3 (Ye et al., 2024).

The satellite-terrestrial SFSC framework uses JOCC at the interface of semantic coding and digital modulation. Here the semantic encoder $w_t$4 and semantic codebook $w_t$5 are jointly optimized, with the codebook acting as both quantizer and constellation lookup. The composite objective is

$w_t$6

where $w_t$7 is end-to-end MSE, $w_t$8 is cross-entropy over indices, and $w_t$9 is a VQ-VAE style regularizer. The framework further injects instantaneous SNR through FiLM layers,

$\bm\phi$0

The reported hyperparameters are $\bm\phi$1, embedding dimension $\bm\phi$2, batch size $\bm\phi$3, initial learning rate $\bm\phi$4 with cosine annealing to $\bm\phi$5, $\bm\phi$6, $\bm\phi$7, and $\bm\phi$8. Under SL-SNR $\bm\phi$9 dB, the reported PSNR is $\psi$0 dB versus $\psi$1 dB for digital joint coding and modulation, with spectral efficiency $\psi$2 versus $\psi$3 for 64-QAM, corresponding to a $\psi$4 bandwidth saving; in the MDMA scenario, CS-MDMA with JOCC achieves a $\psi$5–$\psi$6 dB PSNR improvement at $\psi$7 dB SNR over classical MDMA and NOMA-JSCC (Huang et al., 3 Mar 2026).

ESC-MVQ extends the semantic-communication interpretation of JOCC from one codebook to many. It jointly trains multiple VQ codebooks and their associated bit-flip probabilities with a single encoder-decoder pair, then solves an alternating communication-strategy problem over codebook assignment, modulation order, and power allocation. The reported empirical outcome is up to $\psi$8–$\psi$9 dB PSNR gain over single-codebook schemes under the same rate, adaptation over a $$\min \|\bm\phi^H\widetilde{\mathbf H}(w_t)-(p\odot\psi)^T\|_2^2$$0 dB SNR range, and a $$\min \|\bm\phi^H\widetilde{\mathbf H}(w_t)-(p\odot\psi)^T\|_2^2$$1-fold reduction in model storage compared to separately trained single-codebook networks (Shin et al., 16 Apr 2025).

4. Feedback, coding, and beam-oriented JOCC

In limited-feedback beamforming, JOCC becomes a coupled design problem between measurement codebooks and inference networks. The site-specific Type-II framework defines a probing codebook $$\min \|\bm\phi^H\widetilde{\mathbf H}(w_t)-(p\odot\psi)^T\|_2^2$$2 with unit-norm columns, an inference network $$\min \|\bm\phi^H\widetilde{\mathbf H}(w_t)-(p\odot\psi)^T\|_2^2$$3, and an inferred subspace basis $$\min \|\bm\phi^H\widetilde{\mathbf H}(w_t)-(p\odot\psi)^T\|_2^2$$4 that is orthonormalized so $$\min \|\bm\phi^H\widetilde{\mathbf H}(w_t)-(p\odot\psi)^T\|_2^2$$5. The central objective maximizes the normalized CSI-capture efficiency

$$\min \|\bm\phi^H\widetilde{\mathbf H}(w_t)-(p\odot\psi)^T\|_2^2$$6

through

$$\min \|\bm\phi^H\widetilde{\mathbf H}(w_t)-(p\odot\psi)^T\|_2^2$$7

The RSRP measurement vector is

$$\min \|\bm\phi^H\widetilde{\mathbf H}(w_t)-(p\odot\psi)^T\|_2^2$$8

and the offline solver updates both $$\min \|\bm\phi^H\widetilde{\mathbf H}(w_t)-(p\odot\psi)^T\|_2^2$$9 and $B$00 via mini-batch backpropagation. Under standard smoothness and bounded-variance assumptions, mini-batch SGD converges in expectation to a first-order stationary point at rate $B$01. The reported results include an ablation in “asu_campus_3p5” with $B$02 for JOCC versus $B$03 for random and DFT probing; online UE complexity is $B$04, whereas Type-II requires $B$05 (Zhao et al., 16 Apr 2026).

In code-domain NOMA, JOCC is realized as an autoencoder for multi-user multidimensional modulation. Han et al. formulate a joint optimization over the multi-user constellation $B$06, bit-to-symbol mappings $B$07, and resource-mapping matrix $B$08, subject to a power constraint. The distinctive architectural element is dense resource mapping combined with a global power-normalization layer,

$B$09

so that the sum of powers across all users and resources is fixed while power allocation remains flexible. Training proceeds in two stages: first with a loss weighted by the Hamming distance between true and decoded bits,

$B$10

and then with pure Euclidean shaping. In the reported $B$11, $B$12, $B$13 setting, JOCC reaches BER $B$14 at $B$15 dB $B$16, while the equivalent single-user MDM autoencoder achieves $B$17 dB, conventional SCMA lies at $B$18 dB, and a power-imbalanced SCMA heuristic lies at $B$19 dB (Han et al., 2021).

An earlier coding-theoretic JOCC formulation treats the codebook itself as the object of source-symbol-aware error-control design. Given source symbols $B$20, a codebook $B$21 assigns each $B$22 a binary codeword $B$23, and a decoder $B$24 is chosen to minimize

$B$25

For any fixed codebook, the Bayes-optimal decoder is

$B$26

The JOCC search alternates between Bayes-decoder updates and codebook moves such as flipping one bit in a single codeword, swapping two entire codewords, or permuting columns. At SNR $B$27 dB for rate-$B$28 codes, the reported $B$29 values are $B$30 for Hamming hard/soft/Bayes decoding and $B$31 for JOCC-optimized $B$32 with $B$33; the corresponding $B$34 values are $B$35 for Hamming and $B$36 for JOCC-optimized $B$37 with $B$38 (Wu, 2018).

Near-field XL-RIS beam training provides yet another meaning of JOCC, now as joint construction of BS precoders and RIS phase-shift codewords. At each beam-training level, JOCC minimizes

$B$39

subject to the BS power limit $B$40, unit-modulus phase-adjust variables, and $B$41-bit discrete RIS phases. The alternating-optimization procedure updates the BS codeword in closed form,

$B$42

then updates RIS phases via an IPDD-based projection onto the $B$43-PSK set, and finally updates

$B$44

The reported per-iteration complexity is

$B$45

empirical convergence occurs in $B$46–$B$47 outer iterations, and runtime is reported as $B$48 slower than SOCC for $B$49. In achievable-rate comparisons after beam training, JOCC is reported as approximately equal to ideal-RIS, with SA-BS codebook about $B$50–$B$51 dB worse (Zhang et al., 26 Aug 2025).

5. Compression and representation learning

In network compression, JOCC is centered on multi-codebook weight quantization with no mapping overhead. The method clusters output neurons, permanently reorders rows of the weight matrix so neurons in the same cluster become contiguous, and ties each row to one of $B$52 distinct codebooks by the rule $B$53. The resulting quantization scheme allows different groups to use different codebooks while avoiding the memory-expensive mapping used by prior multi-codebook methods. Optimization is performed one layer at a time on a calibration batch, with gradients propagated through quantized weights and a proximal index update that favors small moves toward nearby codewords rather than jumps toward extreme values (Yvinec et al., 2023).

The reported empirical profile is broad. On ImageNet-trained ResNet-18, with a per-tensor fp16$B$543-bit compression target $B$55, the fully optimized method reaches $B$56 top-1 versus $B$57 for the fp16 baseline and $B$58 for NUPES. On ViT-b16 at the same $B$59, the reported top-1 is $B$60 versus $B$61 fp16. On Stable Diffusion v2.0 at $B$62-bit weights ($B$63), the initialization alone recovers CLIP score $B$64 versus $B$65 for PowerQuant. On Llama-7B, fp16$B$663-bit compression yields $B$67 on the common-sense benchmark versus $B$68 for OPTQ and $B$69 for RED++, while at $B$70 the footprint is $B$71 GB and the retained score is $B$72; the abstract summarizes this as compression to “2Go” and loading on “5-year-old smartphones” (Yvinec et al., 2023).

The second-order representation-learning variant integrates a trainable codebook into compact bilinear pooling. Starting from local descriptors $B$73, the standard bilinear feature

$B$74

is augmented by codeword-conditioned soft assignments $B$75, and then factorized jointly with low-rank projections. The JCF-N form

$B$76

uses $B$77 parameters for $B$78 and the same for $B$79, while JCF-N-R replaces codeword-specific projections with shared basis projections $B$80, $B$81 and recombination matrices $B$82, reducing the count to $B$83 (Jacob et al., 2019).

This JOCC/JCF representation is trained end-to-end under the N-pair loss and optional codeword normalization. On Stanford Online Products, the reported recall@1 is $B$84 for JCF-32 and $B$85 for JCF-32-8, compared with $B$86 for HTL, $B$87 for Proxy-NCA, and $B$88 for Margin. On CUB-200-2011, JCF-32 reaches $B$89 versus $B$90 for Ge. On Cars-196, JCF-32 reaches $B$91 versus $B$92 for Ge. Parameter counts range from $B$93 M for JCF-4-4 to $B$94 M for JCF-32-32, with JCF-32-8 reported at $B$95 M parameters (Jacob et al., 2019).

6. Recurring design patterns, trade-offs, and common misconceptions

A persistent misconception is that JOCC simply means “using a learnable codebook.” The cited work shows a stricter pattern: the codebook is almost always coupled to another optimized object and to an explicit system loss. In semantic communication, that coupling may be entropy regularization, mutual information, or channel-aware semantic distortion; in beamforming it is CSI-capture efficiency through a learned inference subspace; in coding it is Bayes risk under a significance-aware loss; and in second-order retrieval it is factorization under metric learning (Wang et al., 8 Oct 2025, Zhao et al., 16 Apr 2026, Wu, 2018, Jacob et al., 2019). This suggests that the defining characteristic of JOCC is joint system optimization, not merely codeword learning.

A second misconception is that JOCC belongs only to latent quantization. The surveyed papers use the term for VQ codebooks, multiple codebooks with trainable bit-flip probabilities, beam-probing codebooks, NOMA multidimensional constellations, source-symbol ECC codebooks, and joint BS/RIS beam codewords (Shin et al., 16 Apr 2025, Han et al., 2021, Zhang et al., 26 Aug 2025). The shared abstraction is a discrete design space whose geometry is made task-aware by end-to-end optimization.

The main trade-offs also recur across domains. Codebook cardinality affects both representation fidelity and robustness: in the theoretically grounded semantic formulation, $B$96 is chosen by minimizing $B$97 and is swept in practice over values such as $B$98; in the Transformer-based generative system, “the codebook size $B$99 trades off reconstruction detail vs. robustness to index errors” (Wang et al., 8 Oct 2025, Ye et al., 2024). Joint optimization often improves end performance while relocating complexity: the site-specific beamforming design “pushes the heavy inference into the BS,” reducing UE complexity to $\Psi_\Theta$00; the compression method incurs an offline calibration run over each layer and a one-time neuron reordering; the XL-RIS design yields the highest beam-focusing accuracy but with higher design time and memory than SOCC (Zhao et al., 16 Apr 2026, Yvinec et al., 2023, Zhang et al., 26 Aug 2025).

Resource footprints remain application-specific rather than uniformly small. In satellite-terrestrial semantic forwarding, the total model is reported at approximately $\Psi_\Theta$01 M parameters and approximately $\Psi_\Theta$02 GFLOPs per $\Psi_\Theta$03 image, the codebook occupies approximately $\Psi_\Theta$04 KB, and the on-board satellite workload is approximately $\Psi_\Theta$05 GFLOPs per image; in contrast, the Transformer-based generative system explicitly notes that training a large VQ-AE and a nine-block Transformer end-to-end is computationally demanding (Huang et al., 3 Mar 2026, Ye et al., 2024). The overall literature therefore does not support a universal claim that JOCC is either lightweight or heavyweight. It supports a narrower conclusion: JOCC systematically exchanges additional offline or centralized optimization for codebooks whose discrete structure is aligned with the downstream distortion measure, channel model, or task objective.