---
title: Learnable Companding Quantization (LCQ)
url: https://www.emergentmind.com/topics/learnable-companding-quantization-lcq
type: topic
---

# Learnable Companding Quantization (LCQ)

Learnable Companding Quantization (LCQ) refers to a family of quantization schemes in which a parameterized, learnable companding (compressing and expanding) function is integrated into the quantization process. This approach enables adaptive, non-uniform placement of quantization levels, shaped via task-driven optimization rather than analytic or heuristic rules. LCQ has shown significant benefits across domains including low-bit deep neural network inference, sensor-to-network pipelines, and end-to-end neural compression, consistently reducing bitrates or quantization error at fixed task performance while maintaining computational efficiency.

## 1. Mathematical Formulations of Learnable Companding Quantization

All LCQ methods begin with a companding function $c_\theta(\cdot)$, parameterized by $\theta$, that transforms the dynamic range of scalars $x$ (weights, activations, or raw measurements) before quantization. This transformation is typically non-linear, allowing denser placement of quantization points around high-importance value ranges, and more sparse placement elsewhere.

In major LCQ frameworks:

- **Piecewise-Linear Companding (Activation/Weight Quantization):** For $x \in [-\alpha, \alpha]$, $v = |x|/\alpha \in [0,1]$ is mapped via a learnable compressor $f_\Theta$ (with $K$ intervals), then quantized uniformly, and finally reconstructed by an expander $f_\Theta^{-1}$, yielding the overall quantized value:
  $$
  Q_L(x; \alpha, \Theta) = \mathrm{sign}(x) \cdot \alpha \cdot g_\Theta\left( \frac{|x|}{\alpha} \right)
  $$
  with
  $$
  g_\Theta(v) = f_\Theta^{-1}(q_b(f_\Theta(v)))\,.
  $$
  Here, $q_b$ is a uniform $b$-bit quantizer on $[0,1]$, with $f_\Theta$ and $f_\Theta^{-1}$ both parameterized by $K$ learnable slopes and breakpoints [2103.07156].

- **Power-Law or A-Law Companding (ADC/Compression):** For input $x \in D$ (e.g., $[0,1]$ or $[-1,1]$), the companding functions take the form:
  $$
  \gamma(x; \gamma) = x^\gamma \qquad \text{(for $x \ge 0$)},
  $$
  or
  $$
  \gamma(x; \gamma, \mu) = \mathrm{sgn}(x-\mu) \cdot |x-\mu|^\gamma\,.
  $$
  These are followed by affine scaling and rounding to an $N$-bit quantizer, with optional analytic inverse for data reconstruction [2509.22448].
  
  In learned image compression, a spatially-adaptive A-law compander is estimated per spatial location by a neural net: $F(x;A)=\mathrm{sgn}(x) \cdot \frac{A|x|}{1+\ln A}$ for $|x|<1/A$, and a logarithmic mapping for higher $|x|$ [2304.12319].

## 2. Optimization and Gradient Flow

LCQ systems jointly optimize the companding parameters together with network weights or subsequent processing stages. The inherent non-differentiability of the rounding/quantization operation is handled using the straight-through estimator (STE):

- **STE for Quantization:** For scalar rounding,
  $$
  \frac{\partial}{\partial z} \mathrm{round}(z) \approx 1,
  $$
  so the gradient is passed as if rounding were the identity map.

- For piecewise-linear companders in LCQ, gradients with respect to the companding parameters are computed by backpropagating through the compressor and expander using analytic chain rule decomposition [2103.07156].

- All LCQ parameters—clipping thresholds, companding function slopes, and, where present, spatial or channelwise adaptation variables—are trained under the task-specific objective (e.g., classification cross-entropy, detection loss, or rate-distortion tradeoff) using standard optimizers such as SGD or Adam [2103.07156, 2509.22448, 2304.12319].

## 3. Architectural Variants and Application Contexts

LCQ methods have been deployed in several technologically important settings:

- **Neural Network Quantization:** The canonical LCQ method uses a learnable piecewise-linear compressor and expander for both weights and activations, followed by uniform quantization. Layerwise "limited weight normalization" is often introduced at the quantization boundary for stability at ultra-low bitwidths [2103.07156].

- **Sensor/ADC Quantization:** A learnable companding law (e.g., power law $\gamma$, possibly with offset $\mu$) is applied to raw sensor readings prior to quantization, with parameters trained jointly with the downstream network. Both unsigned and signed input domains are supported, and the companding function remains low-dimensional (few parameters) for hardware feasibility [2509.22448].

- **Image Compression:** In end-to-end learned image compression schemes, spatially adaptive companding functions (A-law form with per-location scale $A_j$ predicted from local features) are used to warp latent codes before vector quantization. The companding parameters are output by a small CNN, yielding fine-grained adaptation to local statistics, followed by a diamond-lattice vector quantizer [2304.12319].

## 4. Empirical Performance and Task-Level Impact

LCQ schemes consistently outperform uniform and traditional non-uniform quantizers at low bitwidths:

| Setting                        | Uniform Baseline (2–4 bit) | LCQ (2–4 bit)        | Full Precision Gap |
|------------------------------- |----------------------------|----------------------|-------------------|
| ImageNet (ResNet-50) [2103.07156] | 73.8–75.6% top-1           | 75.1–76.6% top-1     | ≤1.7% (2 bit)     |
| COCO Detection (ResNet-50) [2103.07156] | 36.2 AP (W4/A4)         | 37.1 AP (W4/A4)      | 1.2               |
| HAR (WEAR, DeepConvLSTM) [2509.22448] | 63.7 (4 bit linear)      | 70.1 (4 bit γ-Quant) | 0.9 (vs 12 bit)   |
| Raw Image Object Detection [2509.22448] | 0.31 mAP (4 bit linear) | 0.47 mAP (γ-Quant)   | ≈same as sRGB     |
| Learned Compression BD-Rate [2304.12319]| +3.89% (uniform)        | +3.12% (LVQAC best)  | 0.77%             |

On ImageNet and COCO, LCQ eliminates 60–90% of the accuracy gap to full-precision at 2–4 bit depth [2103.07156]. In ADC or sensor-driven applications, learned companding at 4 bits matches or exceeds 12-bit linear or ISP-sRGB performance (see Table above) [2509.22448]. In neural compression, replacing uniform scalar quantization with spatially adaptive companding and vector quantization yields BD-rate reductions up to 0.5 pp on standard datasets [2304.12319].

## 5. Principles of Adaptivity

A central principle of LCQ is the adaptation of quantization point density to either learned statistics or locality:

- **Statistical Adaptivity:** Companding parameters are optimized to place quantization levels where signal or weight distributions are densest or most task-relevant, refining over time as training progresses [2103.07156, 2509.22448].

- **Spatial/Channel Adaptivity:** In image compression, per-location companders are predicted from spatial context, enabling dynamic range adaptation to local feature map variance [2304.12319]. In sensor pipelines, learned companding is often performed per input channel (e.g., accelerometer axis) or dataset-wide [2509.22448].

- **Hardware Feasibility:** LCQ applied at the sensor (ADC) level employs extremely low-dimensional (often single-parameter) companding functions, ensuring ease of hardware realization and minimal overhead [2509.22448].

## 6. Training Considerations and Extensions

Key methodological components for stable and performant LCQ training include:

- **Straight-Through Estimator:** Uniformly used to enable gradient flow through quantization's non-differentiable steps [2103.07156, 2509.22448, 2304.12319].

- **Limited Weight Normalization:** For quantization of neural network weights, applying standardization at the quantizer interface (not throughout the layer) substantially improves stability at very low bitwidths [2103.07156].

- **Companding Interval Resolution:** In piecewise-linear LCQ, increasing the number of companding intervals ($K$) from 4 to 16 produces measurable gains at 2–3 bits with diminishing returns beyond [2103.07156].

- **Extensions:** Potential avenues include joint (vector) companding for entire weight or activation vectors under shared parameters, learned non-uniform breakpoints, dynamic (layerwise) bitwidth selection, and applying spatial adaptation more generally outside image compression [2103.07156, 2304.12319].

## 7. Significance, Trade-Offs, and Broader Impact

Learnable companding quantization unlocks several benefits across efficiency-critical deployments:

- **Memory and Throughput:** LCQ allows neural networks at 2–4 bit precision to retain accuracy close to full-precision counterparts, facilitating memory- and bandwidth-constrained scenarios [2103.07156].

- **Sensor Power Consumption:** In ADC-driven pipelines, reducing bitdepth from 12–16 down to 2–4 with learned companding substantially lowers energy usage, which can comprise up to half of sensor power [2509.22448].

- **Compression Performance:** In learned image compression, spatially adaptive companded quantization delivers consistent rate–distortion improvement over uniform quantization, with minimal added inference cost [2304.12319].

- **Task Preservation:** Across all domains, non-uniform level placement via learnable companding consistently preserves task-relevant information, quantization error is distributed to least-critical regions of the signal, and empirical results demonstrate either matching or exceeding standard baselines at fixed bitrates.

A plausible implication is that LCQ-based quantization will form a core primitive in emerging low-resource, bandwidth-limited, and edge-deployed AI systems, where every quantization bit and fraction of a percent accuracy is significant. The modularity and training synergy of companding functions with end-task objectives enable tailored quantization, efficiently adapted to both data and hardware constraints.

Source: https://www.emergentmind.com/topics/learnable-companding-quantization-lcq