---
title: 'FedBiF: Federated Bit Freezing'
url: https://www.emergentmind.com/topics/federated-bit-freezing-fedbif
type: topic
---

# FedBiF: Federated Bit Freezing

Federated Bit Freezing (FedBiF) is a communication-efficient federated learning (FL) framework in which the model is represented as a multi-bit quantized tensor, while each client updates only a single bit of each parameter in a given communication round and freezes the remaining bits. In the formulation reported for cross-device FL with a central server and \(K\) clients, the server quantizes model parameters and broadcasts them to selected clients, clients train directly in the quantized representation through virtual bits and a straight-through estimator (STE), and the uplink transmits only the activated bit plane. This yields 1 bit-per-parameter (bpp) uplink and typically 3–4 bpp downlink, while reported accuracies remain comparable to FedAvg across five datasets under IID and non-IID partitions [2509.10161].

## 1. Historical setting and problem formulation

FedBiF is situated within the broader class of communication-efficient FL methods that attempt to reduce the cost of repeated model exchange without sacrificing the statistical performance of standard FedAvg. The baseline objective is the usual federated risk
\[
\min_{\mathbf{w}} F(\mathbf{w}) := \sum_{k=1}^{K} p_k F_k(\mathbf{w}), \quad p_k = \frac{|D_k|}{\sum_{j=1}^K |D_j|},
\]
with server-side aggregation over participating clients \(\mathbb{C}^t\) at round \(t\),
\[
\mathbf{w}^{t+1} = \mathbf{w}^{t} + \sum_{k\in\mathbb{C}^t} p_k^t \big(\mathbf{w}_k^{t+1} - \mathbf{w}^{t}\big), \quad p_k^t = \frac{|D_k|}{\sum_{j\in\mathbb{C}^t} |D_j|}.
\]
Communication cost is measured in bpp, so conventional FedAvg corresponds to 32 bpp for both uplink and downlink [2509.10161].

The specific problem FedBiF addresses is the failure mode of post-training quantization in FL. Methods such as FedPAQ, AdaQuantFL, FedDQ, DAdaQuant, LFL, and sign-based uplink compression reduce transmitted precision only after local training. The reported limitation is that local models are optimized in full precision and then projected onto a coarse grid, so quantization error is injected into trained parameters after the fact; this becomes especially harmful at very low bit-widths and under non-IID data [2509.10161].

This placement makes FedBiF conceptually adjacent to several freezing-based lines of work, but with a distinct granularity. CoCoFL freezes and quantizes entire blocks to satisfy device-specific communication, computation, and memory constraints; SmartFreeze progressively freezes blocks for memory reduction; FedTLU masks block updates at the server during language-model fine-tuning; and task-specific methods such as adaptive encoder freezing for MRI-to-CT conversion treat a stable encoder as a globally frozen module. These works freeze layers, blocks, or subnetworks rather than individual bits, but they share the central principle that stabilized or low-utility components need not remain fully trainable or fully communicated [2203.05468].

## 2. Bit-level parameterization and freezing mechanism

FedBiF replaces full-precision local optimization with direct optimization of a quantized representation. For a weight tensor \(\theta\), the server computes a per-layer step size
\[
\alpha = \frac{\|\theta\|_\infty}{2^{m-1}},
\]
and an \(m\)-bit integer quantization
\[
\bar{\theta} = Q(\theta, \alpha, m)
= \text{clamp}\!\left( \left\lfloor \frac{\theta}{\alpha} \right\rceil, -2^{m-1},\, 2^{m-1}-1 \right).
\]
Dequantization is
\[
\hat{\theta} = \alpha \cdot \bar{\theta}.
\]
The quantized integer is decomposed bitwise as
\[
\bar{\theta}^{t} = \sum_{i=0}^{m-1} 2^{i} \cdot b_{i}^{t} - 2^{m-1},
\]
with \(b_i^t \in \{0,1\}\) [2509.10161].

The defining mechanism of FedBiF is that only one bit \(i\) is activated in a round. All other bits are frozen. Their combined contribution is precomputed as
\[
s^{t} = \sum_{j\in [m],\,j\neq i} 2^{j}\cdot b_{j}^{t} - 2^{m-1}.
\]
The active bit is represented through a floating-point virtual bit \(v_i\), with binary realization
\[
b_i = h(v_i) =
\begin{cases}
1, & v_i > 0,\\
0, & \text{otherwise}.
\end{cases}
\]
The local forward weight then becomes
\[
\hat{\theta}^{t} = g(\alpha^{t}, v_{i}^{t}, s^t) = \alpha^{t} \cdot \big(h(v_{i}^{t}) + s^t\big).
\]
During backpropagation, a STE approximates \(\frac{\partial h}{\partial v_i}\approx 1\), so
\[
\frac{\partial f}{\partial v_{i}^{t}} \approx \frac{\partial f}{\partial \hat{\theta}^{t}}.
\]
Virtual bits are initialized so that their sign matches the currently broadcast bit,
\[
v_i = (2\cdot b_i - 1)\cdot |v_i|,
\]
which preserves the initial forward value of the received global model [2509.10161].

This construction differs from ordinary low-bit update quantization in a precise way: the model is always trained in the quantized domain, and the frozen bits are not merely omitted from communication but also held fixed in local optimization. A plausible implication is that FedBiF couples compression and optimization much more tightly than post-hoc quantization, because the training dynamics themselves adapt to the finite-bit representation rather than encountering it only at transmission time.

## 3. Federated protocol and bit scheduling

A FedBiF communication round begins with server-side quantization of the current global model. The server broadcasts \((\bar{\theta}^t, \alpha^t)\) to selected clients together with the index of the activated bit if that index is not fixed by convention. On each client, the received quantized parameter is decomposed into bit planes, the active bit is converted into a trainable virtual bit, the frozen contribution \(s^t\) is precomputed, and local training is performed for several epochs using the reconstructed weights \(\hat{\theta}^t\). After local training, the client uploads only the updated activated bits \(b_i^{t+1}\), which yields 1 bpp uplink communication [2509.10161].

Server aggregation reconstructs each client’s effective quantized parameter from the uploaded activated bit and the globally shared frozen bits. With bit \(i\) active at round \(t\), the reported aggregation formula is
\[
\theta^{t+1} = \alpha^{t} \cdot \bigg( 2^{i} \sum_{k \in \mathbb{C}^{t}} p_k^t \cdot b_{i,l,k}^{t+1} + s^{t} \bigg),
\]
so only the selected bit plane is averaged across clients while the remaining bit planes stay inherited from the previous global model [2509.10161].

The default schedule activates exactly one bit per parameter in each round and cycles the bit index over \(\{0,\dots,m-1\}\). A random one-bit schedule was reported to behave similarly well, whereas fixing one bit forever was poor. On CIFAR-10, cycling a single bit with 4-bit weights gave \(77.4\), \(69.9\), and \(64.8\) under IID, Non-IID-1, and Non-IID-2 respectively, while training more than one bit per round degraded performance under non-IID partitions: 2-bit activation yielded \(76.8\), \(68.1\), \(61.0\), and full 4-bit activation yielded \(77.5\), \(66.3\), \(59.4\) [2509.10161].

This makes the “freezing” in FedBiF dynamic and reversible. A bit plane is frozen only for the current round, not permanently removed from training. In that respect FedBiF is closer to a cyclic coordinate-descent view over bit planes than to one-shot pruning or irreversible layer freezing. By contrast, adaptive encoder freezing for MRI-to-CT conversion freezes the encoder permanently once a stability criterion is met, while FedTLU can freeze or unfreeze blocks across rounds according to roundwise scores [2512.03054].

## 4. Communication profile, empirical results, and sparsity

FedBiF’s communication profile is asymmetric. Downlink sends \(m\)-bit integers plus per-layer scaling factors, giving an effective cost of roughly \(m\) bpp, while uplink sends only one binary bit per parameter. The canonical reported settings are 1/3 bpp or 1/4 bpp for uplink/downlink. Relative to FedAvg’s 32/32 bpp, this corresponds to roughly 32× compression on uplink and 8–10× on downlink, with convergence curves versus cumulative transmitted bits indicating better bit-efficiency than SignSGD, FedPAQ, FedDQ, DAdaQuant, and LFL [2509.10161].

The main reported results are summarized below.

| Dataset / model | FedAvg | FedBiF |
|---|---:|---:|
| FMNIST / CNN | 88.6, 85.9, 82.0 | 89.0, 86.3, 82.3 with 1 / 3 bpp |
| SVHN / CNN | 88.8, 85.1, 84.0 | 88.9, 85.2, 83.9 with 1 / 3 bpp |
| CIFAR-10 / ResNet-18 | 78.1, 70.1, 64.5 | 77.4, 69.9, 64.8 with 1 / 4 bpp |
| CIFAR-100 / ResNet-18 | 40.7, 38.3, 39.0 | 41.3, 38.7, 39.1 with 1 / 4 bpp |
| TinyImageNet / ResNet-18 | 38.3, 36.5, 34.7 | 38.5, 36.2, 34.7 with 1 / 4 bpp |

The triplets are IID, Non-IID-1, and Non-IID-2, where Non-IID-1 is a Dirichlet label split with concentration \(0.3\) and Non-IID-2 is label-skew with each client seeing only 30% of labels [2509.10161].

The empirical claim is not merely that FedBiF remains competitive under heavy compression, but that accuracy differences relative to FedAvg are usually within \(\pm 0.4\%\), and sometimes slightly positive. Robustness checks with 200 clients on CIFAR-10 and with DenseNet-40 on CIFAR-10 reported similarly small gaps versus FedAvg, while LFL remained lower by about 2–4% or 3–8% depending on the setting [2509.10161].

A second empirical property is induced sparsity. Because weights are encoded as multi-bit signed integers,
\[
\bar{\theta} = \sum_{i=0}^{m-1} 2^i b_i - 2^{m-1},
\]
driving the corresponding virtual bits negative can zero out magnitude bits exactly. Reported 2-bit sparsity levels were 47–56% for FMNIST, 54–62% for SVHN, 32–39% for CIFAR-10, 22–26% for CIFAR-100, and 18–22% for TinyImageNet. The paper positions this as a byproduct rather than an auxiliary regularizer [2509.10161].

This combination of bitwise communication reduction and exact zero creation distinguishes FedBiF from layer-freezing systems whose primary gains are in backward-pass elimination or memory relief. SmartFreeze, for example, reports up to 82% memory reduction and up to \(2.02\times\) faster training through progressive block freezing, but it does not operate at bit granularity and does not target 1 bpp uplink transmission [2408.09101].

## 5. Optimization view and convergence properties

FedBiF is analyzed through a virtual-bit objective
\[
\min_{\mathbf{v}} F(\mathbf{v}) := \frac{1}{m} \sum_{i=1}^m f_i(\mathbf{v}),
\]
where \(f_i\) denotes the loss corresponding to activation of bit \(i\). At iteration \(t\), one updates
\[
\mathbf{v}^{t+1} = \mathbf{v}^t - \eta^t \nabla f^t(\mathbf{v}^t),
\]
under assumptions of bounded gradients, \(L\)-smoothness, bounded virtual bits, and bounded model weights. The reported assumptions are
\[
\mathbb{E}\|\nabla f_i(\mathbf{v})\|^2 \le G^2,
\]
\[
\|\nabla f_i(\mathbf{v}_1) - \nabla f_i(\mathbf{v}_2)\|
\le L \|\mathbf{v}_1 - \mathbf{v}_2\|,
\]
\[
\|\mathbf{v}_1 - \mathbf{v}_2\| \le D,
\]
and
\[
\|\mathbf{w}\|_\infty \le P.
\]
Under these conditions, convex and non-convex convergence bounds are derived for the idealized setting in which all bits are trainable, with error floors that decay exponentially in the bit width \(m\) [2509.10161].

For the convex case, with \(\eta^t = \frac{c}{\sqrt{t}}\),
\[
\mathbb{E}\big[F(\bar{\mathbf{v}}^T) - F(\mathbf{v}^*)\big]
\le
\frac{D^2 P + 2 c^2 G^2 P}{c \sqrt{T} 2^{m+2}}
+
\frac{\sqrt{m d} L D P^2}{2^{2m+1}}.
\]
For the non-convex case, with \(\eta^t = \frac{c}{\sqrt{T}}\),
\[
\frac{1}{T} \sum_{t=0}^{T-1} \mathbb{E}\|\nabla F(\mathbf{v}^t)\|^2
\le
\frac{ 2P\big(F(\mathbf{v}^0) - F(\mathbf{v}^*)\big) + PLG^2 c^2 }{ 2^{m+2} c \sqrt{T} }
+
\frac{P^2 G L \sqrt{m d}}{2^{2m+1}}.
\]
The common interpretation is that there is a vanishing term proportional to \(1/(2^m \sqrt{T})\) and a non-vanishing error floor proportional to \(1/2^{2m}\); increasing \(m\) therefore pushes the quantized optimum toward the full-precision regime [2509.10161].

This theoretical framing suggests a useful contrast with other freezing-based FL methods. In wireless FL with parameter freezing and power control, freezing is analyzed as a source of model bias that trades against outage and energy expenditure on a slower timescale. FedBiF instead analyzes quantized bit-plane optimization as the primary training space. The shared conceptual point is that freezing introduces a controlled approximation error, but the object being frozen differs: stabilized parameters in one case, bit planes of quantized parameters in the other [2504.01752].

## 6. Relations to adjacent freezing paradigms

FedBiF belongs to a broader family of “freeze to save resources” ideas, but the resource targeted and the level of intervention vary substantially across papers.

| Method | Freezing unit | Primary stated benefit |
|---|---|---|
| FedBiF | Single bit plane per parameter active; remaining bits frozen | 1 bpp uplink, 3–4 bpp downlink, sparsity |
| CoCoFL | Blocks/layers | Communication, computation, memory under heterogeneous devices |
| SmartFreeze | Progressive blocks | Memory reduction, faster training |
| FedTLU | Server-selected layers/blocks | Robust fine-tuning under non-IID and noisy updates |
| Adaptive encoder freezing | Entire encoder | Time, energy, and CO\(_2\)eq reduction in clinical FL |
| FedOLF | Ordered lower layers | Energy and memory efficiency on IoT devices |

CoCoFL is the closest architectural analogue among earlier works: devices retain the full network topology but freeze selected blocks and quantize frozen blocks to int8, thereby satisfying time, memory, and uplink budgets in synchronous FL. The paper explicitly characterizes this as “layer-wise bit freezing” in a broad sense, since frozen blocks are not updated or uploaded and are run in low precision. FedBiF can be viewed as a finer-grained generalization in which the active object is no longer a block but a single bit plane per parameter [2203.05468].

FedTLU occupies a different point in the design space. It leaves client computation unchanged and performs freezing as a server-side masking operation on aggregated layer updates. Its score
\[
\text{Score}(W_i) = \frac{\|\Delta W_i\|}{\sqrt{n_i} \cdot \text{std}(\Delta W_i)}
\]
selects only certain layers or blocks for application to the global model during fine-tuning. This is a block-level reliability filter rather than a communication codec, but it shows that freezing can also be used to suppress noisy or poisoned directions in non-IID federated optimization [2412.17692].

Task-specific and systems-oriented methods further broaden the term. Adaptive encoder freezing for MRI-to-CT conversion can be interpreted as a coarse-grained, layer-level analogue of FedBiF: once inter-round encoder changes remain below a threshold for a patience window, the encoder is frozen globally, reducing training time, total energy, and CO\(_2\)eq emissions by up to 23% while largely preserving MAE. FedOLF likewise freezes ordered lower layers and supplements this with Tensor Operation Approximation (TOA), reporting at least 0.3%, 6.4%, 5.81%, 4.4%, 6.27% and 1.29% higher accuracy than existing works on several non-iid benchmarks. These methods suggest that “bit freezing” has both a strict meaning—FedBiF’s explicit single-bit update rule—and a broader family resemblance across parameter, layer, and module freezing mechanisms [2512.23200].

A distinct but terminologically related use appears in secure aggregation. PVF, or Partial Vector Freezing, “freezes” most entries of a private vector through linear transforms so that only a \(1/\lambda\) fraction enters the secure aggregation protocol, after which the aggregate can be exactly “thawed.” This is not parameter training or model freezing in the optimization sense, but it demonstrates that “freeze–thaw” constructions can also target cryptographic aggregation overhead rather than model update bandwidth [2312.04920].

## 7. Limitations, misconceptions, and open directions

A common misconception is that FedBiF is simply sign-based communication or post-training quantization with an unusual schedule. The reported formulation rejects both equivalences. Unlike SignSGD, FedBiF preserves a multi-bit parameter representation rather than a pure sign representation. Unlike update quantization methods, it does not first optimize in full precision and then project the result onto a low-bit code; instead, the optimization variable is the virtual bit associated with the currently active bit plane [2509.10161].

Another misconception is that freezing more bits per round should necessarily improve adaptation because the local model becomes less constrained. The ablation on CIFAR-10 shows the opposite under non-IID partitions: activating 2 bits or all 4 bits per round worsened performance relative to activating 1 bit. The paper interprets this as evidence that minimal bit activity regularizes local overfitting and mitigates non-IID drift, while still allowing all bits to be refined over time through cycling [2509.10161].

The reported limitations are also specific. The STE-based optimization is heuristic; rigorous guarantees are provided for an idealized surrogate setting in which all bits are trainable. Bit selection is currently simple round-robin or random rather than importance-aware. Step size \(\alpha\) is chosen by a max-abs heuristic and could be made learnable. These are algorithmic rather than empirical failures, but they identify the present boundary of the method [2509.10161].

The broader literature suggests several concrete directions for extension. One is heterogeneity-aware scheduling of bit activation, by analogy with CoCoFL’s per-device feasible configuration selection or SmartFreeze’s participant selection. Another is server-side importance scoring for bit groups, in the spirit of FedTLU’s targeted layer updates. A further direction is stability-based bit freezing over longer horizons, reminiscent of adaptive encoder freezing or two-timescale parameter freezing in wireless FL. This suggests a possible future split within the FedBiF label itself: a strict usage for the 2025 method of single-bit-per-round quantized optimization, and a broader methodological usage for federated schemes that freeze stabilized bits or bit groups to balance communication, computation, memory, energy, and convergence.

Source: https://www.emergentmind.com/topics/federated-bit-freezing-fedbif