---
title: Consistency-aware Entropy Reweighting in ML
url: https://www.emergentmind.com/topics/consistency-aware-entropy-reweighting-cer
type: topic
---

# Consistency-aware Entropy Reweighting in ML

Consistency-aware Entropy Reweighting (CER) denotes a class of entropy-guided reweighting mechanisms in which entropy is not treated as a sufficient confidence proxy by itself, but is modulated by some explicit notion of consistency, stability, or reliability. In the explicit use of the term, CER is introduced inside ReTA for test-time adaptation of vision-language models, where prediction entropy is reweighted by semantic committee agreement before samples are admitted to a dynamic cache [2507.09500]. Closely related constructions appear in long-tailed class-incremental learning, multi-head reward aggregation, conformal classification, and rule-based reinforcement learning for large language models, where entropy controls distillation strength, rule weights, conformal score transformations, or local–global loss blending [2605.03364] [2503.20995] [2407.17377] [2508.04138].

## 1. Terminology and scope

The term “CER” is not uniform across the literature. In information-theoretic linguistics, “CER” refers to **Constant Entropy Rate**, the hypothesis that conditional entropy remains constant along a sequence; that usage is unrelated to the machine-learning notion of Consistency-aware Entropy Reweighting [1304.7359]. The linguistic paper states explicitly that it uses “CER” exclusively for Constant Entropy Rate, not for any machine-learning method.

Within machine learning, by contrast, “Consistency-aware Entropy Reweighting” is best understood as a methodological pattern rather than a single canonical algorithm. One formulation uses consistency to inflate or attenuate entropy before entropy is used for cache selection, another uses entropy to modulate distillation strength while consistency is enforced in gradient space, and others use entropy to weight rule heads, transform conformal scores, or interpolate between policy-gradient objectives. This suggests that CER is a family resemblance among methods whose common structure is: an entropy statistic is measured, a consistency statistic is introduced, and some downstream optimization or selection rule is reweighted accordingly.

A frequent misconception is that CER means “use low entropy samples.” The papers do not support that simplified reading. The central point is instead that **low entropy may be unreliable** under distribution shift, imbalance, or outcome collapse, so entropy is reinterpreted through an auxiliary consistency signal.

## 2. Core mechanism

Across the ML formulations, CER-like methods share three ingredients: an entropy term, a consistency term, and a reweighted target. In ReTA, the reweighted target is cache priority, and the entropy of a sample is modified as
\[
H'(\mathbf{z}) = w \cdot H(\mathbf{z}),
\qquad
w = 1 + \log(\mathcal{R}\mathcal{S}),
\]
where \(\mathcal{S} = M/n^\ast\) measures committee stability and \(\mathcal{R}\in\{1,\gamma\}\) measures whether the committee majority agrees with the original prediction [2507.09500].

In long-tailed class-incremental learning, the entropy term is a normalized Shannon entropy over the accumulated class distribution,
\[
\mathcal{H}_{\mathrm{norm}}
=
-\sum_{k=1}^{K_{\mathrm{total}}}
\frac{p_k \log p_k}{\log K_{\mathrm{total}}},
\]
and it modulates the distillation coefficient through
\[
\lambda
=
\sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.
\]
There, the consistency component is not prediction agreement but **gradient consistency**, implemented as
\[
\mathbf{g}_t'
=
\mathbf{g}_t
+
\lambda_{\mathrm{GCR}}
\left(
\mathbf{g}_t - \bar{\mathbf{g}}_{t-1}
\right),
\]
with an exponential moving average \(\bar{\mathbf{g}}_t\) of past gradients [2605.03364].

In COPO, entropy is computed over the empirical distribution of final answers sampled for a single prompt,
\[
H(q)
=
-\sum_{\tau \in T_q} p(\tau)\log p(\tau),
\]
and it controls a soft interpolation between local and global policy optimization,
\[
w_{\mathrm{local}}(H)
=
\sigma\bigl(\gamma(H-\rho)\bigr),
\qquad
w_{\mathrm{global}}(H)
=
1-w_{\mathrm{local}}(H).
\]
High outcome entropy emphasizes local GRPO-style optimization; low outcome entropy shifts weight to a prompt-level global signal [2508.04138].

These formulations are mathematically different, but they instantiate the same design principle: entropy becomes operational only after being filtered through a consistency criterion.

## 3. CER in reliable test-time adaptation of vision-language models

The most direct formulation of Consistency-aware Entropy Reweighting appears in ReTA, a cache-based test-time adaptation method for CLIP-like vision-language models under visual distribution shift [2507.09500]. The problem setting is a dynamic cache of test-time image features and pseudo-labels. Earlier cache-based methods prioritized entries purely by low prediction entropy, but the paper argues that under distribution shift CLIP can be overconfident on wrong predictions, so low-entropy samples are often wrongly labeled and progressively corrupt the cache.

ReTA begins with standard CLIP probabilities
\[
p_{\text{CLIP}}^c
=
\frac{\exp\left(\mathbf{z}\cdot\boldsymbol{t}^c / \tau\right)}
{\sum_{j=1}^C \exp\left(\mathbf{z}\cdot\boldsymbol{t}^j/\tau\right)},
\]
where \(\mathbf{z}=\mathcal{F}_V(x)\) is an image feature and \(\boldsymbol{t}^c=\mathcal{F}_T(\mathcal{P}^c)\) is a class text embedding. Instead of trusting the entropy
\[
H(p) = -\sum_{c=1}^{C} p_{\text{CLIP}}^c \log p_{\text{CLIP}}^c
\]
directly, CER constructs a **semantic committee** of adjacent text embeddings for each class. Multiple prompts \(\{\mathcal{P}_i^c\}_{i=1}^K\) are encoded, intra-class similarities
\[
sim_i^c = \sum_{j=1, j \neq i}^{K} \cos(\boldsymbol{t}_i^c, \boldsymbol{t}_j^c)
\]
are computed, and embeddings are sorted from semantic outliers to centroids. The method then uses ascending progressive binning with
\[
Q_m = \left\lfloor \frac{mK}{M} \right\rfloor
\]
to form adjacent embeddings \(\widehat{\boldsymbol{t}_m^c}\) as averages of the first \(Q_m\) sorted embeddings.

To reduce the modality gap between image and text representations, ReTA applies SVD to the matrix of adjacent text embeddings, keeps the top-\(n\) right singular vectors, and defines a projection
\[
\Phi_{\text{proj}} = \tilde{V}^T\tilde{V},
\qquad
\mathbf{z}_{\text{proj}} = \Phi_{\text{proj}}\mathbf{z}.
\]
The projected feature is then classified by each committee member,
\[
\hat{y}_m
=
\arg\max_c
\left(
\mathbf{z}_{\text{proj}}^\top \widehat{\boldsymbol{t}_m^c}
\right),
\]
yielding a set of committee predictions \(\hat{\mathcal{Y}}=\{\hat{y}_1,\ldots,\hat{y}_M\}\). The majority-voted class \(y^\ast\) and its vote count \(n^\ast\) define a stability factor
\[
\mathcal{S} = \frac{M}{n^\ast}.
\]
Agreement with the original prediction
\[
y = \arg\max_c (\mathbf{z}^\top \widehat{\boldsymbol{t}_M^c})
\]
defines a consistency factor
\[
\mathcal{R}
=
\begin{cases}
1, & \text{if } y^\ast = y,\\
\gamma, & \text{if } y^\ast \neq y.
\end{cases}
\]
These are combined into
\[
w = 1 + \log(\mathcal{R}\mathcal{S}),
\qquad
H'(\mathbf{z}) = w\cdot H(\mathbf{z}).
\]

The practical effect is entirely on cache construction. Samples with low raw entropy but unstable or inconsistent committee behavior receive a larger \(H'(\mathbf{z})\) and are more likely to be evicted or rejected. CER does not alter the retrieval formula itself; it changes which samples are allowed to define the class-wise cache prototypes. The paper characterizes this as a **committee-based, semantic-consistency filter** on top of entropy.

## 4. CER in long-tailed class-incremental learning

A second major instantiation appears in “Dynamic Distillation and Gradient Consistency for Robust Long-Tailed Incremental Learning,” which does not name CER explicitly but matches the pattern exactly [2605.03364]. The setting is long-tailed class-incremental learning (LT-CIL), where new classes arrive sequentially and the class distribution is imbalanced. The paper separates two sources of difficulty: intra-task imbalance and inter-task imbalance.

The entropy-driven part is a dynamic distillation coefficient based on the accumulated global class distribution. If \(p_k=n_k/\sum_j n_j\) is the proportion of class \(k\) among all classes observed so far, then the normalized entropy
\[
\mathcal{H}_{\mathrm{norm}}
=
-\sum_{k=1}^{K_{\mathrm{total}}}
\frac{p_k \log p_k}{\log K_{\mathrm{total}}}
\]
acts as a measure of how balanced the stream is. A time-based schedule
\[
\lambda_{\mathrm{time}} = \sigma\!\left(\frac{t}{T}\right)
\]
is multiplied by this entropy to obtain the final distillation weight
\[
\lambda
=
\lambda_{\mathrm{time}}\cdot \lambda_{\mathrm{entropy}}
=
\sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.
\]
When the class distribution is balanced, \(\mathcal{H}_{\mathrm{norm}}\approx 1\), so distillation can become strong late in the task. Under severe imbalance, \(\mathcal{H}_{\mathrm{norm}}\ll 1\), so distillation remains limited even at late epochs, reducing over-preservation of majority-class bias.

The consistency component is **Gradient Consistency Regularization** (GCR). Let \(\mathbf{g}_t\) be the current gradient and
\[
\bar{\mathbf{g}}_t
=
\beta \bar{\mathbf{g}}_{t-1}
+
(1-\beta)\mathbf{g}_t
\]
its exponential moving average, with \(\beta=0.9\). The update uses
\[
\mathbf{g}_t'
=
\mathbf{g}_t
+
\lambda_{\mathrm{GCR}}
\left(
\mathbf{g}_t - \bar{\mathbf{g}}_{t-1}
\right),
\]
with \(\lambda_{\mathrm{GCR}}=0.1\). The stated effect is to suppress abrupt fluctuations, stabilize training across task boundaries, and reduce forgetting, especially for minority classes. The paper also retains gradient reweighting for intra-task imbalance through
\[
w_c = \frac{\min_{c'} G_{c'}}{G_c},
\]
where \(G_c\) is the cumulative gradient norm for class \(c\).

Viewed through CER, this formulation couples **entropy-aware reweighting of distillation** with **temporal consistency of optimization dynamics**. The resulting objective remains
\[
\mathcal{L}
=
\mathcal{L}_{\text{cls}}
+
\lambda\,\mathcal{L}_{\text{distill}},
\]
but the optimization trajectory is further constrained in gradient space.

## 5. Related CER-style formulations

The same pattern appears in several adjacent areas. The details differ, but each method uses entropy as a reliability-sensitive control variable rather than a standalone score.

| Paper | Consistency signal | Reweighted quantity |
|---|---|---|
| ENCORE [2503.20995] | Rule reliability via low rating entropy | Multi-head reward aggregation weights |
| ER-CP [2407.17377] | Alignment between predictive uncertainty and set size | Input-dependent logit scaling for conformal scores |
| COPO [2508.04138] | Outcome consistency across sampled responses | Mixing weights for local and global policy losses |

ENCORE computes per-rule discrete entropies \(H(\psi_k)\) over rule-level safety ratings and assigns
\[
w_k
=
\frac{\exp(-H(\psi_k)/\tau)}
{\sum_{j=1}^{R}\exp(-H(\psi_j)/\tau)},
\]
with default \(\tau=2\). Rules with high rating entropy are treated as less reliable and receive lower weight in the aggregated scalar reward \(\phi(x,y)=\sum_k w_k \psi_k(x,y)\) [2503.20995]. The paper further argues, through a Bradley–Terry analysis, that maximally entropic rules naturally receive minimal weighting.

Entropy Reweighted Conformal Classification applies entropy directly to logits. For classifier logits \(\mathbf z(X)\) and predictive entropy
\[
H(X)=-\sum_{k=1}^{K} f_k(X)\log f_k(X),
\]
it defines
\[
\tilde z_k(X)=\frac{z_k(X)}{H(X)\cdot T},
\qquad
\tilde f_k(X)
=
\frac{\exp(\tilde z_k(X))}
{\sum_j \exp(\tilde z_j(X))}.
\]
These reweighted probabilities replace the original probabilities inside APS-style conformal scores, and a validation split selects the temperature \(T\) that minimizes average prediction-set size [2407.17377]. The paper interprets this as making conformal set sizes more consistent with local uncertainty.

COPO uses entropy over the empirical distribution of final answers within a sampled response group for a prompt,
\[
H(q)
=
-\sum_{\tau\in T_q} p(\tau)\log p(\tau),
\]
then blends a local GRPO-style objective with a prompt-level global objective:
\[
w_{\text{local}}(H)=\sigma(\gamma(H-\rho)),
\qquad
w_{\text{global}}(H)=1-w_{\text{local}}(H).
\]
Low entropy indicates high intra-group consistency and shifts weight to the global signal; high entropy indicates answer diversity and shifts weight to local optimization [2508.04138]. This is a direct entropy-based reweighting of policy-gradient components.

## 6. Empirical profile, implementation patterns, and limitations

Empirically, CER-style methods are presented as reliability mechanisms under conditions where raw entropy is known to be brittle. In ReTA, Table 3 reports a baseline of \(66.04\%\) on Natural Distribution Shifts and \(69.79\%\) on Cross-Datasets; adding CER alone raises these to \(66.53\%\) and \(70.41\%\), and the full ReTA model reaches \(67.29\%\) and \(71.37\%\). The same study reports that ascending progressive binning reaches \(71.37\%\), versus \(70.47\%\) for descending and \(69.41\%\) for uniform, and that SVD projection reaches \(71.37\%\), versus \(70.57\%\) for PCA and \(70.99\%\) for LDA [2507.09500]. These numbers support the claim that the consistency mechanism is not incidental but structurally tied to performance.

In LT-CIL, experiments on CIFAR-100-LT, ImageNetSubset-LT, and Food101-LT show consistent accuracy improvements of up to \(5.0\%\), with particularly strong gains in the challenging In-ordered setting. The paper also reports a training-time increase of about \(1.3\%\) and no additional inference cost, because the method modifies only training-time weighting and gradients rather than model architecture [2605.03364]. This supports the interpretation of CER as a lightweight control mechanism rather than a heavy auxiliary module.

In multi-head reward modeling, ENCORE reports a RewardBench safety score of \(88.5\), compared with \(85.5\) for uniform weights and \(86.0\) for MoE weights; an entropy-based top-5 rule selection reaches \(87.3\). In conformal classification, ER-CP on AG News at \(\alpha=0.10\) reports coverage \(0.898\) and size \(1.011\), versus APS coverage \(0.903\) and size \(1.163\). In rule-based RL for LLM reasoning, COPO with Qwen2.5-Instruct 7B improves MATH-500 mean@8 from \(63.58\%\) to \(65.8\%\) and maj@8 from \(66.65\%\) to \(69.27\%\) [2503.20995] [2407.17377] [2508.04138]. Taken together, these results suggest that entropy reweighting is most useful when entropy alone is misaligned with the downstream notion of reliability.

The limitations are equally consistent across papers. ENCORE notes that a rule may be high-entropy but still conceptually important, or low-entropy but trivial, and that the method depends on judge quality and does not explicitly encode rule importance. ER-CP leaves the exact effect of validation-based temperature selection on finite-sample validity to future work. COPO assumes enough variation in prompt difficulty across a mini-batch to maintain a non-zero batch standard deviation for the global reward, and it does not outperform GRPO on Qwen2.5-Math-1.5B-Instruct. ReTA, for its part, depends on multiple prompts per class, projection dimensionality, and committee size; the reported best settings use cache size \(3\) per class, \(M=3\), and \(n=64\) [2503.20995] [2407.17377] [2508.04138] [2507.09500].

A final point of clarification follows from the surveyed literature. CER is not a single objective with a universal formula. The explicit ReTA formulation, the LT-CIL formulation, and the CER-like variants in reward modeling, conformal prediction, and RL all use different entropy definitions, different consistency notions, and different reweighted quantities. What unifies them is narrower and more precise: entropy is treated as a useful but unreliable signal, and consistency is introduced to decide when, where, and how that signal should influence learning or selection.

Source: https://www.emergentmind.com/topics/consistency-aware-entropy-reweighting-cer