Consistency-aware Entropy Reweighting in ML

• CER is an entropy-guided reweighting approach that adjusts prediction entropy with consistency signals to mitigate overconfidence during distribution shifts.
• It modulates entropy via measures such as semantic committee agreement or gradient smoothing, recalibrating confidence for improved learning.
• CER enhances mechanisms in cache construction, distillation, and policy optimization, benefiting applications from vision-language models to long-tailed learning.

Consistency-aware Entropy Reweighting (CER) denotes a class of entropy-guided reweighting mechanisms in which entropy is not treated as a sufficient confidence proxy by itself, but is modulated by some explicit notion of consistency, stability, or reliability. In the explicit use of the term, CER is introduced inside ReTA for test-time adaptation of vision-LLMs, where prediction entropy is reweighted by semantic committee agreement before samples are admitted to a dynamic cache (Liang et al., 13 Jul 2025). Closely related constructions appear in long-tailed class-incremental learning, multi-head reward aggregation, conformal classification, and rule-based reinforcement learning for LLMs, where entropy controls distillation strength, rule weights, conformal score transformations, or local–global loss blending (Sakai et al., 5 May 2026, Li et al., 26 Mar 2025, Luo et al., 2024, Han et al., 6 Aug 2025).

1. Terminology and scope

The term “CER” is not uniform across the literature. In information-theoretic linguistics, “CER” refers to Constant Entropy Rate, the hypothesis that conditional entropy remains constant along a sequence; that usage is unrelated to the machine-learning notion of Consistency-aware Entropy Reweighting (Ferrer-i-Cancho et al., 2013). The linguistic paper states explicitly that it uses “CER” exclusively for Constant Entropy Rate, not for any machine-learning method.

Within machine learning, by contrast, “Consistency-aware Entropy Reweighting” is best understood as a methodological pattern rather than a single canonical algorithm. One formulation uses consistency to inflate or attenuate entropy before entropy is used for cache selection, another uses entropy to modulate distillation strength while consistency is enforced in gradient space, and others use entropy to weight rule heads, transform conformal scores, or interpolate between policy-gradient objectives. This suggests that CER is a family resemblance among methods whose common structure is: an entropy statistic is measured, a consistency statistic is introduced, and some downstream optimization or selection rule is reweighted accordingly.

A frequent misconception is that CER means “use low entropy samples.” The papers do not support that simplified reading. The central point is instead that low entropy may be unreliable under distribution shift, imbalance, or outcome collapse, so entropy is reinterpreted through an auxiliary consistency signal.

2. Core mechanism

Across the ML formulations, CER-like methods share three ingredients: an entropy term, a consistency term, and a reweighted target. In ReTA, the reweighted target is cache priority, and the entropy of a sample is modified as

$$H'(\mathbf{z}) = w \cdot H(\mathbf{z}), \qquad w = 1 + \log(\mathcal{R}\mathcal{S}),$$

where $\mathcal{S} = M/n^\ast$ measures committee stability and $\mathcal{R}\in\{1,\gamma\}$ measures whether the committee majority agrees with the original prediction (Liang et al., 13 Jul 2025).

In long-tailed class-incremental learning, the entropy term is a normalized Shannon entropy over the accumulated class distribution,

$$\mathcal{H}_{\mathrm{norm}} = -\sum_{k=1}^{K_{\mathrm{total}}} \frac{p_k \log p_k}{\log K_{\mathrm{total}}},$$

and it modulates the distillation coefficient through

$$\lambda = \sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.$$

There, the consistency component is not prediction agreement but gradient consistency, implemented as

$$\mathbf{g}_t' = \mathbf{g}_t + \lambda_{\mathrm{GCR}} \left( \mathbf{g}_t - \bar{\mathbf{g}}_{t-1} \right),$$

with an exponential moving average $\bar{\mathbf{g}}_t$ of past gradients (Sakai et al., 5 May 2026).

In COPO, entropy is computed over the empirical distribution of final answers sampled for a single prompt,

$H(q) = -\sum_{\tau \in T_q} p(\tau)\log p(\tau),$

and it controls a soft interpolation between local and global policy optimization,

$$w_{\mathrm{local}}(H) = \sigma\bigl(\gamma(H-\rho)\bigr), \qquad w_{\mathrm{global}}(H) = 1-w_{\mathrm{local}}(H).$$

High outcome entropy emphasizes local GRPO-style optimization; low outcome entropy shifts weight to a prompt-level global signal (Han et al., 6 Aug 2025).

These formulations are mathematically different, but they instantiate the same design principle: entropy becomes operational only after being filtered through a consistency criterion.

3. CER in reliable test-time adaptation of vision-LLMs

The most direct formulation of Consistency-aware Entropy Reweighting appears in ReTA, a cache-based test-time adaptation method for CLIP-like vision-LLMs under visual distribution shift (Liang et al., 13 Jul 2025). The problem setting is a dynamic cache of test-time image features and pseudo-labels. Earlier cache-based methods prioritized entries purely by low prediction entropy, but the paper argues that under distribution shift CLIP can be overconfident on wrong predictions, so low-entropy samples are often wrongly labeled and progressively corrupt the cache.

ReTA begins with standard CLIP probabilities

$$p_{\text{CLIP}}^c = \frac{\exp\left(\mathbf{z}\cdot\boldsymbol{t}^c / \tau\right)} {\sum_{j=1}^C \exp\left(\mathbf{z}\cdot\boldsymbol{t}^j/\tau\right)},$$

where $\mathcal{S} = M/n^\ast$0 is an image feature and $\mathcal{S} = M/n^\ast$1 is a class text embedding. Instead of trusting the entropy

$\mathcal{S} = M/n^\ast$2

directly, CER constructs a semantic committee of adjacent text embeddings for each class. Multiple prompts $\mathcal{S} = M/n^\ast$3 are encoded, intra-class similarities

$\mathcal{S} = M/n^\ast$4

are computed, and embeddings are sorted from semantic outliers to centroids. The method then uses ascending progressive binning with

$\mathcal{S} = M/n^\ast$5

to form adjacent embeddings $\mathcal{S} = M/n^\ast$6 as averages of the first $\mathcal{S} = M/n^\ast$7 sorted embeddings.

To reduce the modality gap between image and text representations, ReTA applies SVD to the matrix of adjacent text embeddings, keeps the top-$\mathcal{S} = M/n^\ast$8 right singular vectors, and defines a projection

$\mathcal{S} = M/n^\ast$9

The projected feature is then classified by each committee member,

$\mathcal{R}\in\{1,\gamma\}$0

yielding a set of committee predictions $\mathcal{R}\in\{1,\gamma\}$1. The majority-voted class $\mathcal{R}\in\{1,\gamma\}$2 and its vote count $\mathcal{R}\in\{1,\gamma\}$3 define a stability factor

$\mathcal{R}\in\{1,\gamma\}$4

Agreement with the original prediction

$\mathcal{R}\in\{1,\gamma\}$5

defines a consistency factor

$\mathcal{R}\in\{1,\gamma\}$6

These are combined into

$\mathcal{R}\in\{1,\gamma\}$7

The practical effect is entirely on cache construction. Samples with low raw entropy but unstable or inconsistent committee behavior receive a larger $\mathcal{R}\in\{1,\gamma\}$8 and are more likely to be evicted or rejected. CER does not alter the retrieval formula itself; it changes which samples are allowed to define the class-wise cache prototypes. The paper characterizes this as a committee-based, semantic-consistency filter on top of entropy.

4. CER in long-tailed class-incremental learning

A second major instantiation appears in “Dynamic Distillation and Gradient Consistency for Robust Long-Tailed Incremental Learning,” which does not name CER explicitly but matches the pattern exactly (Sakai et al., 5 May 2026). The setting is long-tailed class-incremental learning (LT-CIL), where new classes arrive sequentially and the class distribution is imbalanced. The paper separates two sources of difficulty: intra-task imbalance and inter-task imbalance.

The entropy-driven part is a dynamic distillation coefficient based on the accumulated global class distribution. If $\mathcal{R}\in\{1,\gamma\}$9 is the proportion of class $$\mathcal{H}_{\mathrm{norm}} = -\sum_{k=1}^{K_{\mathrm{total}}} \frac{p_k \log p_k}{\log K_{\mathrm{total}}},$$0 among all classes observed so far, then the normalized entropy

$$\mathcal{H}_{\mathrm{norm}} = -\sum_{k=1}^{K_{\mathrm{total}}} \frac{p_k \log p_k}{\log K_{\mathrm{total}}},$$1

acts as a measure of how balanced the stream is. A time-based schedule

$$\mathcal{H}_{\mathrm{norm}} = -\sum_{k=1}^{K_{\mathrm{total}}} \frac{p_k \log p_k}{\log K_{\mathrm{total}}},$$2

is multiplied by this entropy to obtain the final distillation weight

$$\mathcal{H}_{\mathrm{norm}} = -\sum_{k=1}^{K_{\mathrm{total}}} \frac{p_k \log p_k}{\log K_{\mathrm{total}}},$$3

When the class distribution is balanced, $$\mathcal{H}_{\mathrm{norm}} = -\sum_{k=1}^{K_{\mathrm{total}}} \frac{p_k \log p_k}{\log K_{\mathrm{total}}},$$4, so distillation can become strong late in the task. Under severe imbalance, $$\mathcal{H}_{\mathrm{norm}} = -\sum_{k=1}^{K_{\mathrm{total}}} \frac{p_k \log p_k}{\log K_{\mathrm{total}}},$$5, so distillation remains limited even at late epochs, reducing over-preservation of majority-class bias.

The consistency component is Gradient Consistency Regularization (GCR). Let $$\mathcal{H}_{\mathrm{norm}} = -\sum_{k=1}^{K_{\mathrm{total}}} \frac{p_k \log p_k}{\log K_{\mathrm{total}}},$$6 be the current gradient and

$$\mathcal{H}_{\mathrm{norm}} = -\sum_{k=1}^{K_{\mathrm{total}}} \frac{p_k \log p_k}{\log K_{\mathrm{total}}},$$7

its exponential moving average, with $$\mathcal{H}_{\mathrm{norm}} = -\sum_{k=1}^{K_{\mathrm{total}}} \frac{p_k \log p_k}{\log K_{\mathrm{total}}},$$8. The update uses

$$\mathcal{H}_{\mathrm{norm}} = -\sum_{k=1}^{K_{\mathrm{total}}} \frac{p_k \log p_k}{\log K_{\mathrm{total}}},$$9

with $$\lambda = \sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.$$0. The stated effect is to suppress abrupt fluctuations, stabilize training across task boundaries, and reduce forgetting, especially for minority classes. The paper also retains gradient reweighting for intra-task imbalance through

$$\lambda = \sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.$$1

where $$\lambda = \sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.$$2 is the cumulative gradient norm for class $$\lambda = \sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.$$3.

Viewed through CER, this formulation couples entropy-aware reweighting of distillation with temporal consistency of optimization dynamics. The resulting objective remains

$$\lambda = \sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.$$4

but the optimization trajectory is further constrained in gradient space.

5. Related CER-style formulations

The same pattern appears in several adjacent areas. The details differ, but each method uses entropy as a reliability-sensitive control variable rather than a standalone score.

Paper	Consistency signal	Reweighted quantity
ENCORE (Li et al., 26 Mar 2025)	Rule reliability via low rating entropy	Multi-head reward aggregation weights
ER-CP (Luo et al., 2024)	Alignment between predictive uncertainty and set size	Input-dependent logit scaling for conformal scores
COPO (Han et al., 6 Aug 2025)	Outcome consistency across sampled responses	Mixing weights for local and global policy losses

ENCORE computes per-rule discrete entropies $$\lambda = \sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.$$5 over rule-level safety ratings and assigns

$$\lambda = \sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.$$6

with default $$\lambda = \sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.$$7. Rules with high rating entropy are treated as less reliable and receive lower weight in the aggregated scalar reward $$\lambda = \sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.$$8 (Li et al., 26 Mar 2025). The paper further argues, through a Bradley–Terry analysis, that maximally entropic rules naturally receive minimal weighting.

Entropy Reweighted Conformal Classification applies entropy directly to logits. For classifier logits $$\lambda = \sigma\!\left(\frac{t}{T}\right)\cdot \mathcal{H}_{\mathrm{norm}}.$$9 and predictive entropy

$$\mathbf{g}_t' = \mathbf{g}_t + \lambda_{\mathrm{GCR}} \left( \mathbf{g}_t - \bar{\mathbf{g}}_{t-1} \right),$$0

it defines

$$\mathbf{g}_t' = \mathbf{g}_t + \lambda_{\mathrm{GCR}} \left( \mathbf{g}_t - \bar{\mathbf{g}}_{t-1} \right),$$1

These reweighted probabilities replace the original probabilities inside APS-style conformal scores, and a validation split selects the temperature $$\mathbf{g}_t' = \mathbf{g}_t + \lambda_{\mathrm{GCR}} \left( \mathbf{g}_t - \bar{\mathbf{g}}_{t-1} \right),$$2 that minimizes average prediction-set size (Luo et al., 2024). The paper interprets this as making conformal set sizes more consistent with local uncertainty.

COPO uses entropy over the empirical distribution of final answers within a sampled response group for a prompt,

$$\mathbf{g}_t' = \mathbf{g}_t + \lambda_{\mathrm{GCR}} \left( \mathbf{g}_t - \bar{\mathbf{g}}_{t-1} \right),$$3

then blends a local GRPO-style objective with a prompt-level global objective: $$\mathbf{g}_t' = \mathbf{g}_t + \lambda_{\mathrm{GCR}} \left( \mathbf{g}_t - \bar{\mathbf{g}}_{t-1} \right),$$4 Low entropy indicates high intra-group consistency and shifts weight to the global signal; high entropy indicates answer diversity and shifts weight to local optimization (Han et al., 6 Aug 2025). This is a direct entropy-based reweighting of policy-gradient components.

6. Empirical profile, implementation patterns, and limitations

Empirically, CER-style methods are presented as reliability mechanisms under conditions where raw entropy is known to be brittle. In ReTA, Table 3 reports a baseline of $$\mathbf{g}_t' = \mathbf{g}_t + \lambda_{\mathrm{GCR}} \left( \mathbf{g}_t - \bar{\mathbf{g}}_{t-1} \right),$$5 on Natural Distribution Shifts and $$\mathbf{g}_t' = \mathbf{g}_t + \lambda_{\mathrm{GCR}} \left( \mathbf{g}_t - \bar{\mathbf{g}}_{t-1} \right),$$6 on Cross-Datasets; adding CER alone raises these to $$\mathbf{g}_t' = \mathbf{g}_t + \lambda_{\mathrm{GCR}} \left( \mathbf{g}_t - \bar{\mathbf{g}}_{t-1} \right),$$7 and $$\mathbf{g}_t' = \mathbf{g}_t + \lambda_{\mathrm{GCR}} \left( \mathbf{g}_t - \bar{\mathbf{g}}_{t-1} \right),$$8, and the full ReTA model reaches $$\mathbf{g}_t' = \mathbf{g}_t + \lambda_{\mathrm{GCR}} \left( \mathbf{g}_t - \bar{\mathbf{g}}_{t-1} \right),$$9 and $\bar{\mathbf{g}}_t$0. The same study reports that ascending progressive binning reaches $\bar{\mathbf{g}}_t$1, versus $\bar{\mathbf{g}}_t$2 for descending and $\bar{\mathbf{g}}_t$3 for uniform, and that SVD projection reaches $\bar{\mathbf{g}}_t$4, versus $\bar{\mathbf{g}}_t$5 for PCA and $\bar{\mathbf{g}}_t$6 for LDA (Liang et al., 13 Jul 2025). These numbers support the claim that the consistency mechanism is not incidental but structurally tied to performance.

In LT-CIL, experiments on CIFAR-100-LT, ImageNetSubset-LT, and Food101-LT show consistent accuracy improvements of up to $\bar{\mathbf{g}}_t$7, with particularly strong gains in the challenging In-ordered setting. The paper also reports a training-time increase of about $\bar{\mathbf{g}}_t$8 and no additional inference cost, because the method modifies only training-time weighting and gradients rather than model architecture (Sakai et al., 5 May 2026). This supports the interpretation of CER as a lightweight control mechanism rather than a heavy auxiliary module.

In multi-head reward modeling, ENCORE reports a RewardBench safety score of $\bar{\mathbf{g}}_t$9, compared with $H(q) = -\sum_{\tau \in T_q} p(\tau)\log p(\tau),$0 for uniform weights and $H(q) = -\sum_{\tau \in T_q} p(\tau)\log p(\tau),$1 for MoE weights; an entropy-based top-5 rule selection reaches $H(q) = -\sum_{\tau \in T_q} p(\tau)\log p(\tau),$2. In conformal classification, ER-CP on AG News at $H(q) = -\sum_{\tau \in T_q} p(\tau)\log p(\tau),$3 reports coverage $H(q) = -\sum_{\tau \in T_q} p(\tau)\log p(\tau),$4 and size $H(q) = -\sum_{\tau \in T_q} p(\tau)\log p(\tau),$5, versus APS coverage $H(q) = -\sum_{\tau \in T_q} p(\tau)\log p(\tau),$6 and size $H(q) = -\sum_{\tau \in T_q} p(\tau)\log p(\tau),$7. In rule-based RL for LLM reasoning, COPO with Qwen2.5-Instruct 7B improves MATH-500 mean@8 from $H(q) = -\sum_{\tau \in T_q} p(\tau)\log p(\tau),$8 to $H(q) = -\sum_{\tau \in T_q} p(\tau)\log p(\tau),$9 and maj@8 from $$w_{\mathrm{local}}(H) = \sigma\bigl(\gamma(H-\rho)\bigr), \qquad w_{\mathrm{global}}(H) = 1-w_{\mathrm{local}}(H).$$0 to $$w_{\mathrm{local}}(H) = \sigma\bigl(\gamma(H-\rho)\bigr), \qquad w_{\mathrm{global}}(H) = 1-w_{\mathrm{local}}(H).$$1 (Li et al., 26 Mar 2025, Luo et al., 2024, Han et al., 6 Aug 2025). Taken together, these results suggest that entropy reweighting is most useful when entropy alone is misaligned with the downstream notion of reliability.

The limitations are equally consistent across papers. ENCORE notes that a rule may be high-entropy but still conceptually important, or low-entropy but trivial, and that the method depends on judge quality and does not explicitly encode rule importance. ER-CP leaves the exact effect of validation-based temperature selection on finite-sample validity to future work. COPO assumes enough variation in prompt difficulty across a mini-batch to maintain a non-zero batch standard deviation for the global reward, and it does not outperform GRPO on Qwen2.5-Math-1.5B-Instruct. ReTA, for its part, depends on multiple prompts per class, projection dimensionality, and committee size; the reported best settings use cache size $$w_{\mathrm{local}}(H) = \sigma\bigl(\gamma(H-\rho)\bigr), \qquad w_{\mathrm{global}}(H) = 1-w_{\mathrm{local}}(H).$$2 per class, $$w_{\mathrm{local}}(H) = \sigma\bigl(\gamma(H-\rho)\bigr), \qquad w_{\mathrm{global}}(H) = 1-w_{\mathrm{local}}(H).$$3, and $$w_{\mathrm{local}}(H) = \sigma\bigl(\gamma(H-\rho)\bigr), \qquad w_{\mathrm{global}}(H) = 1-w_{\mathrm{local}}(H).$$4 (Li et al., 26 Mar 2025, Luo et al., 2024, Han et al., 6 Aug 2025, Liang et al., 13 Jul 2025).

A final point of clarification follows from the surveyed literature. CER is not a single objective with a universal formula. The explicit ReTA formulation, the LT-CIL formulation, and the CER-like variants in reward modeling, conformal prediction, and RL all use different entropy definitions, different consistency notions, and different reweighted quantities. What unifies them is narrower and more precise: entropy is treated as a useful but unreliable signal, and consistency is introduced to decide when, where, and how that signal should influence learning or selection.