Inference-Time Logit-Level Debiasing

• The paper introduces inference-time logit-level debiasing methods that adjust pre-softmax scores to mitigate bias, toxicity, and class imbalance with minimal changes.
• These methods leverage techniques like multiplicative scaling, additive offsets, and gradient-based adjustments, tailored for binary classifiers, language models, and multimodal retrieval.
• The approach offers flexible, deployment-ready solutions that preserve model fluency and calibration while addressing fairness and performance trade-offs.

Inference-time logit-level debiasing is a family of post-hoc intervention methods that modify a model’s output distribution at deployment by operating on logits, log-probabilities, or decode-time quantities with an explicit logit-level interpretation. Across binary and multiclass classifiers, autoregressive LLMs, and multimodal retrieval systems, the intervention is applied after the base model has produced its native scores, with the goal of reducing undesirable dependence on sensitive attributes, toxicity, stereotype cues, class imbalance, or positional artifacts while preserving utility, fluency, calibration, or prediction stability. Recent formulations instantiate this idea through multiplicative logit scaling in classifier post-processing, additive token-level steering in language generation, per-token adaptive optimization of small parameter subsets, class-dependent additive offsets, attention-guided log-domain calibration, and cross-vocabulary mixing of next-token distributions (Gennaro et al., 2024, An et al., 16 Jan 2026, Yang et al., 2022, Li et al., 2022, Xian et al., 12 May 2026, Chawla et al., 10 Jun 2026).

1. Formal object of intervention

The common object manipulated by these methods is the model score prior to the final normalization. In binary classification, a black-box classifier $f$ outputs a probability $p(x)=f(x)\in[0,1]$ for $Y=1$, with logit

$$\ell(x)=\operatorname{logit}(p(x))=\log\frac{p(x)}{1-p(x)}.$$

In multiclass classification, logits are $z(x)\in\mathbb{R}^K$ and softmax probabilities are

$$\pi_k(x)=\frac{e^{z_k(x)}}{\sum_{j=1}^K e^{z_j(x)}}.$$

In language generation, the relevant quantity is the next-token logit vector $l_t\in\mathbb{R}^{|V|}$ or $L_t=\mathrm{LM}_\theta([c;x_{<t}])$, while in discrete-choice retrieval the model can be restricted to a candidate set and scored through candidate-specific log-probabilities computed by restricted softmax over valid continuation tokens (Gennaro et al., 2024, An et al., 16 Jan 2026, Yang et al., 2022, Xian et al., 12 May 2026).

A crucial technical point is that “logit-level” does not denote a single algebraic form. Some methods directly rewrite logits, as in $z'_c=z_c+\Delta_c$ or $l'_t=l_t+\alpha s_t$. Others modify a scalar logit multiplicatively, as in $p(x)=f(x)\in[0,1]$0. Others still act in log-probability space via subtraction of an estimated prior, or induce a logit shift indirectly through a decode-time gradient step on a small parameter subset so that

$p(x)=f(x)\in[0,1]$1

This suggests that the defining property is not a specific operator such as addition, but intervention on the terminal scoring layer or an equivalent quantity immediately upstream of sampling or thresholding.

2. Main intervention families

The contemporary design space can be organized by the transformation applied at inference and by the access assumptions imposed on deployment.

Method	Core intervention	Test-time requirements
RBMD	$p(x)=f(x)\in[0,1]$2	$p(x)=f(x)\in[0,1]$3 and $p(x)=f(x)\in[0,1]$4; no $p(x)=f(x)\in[0,1]$5
LPL	$p(x)=f(x)\in[0,1]$6	logits and class offsets
SWAI	$p(x)=f(x)\in[0,1]$7 or top-$p(x)=f(x)\in[0,1]$8 biasing	static score table
UDDIA	$p(x)=f(x)\in[0,1]$9 via one gradient step/token	white-box gradients and attribute signals
LAD	$Y=1$0	attention access and calibration set
ALIGNBEAM	probability-space buffer with translated anchor mass	target model, anchor model, optional judge

In RBMD, the post-processing transformation is explicitly multiplicative in logit space:

$Y=1$1

The ratio network $Y=1$2 depends on $Y=1$3 but not on the sensitive attribute $Y=1$4 at inference time. In an ablation, $Y=1$5 is parameterized linearly as $Y=1$6, but the debiasing remains multiplicative in the logit (Gennaro et al., 2024).

SWAI instead uses additive token-level steering. In its general form,

$Y=1$7

or for multiple attributes,

$Y=1$8

with

$Y=1$9

Its implemented targeted-bias variant restricts intervention to the top-$$\ell(x)=\operatorname{logit}(p(x))=\log\frac{p(x)}{1-p(x)}.$$0 candidates under the base logits, ranks them by the attribute score table, defines a favored set $$\ell(x)=\operatorname{logit}(p(x))=\log\frac{p(x)}{1-p(x)}.$$1, and adds a constant bias $$\ell(x)=\operatorname{logit}(p(x))=\log\frac{p(x)}{1-p(x)}.$$2 only to those tokens (An et al., 16 Jan 2026).

UDDIA occupies a different point in the design space. It does not inject a fixed additive or multiplicative offset directly. Instead, it updates a small parameter subset $$\ell(x)=\operatorname{logit}(p(x))=\log\frac{p(x)}{1-p(x)}.$$3—bias terms in transformer layers—during decoding, and the modified parameters induce a logit shift

$$\ell(x)=\operatorname{logit}(p(x))=\log\frac{p(x)}{1-p(x)}.$$4

The update is driven by mutual-information minimization, context-aware regularization, and, for detoxification, perplexity-constrained replay (Yang et al., 2022).

LPL uses class-dependent additive perturbations,

$$\ell(x)=\operatorname{logit}(p(x))=\log\frac{p(x)}{1-p(x)}.$$5

and the accompanying inference-time recipe instantiates $$\ell(x)=\operatorname{logit}(p(x))=\log\frac{p(x)}{1-p(x)}.$$6 as frequency- or tail/head-based class offsets. For single-label tasks, one described choice is

$$\ell(x)=\operatorname{logit}(p(x))=\log\frac{p(x)}{1-p(x)}.$$7

where $$\ell(x)=\operatorname{logit}(p(x))=\log\frac{p(x)}{1-p(x)}.$$8 for tail classes and $$\ell(x)=\operatorname{logit}(p(x))=\log\frac{p(x)}{1-p(x)}.$$9 for head classes (Li et al., 2022).

LAD and ALIGNBEAM both operate on discrete candidate distributions rather than unrestricted vocabulary logits, but each retains a logit-level formulation. LAD subtracts an instance-specific expected positional prior in log space,

$z(x)\in\mathbb{R}^K$0

where $z(x)\in\mathbb{R}^K$1 is mixed from conditional bias profiles using a purified attention posterior $z(x)\in\mathbb{R}^K$2 (Xian et al., 12 May 2026). ALIGNBEAM mixes a target model’s next-token distribution with an anchor model’s distribution after token-by-token translation into the target vocabulary. Its implementation uses a probability buffer

$z(x)\in\mathbb{R}^K$3

followed by renormalization, while an optional logit-space view defines

$z(x)\in\mathbb{R}^K$4

This permits cross-vocabulary logit mixing without shared token IDs (Chawla et al., 10 Jun 2026).

3. Fairness-preserving post-processing for predictive models

In predictive fairness, the most explicit inference-time logit-level debiasing formulation in the provided corpus is RBMD, which targets demographic parity through a post-processing layer applied to an arbitrary black-box classifier. The method is model-agnostic, does not require the sensitive attribute at test time, and makes “minimal changes” an explicit objective rather than a side effect. The ideal constrained problem is stated as

$z(x)\in\mathbb{R}^K$5

and the implemented min–max surrogate uses classification loss, an adversarial loss on $z(x)\in\mathbb{R}^K$6, and the penalty

$z(x)\in\mathbb{R}^K$7

which discourages unnecessary deviations from the original logit (Gennaro et al., 2024).

The fairness notion used in the experiments is demographic parity, measured empirically by the P-rule,

$z(x)\in\mathbb{R}^K$8

Equalized Odds is discussed as an extension by modifying the adversary to take $z(x)\in\mathbb{R}^K$9 as input, following Zhang et al. (2018), but this extension is not implemented in the paper’s experiments. Training uses $$\pi_k(x)=\frac{e^{z_k(x)}}{\sum_{j=1}^K e^{z_j(x)}}.$$0 offline to teach an adversary to reconstruct the sensitive attribute from the scaled logit, and fairness improves when the adversary performs poorly. Deployment then requires only $$\pi_k(x)=\frac{e^{z_k(x)}}{\sum_{j=1}^K e^{z_j(x)}}.$$1 and the original score $$\pi_k(x)=\frac{e^{z_k(x)}}{\sum_{j=1}^K e^{z_j(x)}}.$$2:

1. compute $$\pi_k(x)=\frac{e^{z_k(x)}}{\sum_{j=1}^K e^{z_j(x)}}.$$3,
2. compute $$\pi_k(x)=\frac{e^{z_k(x)}}{\sum_{j=1}^K e^{z_j(x)}}.$$4 from the ratio network,
3. output $$\pi_k(x)=\frac{e^{z_k(x)}}{\sum_{j=1}^K e^{z_j(x)}}.$$5.

The explicit minimal-change bias is also supported analytically. Since $$\pi_k(x)=\frac{e^{z_k(x)}}{\sum_{j=1}^K e^{z_j(x)}}.$$6,

$$\pi_k(x)=\frac{e^{z_k(x)}}{\sum_{j=1}^K e^{z_j(x)}}.$$7

and if $$\pi_k(x)=\frac{e^{z_k(x)}}{\sum_{j=1}^K e^{z_j(x)}}.$$8 and $$\pi_k(x)=\frac{e^{z_k(x)}}{\sum_{j=1}^K e^{z_j(x)}}.$$9, then $l_t\in\mathbb{R}^{|V|}$0. The paper’s reported findings state that RBMD outperforms LPP on Law School and matches it on COMPAS, nearly matches AdvDebias despite explicitly penalizing changes, and consistently achieves lower mean percentage of changes $l_t\in\mathbb{R}^{|V|}$1 than competitors across discretized fairness–accuracy bins on both datasets. Oracle and ROC, which use $l_t\in\mathbb{R}^{|V|}$2 at inference, can outperform RBMD. The example calibration comparison reports AdvDebias at fairness $l_t\in\mathbb{R}^{|V|}$3, accuracy $l_t\in\mathbb{R}^{|V|}$4, versus RBMD at fairness $l_t\in\mathbb{R}^{|V|}$5, accuracy $l_t\in\mathbb{R}^{|V|}$6, with RBMD scores clustering near the diagonal (Gennaro et al., 2024).

A distinct predictive application is long-tail correction through class-level offsets. The LPL synthesis describes an inference-time plug-in in which per-class offsets shift decision boundaries toward tail classes. In single-label softmax classification,

$l_t\in\mathbb{R}^{|V|}$7

and in multi-label settings one applies $l_t\in\mathbb{R}^{|V|}$8. The reported empirical evidence includes CIFAR100-LT results of 55.75% Top-1 error at 100:1 and 39.03% at 10:1, compared with LA at 56.11% and 41.66%, as well as improvements on iNat2017, iNat2018, VOC-MLT, COCO-MLT, and MS-COCO. Here “debiasing” is not demographic fairness but correction of class imbalance and head-class dominance through class-specific logit adjustment (Li et al., 2022).

4. Language-model steering, detoxification, and alignment transfer

In language generation, inference-time logit-level debiasing spans at least three distinct regimes: training-free statistical steering, adaptive optimization during decoding, and mixing with an external aligned model.

SWAI is training-free and architecture-agnostic because it acts on the universal logit layer rather than internal activations. Its token score table is built from z-normalized log-odds of labeled corpora. For a two-class setting, token counts are converted into smoothed probabilities, log-odds, log-odds ratios, and a Monroe-style standard error, yielding

$l_t\in\mathbb{R}^{|V|}$9

Tokens overrepresented in the desired class get positive scores and those overrepresented in the contrast class get negative scores. The paper’s implementation uses a pooled Dirichlet prior with $L_t=\mathrm{LM}_\theta([c;x_{<t}])$0, top-$L_t=\mathrm{LM}_\theta([c;x_{<t}])$1, favored fraction $L_t=\mathrm{LM}_\theta([c;x_{<t}])$2, and bias $L_t=\mathrm{LM}_\theta([c;x_{<t}])$3. Reported results include OSE complexity control on Llama 3.1 8B from 37.11% to 84.53% accuracy and macro-F1 from 0.363 to 0.845, WikiPol politeness from 56.70% to 77.20% accuracy and macro-F1 from 0.569 to 0.730, and RealTox toxicity control from F1 0.010 to 0.552 with precision 97.56% and recall 38.46%, described as a 50×+ F1 improvement (An et al., 16 Jan 2026).

UDDIA addresses a different problem: the joint detoxification and debiasing of language generation through inference-time adaptive optimization. Its theoretical starting point is minimization of mutual information between the next token $L_t=\mathrm{LM}_\theta([c;x_{<t}])$4 and attributes $L_t=\mathrm{LM}_\theta([c;x_{<t}])$5, yielding a token-level loss

$L_t=\mathrm{LM}_\theta([c;x_{<t}])$6

The context-aware extension minimizes $L_t=\mathrm{LM}_\theta([c;x_{<t}])$7 through an additional regularizer

$L_t=\mathrm{LM}_\theta([c;x_{<t}])$8

Debiasing tunes bias terms in all transformer layers during decoding; detoxification tunes only upper-layer bias terms and uses a redo mechanism if sequence perplexity exceeds a threshold. Reported results include, for gender debiasing, best overall global bias $L_t=\mathrm{LM}_\theta([c;x_{<t}])$9, best local bias $z'_c=z_c+\Delta_c$0, and best overall quality $z'_c=z_c+\Delta_c$1; for detoxification, UDDIA-t with $z'_c=z_c+\Delta_c$2 reaches Avg.Max.Tox $z'_c=z_c+\Delta_c$3, ToxProb $z'_c=z_c+\Delta_c$4, and PPL $z'_c=z_c+\Delta_c$5; and the unified variant UDDIA-u achieves Avg.Max.Tox $z'_c=z_c+\Delta_c$6, ToxProb $z'_c=z_c+\Delta_c$7, PPL $z'_c=z_c+\Delta_c$8, Regard diff $z'_c=z_c+\Delta_c$9, Toxicity diff across groups $l'_t=l_t+\alpha s_t$0, and best overall $l'_t=l_t+\alpha s_t$1 (Yang et al., 2022).

ALIGNBEAM transfers safety alignment from an anchor model to a target model during decoding, even when the two models have different vocabularies. At each step, the anchor’s top-$l'_t=l_t+\alpha s_t$2 tokens are decoded to strings and re-encoded under the target tokenizer; mapped single-token hits accumulate in a probability buffer, which is then renormalized and used to extend beams. With defaults $l'_t=l_t+\alpha s_t$3, $l'_t=l_t+\alpha s_t$4, and shallow mixed depth $l'_t=l_t+\alpha s_t$5 for most models, the paper reports macro refusal gains from 38.1% to 91.5% on AdvBench and from 25.9% to 76.4% on HarmBench-Standard across five evaluation pairs, with task-accuracy changes of $l'_t=l_t+\alpha s_t$6 percentage points on GSM8K and MedQA. A central empirical claim is that three mixed steps capture more than 92% of the safety gain, indicating that much of the transferred alignment is concentrated in the initial decode positions (Chawla et al., 10 Jun 2026).

Taken together, these methods show that generation-time debiasing can be static and table-driven, dynamic and optimization-based, or relational and anchor-guided. They also show that “training-free” and “inference-time” are not equivalent categories: SWAI and ALIGNBEAM are training-free, whereas UDDIA performs one optimization step per token without training data, and RBMD requires offline post-processing training before deployment.

5. Attention-guided calibration and multimodal retrieval

In multimodal retrieval, the debiasing target may be neither demographic disparity nor toxicity but position bias. LAD identifies a phenomenon termed Logit-Attention Divergence, in which output logits are dominated by positional priors even when internal attention remains aligned with the relevant visual evidence. For a query $l'_t=l_t+\alpha s_t$7 and candidate images $l'_t=l_t+\alpha s_t$8, the method aggregates attention from the last query token $l'_t=l_t+\alpha s_t$9 to the visual token span $p(x)=f(x)\in[0,1]$00 of image $p(x)=f(x)\in[0,1]$01:

$p(x)=f(x)\in[0,1]$02

Candidate probabilities are obtained from restricted softmax over valid candidate-index continuations, and the observed distribution is factorized into structural bias and visual gain (Xian et al., 12 May 2026).

The proposed correction is training-free but not purely static. A symmetrized calibration set of size $p(x)=f(x)\in[0,1]$03 is used to estimate conditional structural bias $p(x)=f(x)\in[0,1]$04 and a per-layer static attention prior $p(x)=f(x)\in[0,1]$05. At inference, the model selects the top-$p(x)=f(x)\in[0,1]$06 layers by total attention mass, with default $p(x)=f(x)\in[0,1]$07, computes a purified visual posterior

$p(x)=f(x)\in[0,1]$08

sharpens it by $p(x)=f(x)\in[0,1]$09, mixes the conditional bias profiles,

$p(x)=f(x)\in[0,1]$10

and then subtracts the resulting instance-specific expected prior in log space.

The reported improvements are large. In the random $p(x)=f(x)\in[0,1]$11 setting, LLaVA-OneVision-8B moves from 67.52$p(x)=f(x)\in[0,1]$121.06 accuracy under Vanilla inference to 98.66$p(x)=f(x)\in[0,1]$130.36 under the proposed method, while recall standard deviation falls from 25.43 to 0.88 and consistency rises from 19.90 to 96.50. In the random $p(x)=f(x)\in[0,1]$14 setting, Vanilla accuracy is 31.56$p(x)=f(x)\in[0,1]$152.01 and the calibrated method reaches 94.92$p(x)=f(x)\in[0,1]$160.40. The appendix-level “attn_validation” result further reports Purified Attention outperforming Vanilla in every listed setting, including 96.14% versus 67.52% for Random $p(x)=f(x)\in[0,1]$17 and 91.16% versus 31.56% for Random $p(x)=f(x)\in[0,1]$18. These results support the paper’s claim that attention is substantially more position-invariant than the final logits, and that purely static logit priors such as PriDe fail because the bias is conditional rather than content-independent (Xian et al., 12 May 2026).

6. Limitations, misconceptions, and open directions

Several misconceptions recur across this literature. One is that inference-time logit-level debiasing is always a zero-training technique. RBMD requires offline adversarial training of the ratio network and adversary before deployment, even though the sensitive attribute is not needed at test time (Gennaro et al., 2024). Another is that logit-level intervention must be a direct additive offset. The surveyed methods include multiplicative scaling, dynamic subtractive calibration in log space, and gradient-induced shifts through small parameter updates (Yang et al., 2022, Xian et al., 12 May 2026). A third misconception is that token-level steering necessarily addresses substantive bias. The SWAI synthesis explicitly notes that token-level debiasing primarily reduces surface markers, while substantive bias rooted in world knowledge may require context-aware discriminators, activation-based edits, or training-based alignment (An et al., 16 Jan 2026).

The practical limitations are heterogeneous. RBMD can fail under distribution shift because fairness without $p(x)=f(x)\in[0,1]$19 at test time relies on adversarial training with offline sensitive labels; too small $p(x)=f(x)\in[0,1]$20 can cause excessive changes and accuracy loss, too large can fail to achieve fairness, and experiments focus on demographic parity rather than implemented Equalized Odds (Gennaro et al., 2024). LPL-style offsets may overcompensate, inflate false positives for tails, or lose effectiveness when the test class prior changes (Li et al., 2022). SWAI can induce semantic drift, reduced diversity, style–content trade-offs, and vulnerability to adversarial prompts, because static token tables are unaware of context (An et al., 16 Jan 2026). UDDIA incurs higher runtime than some baselines because it performs backpropagation per token, assumes simplified demographic groupings in the reported debiasing experiments, and leaves downstream task-specific performance as a further study (Yang et al., 2022). LAD requires white-box access to attention maps and is strictly tailored to discrete-choice retrieval rather than open-ended generation (Xian et al., 12 May 2026). ALIGNBEAM can suffer from mapping errors under one-to-many tokenization mismatches, judge miscalibration, and the “sorry-but-continues” failure mode when the mixed prefix is too short (Chawla et al., 10 Jun 2026).

The open directions listed in these works point to a broader unification. Proposed extensions include multiclass logit scaling with minimal-change regularization, regression or risk scoring analogues, pairwise fairness constraints for ranking, language-model token-logit scaling, context-conditioned score tables, adaptive time-varying steering strength, integration with safety filters and KL-based controllers, and multi-passage or text-retrieval versions of attention-guided calibration (Gennaro et al., 2024, An et al., 16 Jan 2026, Xian et al., 12 May 2026). A plausible implication is that future research will increasingly combine three ingredients already present in separate strands of the literature: explicit deviation control from the base model, instance-specific priors rather than global corrections, and modular deployment that avoids retraining the underlying backbone unless strictly necessary.