Auxiliary Information Induced Biases

• Auxiliary information induced biases are systematic distortions arising when extra data, like references or rewards, alter decision-making processes.
• They manifest across domains such as LLM evaluation, reinforcement learning, transfer learning, and causal inference, often improving performance while risking misalignment.
• Mitigation strategies—such as calibration, learned representations, and controlled integration—help balance improved accuracy against potential bias.

Searching arXiv for the cited papers and closely related work on auxiliary-information-induced biases. Auxiliary information induced biases are systematic distortions that arise when side information intended to improve inference, evaluation, or learning also changes the decision rule itself. Across recent work, the relevant auxiliary information includes reference answers, rubrics, and background knowledge in LLM judging; human-designed shaping signals and demonstrations in reinforcement learning; auxiliary outcomes in transfer learning; cheap side objectives in optimization; mediators, covariates, and glue datasets in causal and statistical inference; and count metadata in social choice. A recurring theme is a help–harm duality: auxiliary information can improve accuracy, sample efficiency, or identifiability on clean or well-specified problems, yet also introduce bias when it is mismatched, incomplete, adversarially manipulated, or processed through an inadequate model class (Li et al., 3 Sep 2025, Li et al., 2024, Chayti et al., 2022).

1. Conceptual scope and recurring structure

The common structure is that a primary task is underdetermined, sparse, noisy, or expensive, and auxiliary information is added to stabilize or accelerate the procedure. The same addition that supplies extra signal can also impose a proxy, an inductive bias, or a dependence assumption that shifts the outcome. In LLM evaluation, the judge may over-rely on references or under-separate rubric dimensions. In reinforcement learning, human-designed auxiliary rewards may encode mistaken assumptions about progress. In transfer learning, pooled estimation over auxiliary outcomes can move the decision boundary away from the target. In causal inference, ignoring an observed or latent sufficient covariate can turn valid bounds into misleading ones. In data fusion, auxiliary “glue” can relax conditional-independence assumptions, but nonrepresentative glue can itself bias the fused distribution (Li et al., 3 Sep 2025, Li et al., 2024, Liang et al., 2020, Musio et al., 2023, Fosdick et al., 2015).

Domain	Auxiliary information	Bias mechanism
LLM judging	References, rubrics, background knowledge	Anchoring, entanglement, attention limits
Reinforcement learning	Human-designed rewards, demonstrations	Cognitive bias, design bias
Transfer learning	Auxiliary outcomes	Negative transfer under misspecification
Optimization	Side objective $h$	Gradient mismatch
Causal inference	Covariates, mediators	Confounding if omitted
Data fusion	Glue dataset	Distorted dependence if nonrepresentative
Social choice	Count metadata	Deliberate non-anonymous weighting

This suggests a broad definition: auxiliary-information-induced bias is not restricted to one discipline or one notion of unfairness. It includes evaluation failures, estimation bias, negative transfer, confounding, and intentionally biased reweighting when a procedure changes because side information changes the effective geometry of the problem.

2. Complex evaluation and LLM judges

In complex evaluation, a judge combines the task input and candidate response with reference answers, rubrics or principles, and background knowledge, and is expected to produce a judgment that is both accurate and verifiable. The central finding of "Curse of Knowledge: When Complex Evaluation Context Benefits yet Biases LLM Judges" is that this setting exposes a family of auxiliary information induced biases that are largely hidden in simpler evaluation regimes (Li et al., 3 Sep 2025).

The paper organizes six biases into reference biases and rubric biases. Format Bias is a preference for answers that superficially resemble the reference. Solution Fixation Bias is anchoring to the reference solution path and penalizing valid alternatives. Stereotype Amplification Bias is the activation or amplification of latent stereotypes through contextual cues. Criteria Loophole Bias appears when unlisted rubric dimensions are ignored. Criteria Entanglement Bias is correlated scoring across distinct rubric dimensions. Attention Limit Phenomenon is a saturation effect in issue tracking as the number of criteria or defects increases (Li et al., 3 Sep 2025).

The formal evaluation setup writes the judge score as

$S = M(Q, R, A),$

where $Q$ is the query, $R$ the response, and $A$ the auxiliary information. The benchmark ComplexEval has two tiers. ComplexEval-Basic contains $1{,}056$ samples across $12$ domains and uses LLM-generated references and rubrics together with a Comprehensive Attack. ComplexEval-Advanced uses human-written references and rubrics in three high-complexity domains—Mathematical Reasoning, Creative Writing, and Role-Playing—and applies Targeted Attack to isolate bias types (Li et al., 3 Sep 2025).

The empirical pattern is explicitly paradoxical. Auxiliary information improves accuracy on original samples; for example, the paper reports increases such as DeepSeek-V3 from $69.24$ to $73.98$ with reference information and o4-mini from $73.58$ to $S = M(Q, R, A),$0. Under attack, however, accuracy drops, and the paper reports that all evaluated models exhibit significant susceptibility to these biases, with bias magnitude scaling with task complexity. Large Reasoning Models are described as paradoxically vulnerable, with advanced-scenario ASR values such as approximately $S = M(Q, R, A),$1, $S = M(Q, R, A),$2, and $S = M(Q, R, A),$3 for solution fixation bias in ProcessBench, WritingBench, and CoSER, respectively (Li et al., 3 Sep 2025).

3. Learning-time bias in reinforcement learning, transfer learning, and optimization

In reinforcement learning, "Auxiliary Reward Generation with Transition Distance Representation Learning" frames auxiliary-information-induced bias as a practical consequence of reward engineering. Real-world rewards are usually human-designed, and therefore can reflect human cognitive biases or design biases. The paper gives two concrete examples: the mistaken assumption that Euclidean distance in raw state space tracks task progress, and reward shaping based on domain heuristics that overfit the designer’s assumptions (Li et al., 2024).

The proposed response is Transition Distance Representation Learning, which learns an embedding $S = M(Q, R, A),$4 such that

$S = M(Q, R, A),$5

Positive and negative state sets are sampled from trajectories,

$S = M(Q, R, A),$6

and a triplet-style contrastive objective encourages nearby trajectory states to be close and farther-apart states to be distant in latent space. Auxiliary rewards are then defined from latent distances, for example

$S = M(Q, R, A),$7

in the goal-conditioned setting, or by distance to latent goal-cluster centers when explicit goals are not given. In skill chaining, the same construction rewards a previous skill for moving toward the initial-state set of the next skill. The paper’s claim is not that bias is eliminated in a philosophical sense, but that dependence on brittle human priors is reduced by replacing handcrafted progress signals with learned transition-distance signals (Li et al., 2024).

In high-dimensional classification with auxiliary outcomes, "Robust and flexible learning of a high-dimensional classification rule using auxiliary outcomes" treats bias as negative transfer from approximately related outcomes. The pooled MTL estimator $S = M(Q, R, A),$8 need not match the target coefficient $S = M(Q, R, A),$9, especially under model misspecification. The paper decomposes

$Q$0

where $Q$1 is the within-subspace bias and $Q$2 is the against-subspace bias. Its two-stage method first fits a pooled MTL model and then calibrates using only the target outcome to correct both components. The stated rationale is that auxiliary outcomes can help, but only if the induced bias is explicitly removed (Liang et al., 2020).

In optimization with access to a target $Q$3 and a cheaper side objective $Q$4, the same theme appears as gradient mismatch. The naive strategy of using $Q$5-gradients in place of $Q$6-gradients is modeled by

$Q$7

and the paper shows an error floor of order $Q$8 even with $Q$9 and noiseless gradients. AuxMOM and AuxMVR estimate the correction $R$0 and rely on bounded Hessian dissimilarity,

$R$1

so that the auxiliary objective is used as a controlled bias rather than an uncontrolled one. Here the bias is neither purely harmful nor purely accidental: it becomes beneficial when similarity is high and the correction term is maintained (Chayti et al., 2022).

4. Statistical inference, causation, and data integration

In individual causal inference, auxiliary information enters through mediators and sufficient covariates. "Individual Causation with Biased Data" studies the probability of causation

$R$2

which is not point-identified from prospective probabilities alone. The paper shows that mediators can sharpen bounds, but omission of a sufficient covariate $R$3 can produce severely biased conclusions. In one binary example, the covariate-adjusted analysis yields

$R$4

whereas ignoring $R$5 produces the false bound

$R$6

In the combined mediator-plus-covariate case, ignoring $R$7 while using $R$8 gives bounds inconsistent with the correct interval. The paper’s practical conclusion is that auxiliary information can either improve causal attribution or, if omitted, make it seriously wrong (Musio et al., 2023).

In categorical data fusion, the unidentified component is the dependence between variables observed in separate databases. "Categorical Data Fusion Using Auxiliary Information" refers to external joint information as glue. Standard fusion assumes

$R$9

but the paper treats this as potentially unrealistic. Glue weakens the need for that conditional-independence assumption by providing joint observations on at least one $A$0-variable and one $A$1-variable. The model is a Dirichlet process mixture of products of multinomials, fit after concatenating the main data and the glue data. Yet the paper is explicit that nonrepresentative glue can bias the estimated joint distribution and downstream regression analyses; this is why it proposes conditional glue construction when only conditional representativeness is plausible (Fosdick et al., 2015).

In finite-population sampling, auxiliary variables and attributes alter bias beyond first-order approximations. "Study of some improved ratio type estimators using information on auxiliary attributes under second order approximation" shows that estimators that appear nearly equivalent at first order separate at second order because higher-order mixed moments generate additional bias and MSE terms. In the reported numerical illustration, first-order MSEs are identical under optimal tuning, while second-order MSEs rank the estimators as

$A$2

Similarly, "Efficient class of estimators for population median using auxiliary information" derives asymptotic bias and MSE for median estimators using an auxiliary variable and shows that correlation structure, auxiliary variability, and tuning constants all enter the bias directly (Sharma et al., 2013, Sharma et al., 2014).

5. Deliberate bias, measurement protocols, and empirical signatures

Not all auxiliary-information-induced bias is treated as a pathology. In "Objective Social Choice: Using Auxiliary Information to Improve Voting Outcomes," auxiliary count information $A$3 is used to intentionally weight voters by reliability under a multi-arm bandit noise model. For cardinal means, the MLE weights are proportional to counts,

$A$4

and for two-arm ordinal or cardinal comparisons the weights depend on inverse variance or its square root. The paper states that this creates a bias toward more informative voters, but interprets it as a correction to the bias of anonymous aggregation rules that treat heterogeneous voters equally (Pitis et al., 2020).

Across the literature, empirical evidence is typically organized around task-specific diagnostics rather than a universal scalar measure of bias. ComplexEval uses accuracy on clean and attacked samples, pairwise preference accuracy $A$5, robustness rate $A$6, and Attack Success Rate $A$7 (Li et al., 3 Sep 2025). The TDRP paper reports better sample efficiency, higher converged returns or success rates, more stable learning, and a $A$8 increase in Pick-Place success rate in skill chaining, but explicitly does not provide a formal metric of human-design bias reduction (Li et al., 2024). The high-dimensional classification paper compares the proposed estimator with target-only lasso, transfer baselines, and MTL baselines in simulations and a total hip arthroplasty application with $A$9 patients and $1{,}056$0 covariates, reporting the highest accuracy for the proposed method (Liang et al., 2020). The data fusion paper uses Hellinger distance, contingency-table recovery, regression inference, and Fréchet-bound width, reporting widths around $1{,}056$1 to $1{,}056$2 for the six cells of $1{,}056$3 in its simulation (Fosdick et al., 2015).

A plausible implication is that auxiliary-information-induced bias is often easier to detect through stress tests, attacks, subgroup conditioning, higher-order asymptotics, or counterfactual inconsistency than through ordinary average-case performance. This matches the repeated observation that auxiliary information improves clean or nominal performance while masking failure modes that only appear under mismatch, attack, or omitted-variable structure.

6. Misconceptions, limitations, and open directions

A common misconception is that more auxiliary information is unconditionally beneficial. The LLM-judge results directly contradict this: richer references and rubrics can improve original-sample accuracy while making judges more attackable (Li et al., 3 Sep 2025). A second misconception is that auxiliary-induced bias is always harmful. The optimization and social-choice papers explicitly use controlled or intentional bias—through correction terms or reliability weighting—to improve the target procedure (Chayti et al., 2022, Pitis et al., 2020). A third misconception is that performance gains automatically demonstrate bias reduction. The TDRP paper provides indirect evidence through learning efficiency, stability, and trajectory visualization, but does not formally measure “bias reduction” as an explicit quantity (Li et al., 2024).

The limitations named across the papers are domain-specific but structurally similar. The ComplexEval study focuses mainly on references and rubrics, not multimodal context or tool-use settings, and leaves mitigation open (Li et al., 3 Sep 2025). The RL paper reduces one form of human prior dependence, not all forms of bias (Li et al., 2024). The transfer-learning and optimization papers require assumptions about sparsity of the correction or similarity of curvature, respectively (Liang et al., 2020, Chayti et al., 2022). The causal paper shows that mediators cannot rescue confounding induced by omission of a sufficient covariate (Musio et al., 2023). The data-fusion paper emphasizes that naive use of convenience-sample glue is dangerous (Fosdick et al., 2015).

The emerging mitigation directions are correspondingly heterogeneous: controlled information integration rather than indiscriminate context expansion in evaluation; learned task-relevant latent structure rather than hand-specified progress proxies in RL; target-only calibration after pooled learning in transfer settings; explicit correction for $1{,}056$4 in optimization; covariate adjustment in causal attribution; conditional glue construction in data fusion; and count-aware weighting rules in objective social choice (Li et al., 3 Sep 2025, Li et al., 2024, Liang et al., 2020, Chayti et al., 2022, Musio et al., 2023, Fosdick et al., 2015, Pitis et al., 2020).

Taken together, these works support a precise but non-universal conclusion: auxiliary information is best understood as a double-edged structural intervention. It can densify reward, reduce variance, sharpen bounds, relax identification assumptions, and improve aggregation; but it can also anchor, entangle, confound, overfit, and distort. The central research problem is therefore not whether to use auxiliary information, but how to characterize the regime in which the induced bias is controlled, measurable, and aligned with the target objective.