Human-in-the-Loop Calibration (HITLC)

• Human-in-the-Loop Calibration (HITLC) is a hybrid approach that integrates human feedback into automated workflows to address uncertainty and subjective parameters.
• It employs entropy-based escalation and continual optimization methods to selectively involve human judgment, thereby enhancing convergence and efficiency.
• HITLC has proven effective in reducing costs and calibration time in applications like LLM evaluation and VR/AR device personalization.

Human-in-the-Loop Calibration (HITLC) encompasses a diverse set of methodologies where algorithmic search and optimization processes actively incorporate human feedback or decision-making to achieve accurate, user-aligned calibration of parameters, evaluations, or models. HITLC is motivated by the presence of subjective, context-dependent, or qualitative dimensions that cannot be robustly quantified or optimized in an entirely automated fashion, necessitating explicit integration of human judgments within the loop. Recent advances in HITLC include entropy-based selective escalation of ambiguous cases to humans, population-aware continual optimization across users, and sensor-guided grey-box preference learning, thereby significantly improving calibration fidelity, convergence speed, and cost-efficiency.

1. Formal Definition and Principle of HITLC

Human-in-the-Loop Calibration is characterized by algorithmic workflows that invoke human judgment whenever a system’s automated evaluation exhibits uncertainty, ambiguity, or misalignment with user preferences. The mechanism leverages human annotation, subjective feedback, or preference comparisons either at initialization or at targeted decision nodes within an iterative optimization scheme. A canonical formalization, as in positional-bias mitigation for LLM evaluation, involves monitoring entropy across model outputs to stratify instances into those resolved fully automatically and those routed to human intervention, thus constructing a hybrid decision system where automation and manual oversight are dynamically blended (Wang et al., 2023).

2. Entropy-based Selective Escalation in Model Evaluation

One central paradigm is the entropy-based HITLC for pairwise comparison tasks, especially in LLM evaluation. Here, the Balanced Position Diversity Entropy (BPDE) quantifies the disagreement across multiple model scoring rounds and response orderings: for each test tuple $(q, r_1, r_2)$, a series of $2k$ outcomes $$\{\mathrm{ER}_i\}_{i=1}^k \cup \{\mathrm{ER}'_i\}_{i=1}^k$$—each in $\{\mathrm{win}, \mathrm{tie}, \mathrm{lose}\}$—yield empirical probabilities $p_{\mathrm{er}}$, and entropy

$$\mathrm{BPDE} = -\sum_{\mathrm{er} \in \{\text{win},\text{tie},\text{lose}\}} p_{\mathrm{er}} \log p_{\mathrm{er}}$$

is computed per instance. High-BPDE examples (top $\beta\%$) are escalated for manual annotation, typically by majority vote among multiple human annotators. This yields maximally efficient human resource allocation and low-cost, high-fidelity hybrid evaluation (Wang et al., 2023).

Configuration	Accuracy	Kappa	Cost (USD)
Vanilla LLM evaluator	52.7%	0.24	$2.00
MEC+BPC	62.5%	0.37	$6.38
MEC+BPC+HITLC (20% human)	73.8%	0.56	$23.10
Human annotator	71.7%	0.54	$30.00

Results demonstrate that entropy-based HITLC surpasses both fully automated and completely manual schemes in accuracy, while lowering overall annotation costs.

3. Continual HITLC and Population-level Surrogate Models

In settings where calibration parameters vary across users—such as VR/AR input devices—continual HITLC exploits accumulated calibration data to accelerate optimization for new users. The continual human-in-the-loop optimization framework (Liao et al., 7 Mar 2025) combines a Bayesian Neural Network (BNN) population surrogate with user-specific Gaussian Processes (GPs), leveraging generative replay to mitigate catastrophic forgetting and facilitate knowledge transfer:

• For each new user, Bayesian optimization initially exploits the BNN prior, gradually shifting to the adaptation-specific GP as user data accrues.
• Cumulative regret $R(N,T)$ across users and steps quantifies effectiveness:

$$R(N,T) = \sum_{u=1}^N \sum_{t=1}^T \bigl[f_u(\bm{x}_u^*) - f_u(\bm{x}_u^t)\bigr]$$

Empirical analysis in text-entry VR keyboard calibration shows significant reductions in adaptation time and regret for later users, scalable learning across user populations, and robust knowledge retention via replay (Liao et al., 7 Mar 2025).

4. Preference-based and Sensor-Guided Calibration Approaches

Preference-based HITLC frameworks employ pairwise comparison queries to infer the latent objective $f(x)$ over a candidate design space, as described in GLISp and Preferential Bayesian Optimization. The Regularized GLISp extension (Cercola et al., 6 Nov 2025) introduces grey-box calibration by embedding measurable sensor descriptors $J_1(x),\dots,J_p(x)$ into the optimization loop:

• The hypothesis function $h(x;\theta) = \sum_{r=1}^p w_r J_r(x)$ injects physics-informed structure.
• The surrogate $\hat{y}(x)$ (RBF-based) and weights $\theta$ are jointly optimized with a regularization penalty $$\lambda_{LS} \sum_{i=1}^N(\hat{y}(x_i)-h(x_i;\theta))^2$$, balancing user-driven preferences against quantitative sensor alignment.
• Validation across analytical and vehicle tuning tasks shows dramatic reductions in convergence error and variance, especially in higher-dimensional and data-sparse settings (Cercola et al., 6 Nov 2025).

5. Workflow Illustrations and Algorithmic Scheme

A typical HITLC pipeline may be described as follows:

1. Automated model calibration via ensemble or Bayesian sampling (e.g., MEC + BPC, ConBO).
2. Diversity or uncertainty assessment (BPDE, predictive variance) to separate ‘easy’ and ‘hard’ instances.
3. Selective routing of high-uncertainty cases to human annotators for majority-vote ground-truth assignment.
4. Surrogate retraining or decision overwriting in hybrid loop systems.
5. Optional integration of sensor information or population priors for accelerated convergence and preference-based alignment.

Algorithmic pseudocode appears in (Wang et al., 2023, Liao et al., 7 Mar 2025), and (Cercola et al., 6 Nov 2025), specifying stepwise procedures for entropy computation, replay buffering, and sensor-guided acquisition.

6. Key Parameters, Implementation, and Limitations

HITLC frameworks employ sample sizes ($k$), temperature settings, threshold fractions ($\beta$), and regularization strengths ($\lambda_{LS},\lambda_\beta$) tuned through cross-validation and/or population statistics. Human annotation cost, deliberation latency, and inter-user variability are explicit considerations in design. Limitations include:

• Under-representation of outlier users in population models.
• Sensitivity to the order of data accrual (“sequential effects”).
• Assumed time-invariance in repeated-user calibration cycles.
• Restricted expressiveness in linear hypothesis functions for sensor alignment. Extensions such as importance sampling, online re-adaptation, multi-objective preference-based optimization, context-dependent regularization, and edge deployment have been proposed to enhance scalability, robustness, and personalization (Liao et al., 7 Mar 2025, Cercola et al., 6 Nov 2025).

7. Practical Impact and Future Directions

HITLC achieves cost-effective, user-aligned calibration in automated evaluation, device personalization, and preference-driven engineering tasks. Notable empirical findings include:

• Surpassing human-alone and LLM-alone benchmarks in LLM evaluation with 20% human review (Wang et al., 2023).
• Reducing VR calibration time and regret via continual optimization across user cohorts (Liao et al., 7 Mar 2025).
• Faster, more stable convergence in vehicle suspension and analytical benchmark tasks with sensor-guided preference learning (Cercola et al., 6 Nov 2025). A plausible implication is that HITLC methodologies will underpin next-generation adaptive systems, embedding domain expertise, sensor data, and human feedback for superior performance at scale.

Limitations in algorithmic expressiveness, representation of atypical human preferences, and computational latency remain active areas of research, with ongoing work on non-linear surrogates, context-sensitive regularization, preferential multi-objective optimization, and cross-domain population transfer.