Human-Augmented Models (HAM)

• Human-Augmented Model (HAM) is a framework that integrates human inputs—via feedback, control signals, or priors—directly into computational models.
• HAMs enhance task performance and robustness in fields like navigation, recommendation, and epidemiological forecasting by fusing human knowledge with algorithmic processing.
• These models use adaptive fusion and hybrid architectures to dynamically merge human and machine data, improving interpretability and resilience to uncertainty.

A Human-Augmented Model (HAM) is a class of computational or machine learning system wherein human perception, cognition, or data is explicitly incorporated into model structure, learning, or inference to improve task performance, robustness, or interpretability. Across the literature, the term encompasses a spectrum of architectures and frameworks, including human-in-the-loop sequential decision makers, hybrid symbolic–neural models guided by human-derived structures, and control or sensory systems that fuse human and machine input. The defining feature is the formal and algorithmic mechanism by which the human component—via priors, feedback, control signals, or knowledge representations—augments and dynamically integrates with the core algorithmic model.

1. Fundamental Principles and Definitions

The essential principle of a Human-Augmented Model is the explicit integration of human inputs, goals, or modeling constructs with algorithmic agents or learning systems. HAMs are most commonly defined by several axes:

• Human agency as a control or input: Human actions, behaviors, or intentions are modeled as variables or exogenous signals explicitly shaping the system's evolution or output. For example, in epidemiological modeling, aggregated human mobility data is treated as a dynamic control signal influencing disease dynamics (Mustavee et al., 2021).
• Direct data or feedback integration: Raw or processed human data (drawings, sensor readings, voice prompts, haptic signals) is used to steer, calibrate, or contextualize machine models, as in hybrid navigation (Tan et al., 31 Jan 2025) or collaborative tracking (Li et al., 2020).
• Model-structural augmentation: Human-derived structures, such as hand-drawn maps, interpretability-centric symbolic constraints, or expert priors, are built directly into the architecture, shaping state spaces, search, or inference pathways.
• Sequential and adaptive fusion: Dynamic, closed-loop information flows allow model parameters or latent states to update responsively given ongoing human interaction or intervention (Tulli et al., 13 May 2024, Li et al., 2020).

Editor's term: "Human-Augmented Model" is used to reference both algorithmic frameworks and practical instantiations where machine and human information pathways are algorithmically intertwined.

2. Classes of Human Augmentation in Modeling

HAMs manifest in a diverse range of technical instantiations, typically falling into several paradigmatic forms:

• Human-State or Goal Integration: The model infers or tracks an explicit state variable signifying human intention, goal, or plan, e.g., via observer/predictor frameworks employing Kalman filters or Bayesian estimation to fuse human and robot perspectives during cooperative control tasks (Li et al., 2020).
• Human-Provided Priors and Representations: Direct provision of human cognitive output as non-parametric priors or representations. A salient example is the use of imprecise, hand-drawn spatial maps for robot navigation, where these maps—containing inherent human cognitive and perceptual bias—are algorithmically aligned with onboard sensory data via large vision-LLMs (VLMs) (Tan et al., 31 Jan 2025).
• Control Signal Augmentation: Human activity or behavior signals, e.g., mobility traces, are formalized as dynamic inputs to dynamical systems (e.g., in Koopman operator-based epidemiological models (Mustavee et al., 2021)), enabling data-driven coupling of behavioral exogenous control with endogenous process evolution.
• Sequential and Interactive Decision Making: HAMs in this context incorporate explicit models of human beliefs, preferences, or cognitive limitations within the agent’s policy or planning framework, including model reconciliation and interpretability-driven feedback, as synthesized in the human-aware AI literature (Tulli et al., 13 May 2024).
• Hybrid Model-Data Fusion: In domains such as TTS, HAMs may utilize human-like synthetic data augmentation (e.g., via voice conversion) or latent state representations engineered to encode human-perceptible cues such as style or timbre (Wang et al., 9 Mar 2024).

3. Formal and Algorithmic Frameworks

3.1 State and Control Fusion

Many HAMs are rooted in formal systems-theoretical or control-based frameworks, where human-derived signals ($u_h$, goals, plans, or measured variables) are treated as system states, observable variables, or control inputs. Notable mathematical treatments include:

• Observer-augmented state estimation:

$$\dot{\bar{\xi}} = \bar{A} \bar{\xi} + \bar{B}(u + \epsilon)$$

where $\bar{\xi}$ contains both physical and human goal parameters (Li et al., 2020).

• Koopman operator representations with exogenous human inputs:

$x_{k+1}^{(h)} = A x_k^{(h)} + B u_k$

Here, $u_k$ encodes temporally-resolved human activity (e.g., aggregated mobility rates) (Mustavee et al., 2021).

3.2 Information and Sensory Integration

HAMs often employ information-theoretic or Bayesian fusion, where human inputs are weighted or combined with machine-generated estimates according to uncertainty or context. In robotics, this includes Kalman-filter-based augmentation where human-planned trajectories are treated as additional, uncertainty-weighted sensory channels in the overall estimation process. The robot’s measurement update becomes:

$$y = [x - \tau,\, \dot{x},\, x - \hat{\tau}_h]^\top + \varepsilon$$

where $\hat{\tau}_h$ is estimated human desired trajectory (Li et al., 2020).

3.3 Hybrid Model Structures and Adaptation

In sequential recommendation, HAMs may explicitly encode user long-term preferences, high/low-order sequential associations, and synergistic cross-item effects via pooling and algebraic combination of embedding representations (Peng et al., 2020):

$$s_{ij} = \mathbf{u}_i \cdot \mathbf{q}_j^\top + \mathbf{h}^{\text{cross}} \cdot \mathbf{q}_j^\top + \mathbf{h}^{(n_l)} \cdot \mathbf{q}_j^\top$$

In continual learning, HAMs manage large libraries of model adapters—each encoding new task knowledge—with dynamic, hierarchical merging mechanisms to maximize cross-task transfer and minimize forgetting (Coleman et al., 16 Sep 2025):

$$\Delta W_{\mathrm{merged}} = \frac{1}{M} \sum_{i=1}^M \alpha_{G_i} B_{G_i} A_{G_i}$$

4. Empirical Applications and Performance

HAMs operationalize human–machine complementarity across various domains. Empirical evaluation frameworks typically compare HAM-enabled architectures against non-augmented or fully automated baselines using task-specific success metrics that expose the utility of human augmentation.

Domain	Human Input	Augmentation Mechanism	Empirical Benefit
Collaborative tracking (Li et al., 2020)	Real-time human haptic input	Sensory estimate fusion via observer	Lower tracking error under uncertainty
Mobile navigation (Tan et al., 31 Jan 2025)	Hand-drawn spatial map	VLM-based multimodal alignment, retrieval	Robust navigation from imprecise input
Epidemiology (Mustavee et al., 2021)	Aggregated mobility time series	Koopman/HDMDc input-output modeling	Accurate multi-week infection forecasts
Recommender systems (Peng et al., 2020)	User-item sequence data	High/low-order structure+synergy pooling	Higher recall/NDCG, fast inference
CL / Task adaptation (Coleman et al., 16 Sep 2025)	Task-identity, content structure	Hierarchical adapter merging	Lower forgetting, scalable adaptation

Empirical analyses repeatedly highlight that the inclusion of human-structured signals or representations enables superior robustness to input noise, data sparsity, and dynamic or nonstationary environments. Notably, in navigation (Tan et al., 31 Jan 2025), HAMs outperform traditional SLAM and non-human-augmented mapless methods—especially under severe sensing ambiguity or when map data acquisition is constrained.

5. Theoretical and Design Implications

HAMs underscore several theoretical design imperatives:

• Model alignment and interpretability: Ensuring the human-augmented component is structured such that its influence is legible and controllable by both human and algorithmic stakeholders (Tulli et al., 13 May 2024).
• Adaptive mechanism design: Algorithms must flexibly weight, fuse, or reject human input according to dynamic circumstances, for example via uncertainty-adaptive filtering (Li et al., 2020) or selection of importance scalars in adapter merging (Coleman et al., 16 Sep 2025).
• Convergence and correctness: When HAMs include parameters for control of influence (e.g., convergence rate $c_0$ in iterative differential equation solvers (Moreira, 2017)), formal guarantees are often established for restricted parameter domains, with empirical studies pointing to broader practical utility under tuned hyperparameters.

A plausible implication is that the growing diversity of HAMs in both vision-language, control, and sequence modeling pipelines reflects a broader shift from user-independent automation towards models designed for resilient, explainable, and user-centric operation.

6. Limitations, Open Problems, and Future Directions

Limitations of current HAMs include:

• Robustness to misleading or erroneous human input: Systems must be hardened against incorrect, ambiguous, or adversarially perturbed human signals.
• Scalability and computational complexity: Some HAM architectures (e.g., adapter libraries) may face barriers at large task or data scales absent pooling, merging, or pruning innovations (Coleman et al., 16 Sep 2025).
• Generalization and transferability: The capability of HAMs to leverage human augmentation in truly open-world or cross-domain settings remains an open empirical and theoretical problem.

Future directions are likely to include deeper fusion of formal human modeling (theory of mind, preference elicitation, intent prediction) with adaptive learning systems, real-time closed-loop interaction with users, and broadening the class of human data and knowledge types that can be consistently and efficiently assimilated.

7. Canonical Examples and Taxonomies

Notable exemplars include:

• Hierarchical Adapter Merging (HAM) for continual learning: Dynamic, similarity-driven grouping and merging of LoRA low-rank adapters for task-robust and scalable adaptation in vision backbones (Coleman et al., 16 Sep 2025).
• Hand-drawn Map Navigation (HAM-Nav): Zero-shot navigation in realistic environments using human spatial cognitive priors and VLM-driven multimodal alignment (Tan et al., 31 Jan 2025).
• Hybrid Associations Model (HAM) for recommendation: Joint modeling of user preferences, sequential dependence, and item synergies via explicit pooling and interaction products (Peng et al., 2020).
• Sensory-augmented collaborative control: Observer/predictor systems fusing human and robot control perspectives for shared tracking tasks, leveraging Kalman filtering (Li et al., 2020).

The literature situates HAMs within a taxonomy of human-aware AI systems characterized by dimensions of human involvement, model representation of human factors, and adaptive interaction protocols (Tulli et al., 13 May 2024). Key criteria include the explicitness, adaptive weighting, and interpretability of the human-augmented pathways.

---

In summary, Human-Augmented Models embody algorithmic frameworks that structurally and adaptively integrate human knowledge, signals, or representations into computational models. Across domains, empirical evidence substantiates their superiority over non-augmented and fully autonomous systems when it comes to resilience, practical utility, and explainability in the presence of real-world uncertainty, data gaps, and complex task demands.