Human-in-Context (HiC): Lifelong Context Recognition

• Human-in-Context (HiC) is a framework that continuously recognizes personal contexts by integrating multi-sensor streams with structured, ego-centric knowledge graphs.
• It employs uncertainty-aware learning techniques, including convolutional-LSTM and incremental Gaussian Process classifiers, to adapt to evolving user data.
• The system maintains semantic consistency through bi-directional alignment, using drift detection and interactive updates to recover from performance drops.

Searching arXiv for the primary paper and closely related work on personal context recognition / human intent inference, plus the unrelated Hi-C acronym usage to clarify ambiguity. {"query":"arXiv (Bontempelli et al., 2022) Lifelong Personal Context Recognition Human-in-Context", "max_results": 5} {"query":"\"Lifelong Personal Context Recognition\" (Bontempelli et al., 2022)", "max_results": 10} {"query":"arXiv context-aware human intent inference (Zhang, 2018)", "max_results": 10} Human-in-Context (HiC) denotes a research program for building AIs that live in lifelong symbiosis with a human by recognizing, updating, and explaining the human’s personal situational context at any moment in time. In "Lifelong Personal Context Recognition," HiC is realized by combining a structured, ego-centric knowledge representation of contexts as Knowledge Graphs (KGs), a suite of incremental, uncertainty-aware learning algorithms for inferring the current context from streams of multi-sensor observations, and a continuous, bi-directional human–AI alignment protocol that interleaves automated recognition, drift detection, and interactive clarification. The central claim is that this objective lies at the intersection of knowledge representation and machine learning: neither technology can achieve the goal without the other (Bontempelli et al., 2022).

1. Definition and problem setting

HiC frames context recognition as a lifelong, personal, and ego-centric problem rather than as a fixed-label activity-recognition task. The AI must understand the user’s context in a form that is human-like and ego-centric, robustly recognize that context under continual change, and maintain alignment between the AI’s and human’s representations of the world through continual bidirectional interaction (Bontempelli et al., 2022).

Within this formulation, the observed input at time $t$ is a multimodal signal $x_t$, such as accelerometer, GPS, or microphone data, and the latent state is a structured context object $C_t$, identified with the current KG. Context recognition is therefore posed as a structured prediction problem over time rather than a flat classification problem. This framing makes explicit that the target is not merely an activity label but a structured representation of the user’s current situation.

Component	Function
Structured, ego-centric KGs	Represent personal contexts
Incremental, uncertainty-aware learning	Infer context from multi-sensor streams
Continuous, bi-directional alignment	Interleave recognition, drift detection, and clarification

A plausible implication is that HiC treats context as an evolving semantic object whose meaning depends on the user’s own terminology, routines, locations, objects, events, and social entities, rather than as a universal ontology fixed in advance.

2. Knowledge representation of personal context

In HiC, the AI’s world is the union of all contexts ever experienced by the user, organized as a sequence of KGs. At time $t$, the current context graph is

$G_t = (V_t, E_t),$

where $V_t$ includes nodes for the user ("me"), Location, Event, Objects, Social entities, and related entities, while $E_t$ is the set of labeled relations. A Life Sequence is the ordered sequence $(G_1, \ldots, G_T)$ (Bontempelli et al., 2022).

To embed and compare contexts, the system uses a relational GNN encoder. Node states are updated by

$$h_v^{(k+1)} = \sigma \left( \sum_{(u,r,v)\in E} W_r h_u^{(k)} + b \right), \qquad h_v^{(0)} = \phi(\text{label}_v).$$

A global context vector is then obtained as

$g_t = \text{READOUT}(\{h_v^{(K)}\}),$

and this vector is used both to condition the ML predictor and to measure KG drift. When the user teaches a new concept or corrects a label, the graph is updated through graph-editing operations, specifically add/remove nodes or edges, applied to $x_t$0 to yield $x_t$1, with identity maintained via persistent entity IDs.

This representational scheme gives HiC a semantics-bearing substrate that can encode relations such as location, event membership, synonymy, and social linkage. The paper’s position is that such structure is indispensable because raw sensor features alone do not maintain long-term semantic consistency under evolving user practices.

3. Machine-learning architecture for context inference

HiC develops two complementary machine-learning approaches for inferring the current context from streaming observations. The first is a dynamic Bayesian or state-space formulation:

$x_t$2

Here $x_t$3 can be a deep-network likelihood and $x_t$4 a learned transition prior. In practice, the observation model is instantiated with a convolutional-LSTM encoder that maps windows of sensor data to context embeddings, and the transition model is instantiated with either a small MLP or a parameterized Gaussian (Bontempelli et al., 2022).

The second approach is an incremental skeptical Gaussian Process classifier designed to handle non-stationarity and quantify uncertainty over discrete context labels. At step $x_t$5, the posterior over the latent function $x_t$6 is updated by

$x_t$7

Uncertain or highly inconsistent annotations trigger a skeptical query to the user for correction. The selective update is governed by the acquisition criterion

$x_t$8

together with a conflict score comparing the user label $x_t$9 to the model’s most likely label $C_t$0.

Across learning phases, the loss takes the form

$C_t$1

where the regularizer constrains the current parameters to remain close to the previous parameters, smoothing abrupt online adaptation. This architecture gives HiC both temporal modeling capacity and uncertainty-aware selective supervision.

4. Lifelong adaptation, drift, and consistency

HiC treats lifelong learning as a dual drift problem. Human behavior and semantics exhibit both data-level drift, in which sensor distributions change, and knowledge-level drift, in which the user’s own concepts and social circles evolve (Bontempelli et al., 2022).

Automated drift detection is performed by monitoring

$C_t$2

If $C_t$3, a knowledge-drift event is raised. When drift cannot be resolved from data alone, the AI enters interactive disambiguation, asking targeted questions such as whether one label is the same as another label or whether concepts should be merged. Regularization against catastrophic forgetting is provided either by the term $C_t$4 or by more sophisticated elastic-weight consolidation.

The paper further states that, under mild assumptions on the smoothness of $C_t$5 and on the frequency of user feedback, the online GP converges to a stable classifier and the KG remains consistent with probability approaching $C_t$6 as $C_t$7. This suggests that HiC aims not only at empirical adaptation but also at asymptotic stability of the joint semantic and statistical state.

5. Bidirectional alignment and the ML–KR coupling

All context updates rely on a three-phase loop run at each time step: prediction, selective clarification, and model/knowledge update. Operationally, the loop predicts $C_t$8; queries the user if uncertainty or conflict is high; updates the model by minimizing $C_t$9; computes the context embedding $t$0 from $t$1; and, if $t$2, asks the user to resolve knowledge drift and edits the graph accordingly (Bontempelli et al., 2022).

The interaction between machine learning and knowledge representation is explicit. Context KGs provide structured constraints on permissible context assignments through the soft constraint

$t$3

which penalizes predictions that violate known relations. Conversely, high-confidence ML predictions are used to propose KG expansions, including new event types, synonyms, or functional relations, subject to user acceptance or rejection.

The resulting joint optimization is

$t$4

solved online by alternating ML updates and KG edits. This is the core technical expression of bidirectional alignment: symbolic structure constrains inference, while statistical predictions drive graph revision.

6. Empirical findings and major lessons

The methods are evaluated on the SmartUnitn2 dataset, comprising 158 students over four weeks with multi-sensor and self-report data (Bontempelli et al., 2022). The reported findings are specific. Off-the-shelf deep nets, namely CNN+LSTM, reach approximately $t$5 context-classification accuracy in a static split. The incremental skeptical GP reduces labeling noise by $t$6 and raises online accuracy to approximately $t$7. Automated drift detection flags real knowledge shifts, including semester end and relocation, with $t$8 recall; after interactive adaptation, the system recovers from a $t$9 accuracy drop back to within $G_t = (V_t, E_t),$0. User burden remains low, averaging one clarification request every $G_t = (V_t, E_t),$1–$G_t = (V_t, E_t),$2 hours of usage, with approximately $G_t = (V_t, E_t),$3 of those resolved in one step.

The paper draws several lessons from these results. A purely ML-centric approach quickly breaks under concept and knowledge drift, while KR alone cannot infer contexts from raw signals; hybridization is therefore described as mandatory. Bi-directional explainability, implemented through skeptical queries and model explanations, is described as essential for maintaining user trust and data consistency. Real-world, in-the-wild experiments are further presented as necessary for surfacing issues in CV and NLP that do not appear in traditional activity-recognition benchmarks.

Future directions are stated as richer CV/NLP integration for zero-shot context discovery, large-scale studies on social-practice variation using WeNet datasets, and theoretical analysis of regret bounds for interactive, drifting-environment learning.

7. Related research and terminological ambiguity

HiC belongs to a broader family of context-aware human-centered inference systems, but its objective is more general than intent recognition alone. A related line of work, "Context-aware Human Intent Inference for Improving Human–Machine Cooperation," develops a three-stage pipeline that combines heterogeneous wearable sensing, a multi-task adversarial network, contextual fusion through a Dynamic Bayesian Network augmented by an RNN, and ontological calibration via probabilistic first-order rules. In that setting, Stage 1 reaches $G_t = (V_t, E_t),$4 accuracy, Stage 2 reaches $G_t = (V_t, E_t),$5, and Stage 3 reaches a final $G_t = (V_t, E_t),$6 accuracy with macro-F1 of $G_t = (V_t, E_t),$7 on five scripted daily-life intents under varied contexts (Zhang, 2018). This adjacent work shares HiC’s emphasis on multimodal sensing, temporal dependence, context variables, and symbolic constraints, but it is framed around real-time intent inference rather than lifelong personal context recognition.

The acronym "HiC" is also used in an unrelated molecular-biology literature concerning Hi-C contact matrices. In "Fast Detection of Block Boundaries in Block Wise Constant Matrices: An Application to HiC data," the problem is the estimation of block boundaries in noisy chromosomal contact matrices via an $G_t = (V_t, E_t),$8-penalized least-squares criterion and a fast 2D-LARS implementation, with application to topological domain discovery (Brault et al., 2016). That usage is terminologically distinct from Human-in-Context.

Taken together, these neighboring usages clarify the specificity of Human-in-Context as a lifelong personal context-recognition framework grounded in KGs, uncertainty-aware online learning, and continual human–AI semantic alignment.