In-Context Curiosity in AI & Learning

• In-context curiosity is a form of epistemic curiosity defined by context-dependent uncertainty in activities, dialogue, and knowledge networks.
• It is operationalized through mechanisms like question generation, prediction error, and interactive exploration in educational and AI environments.
• Applications span classroom learning, human–AI coordination, and reinforcement learning, highlighting its role in adaptive decision-making and knowledge growth.

In-context curiosity is a context-conditioned form of epistemic curiosity in which the immediate state of activity, dialogue, knowledge, or belief determines what appears uncertain, surprising, actionable, or worth exploring. Across the literature, the relevant context may be an interactive simulation, a preceding sentence in a paragraph, a peer interaction, a local region of a knowledge network, or an agent’s current latent world model. Curiosity is then operationalized through question generation, exploratory behavior, prediction error, information gain, social coordination, or structurally defined knowledge gaps (Kowalski et al., 2013, Scialom et al., 2019, Patankar et al., 2022).

1. Definitions and core dimensions

A recurring definition treats curiosity as a threshold of desired uncertainty that leads to exploratory behavior. In a junior-level engineering physics setting, this threshold was operationalized as the questions students composed while exploring interactive simulations, with explicit emphasis on state curiosity as situational, in-context, and malleable, and the further suggestion that repeated reinforcement might strengthen trait curiosity (Kowalski et al., 2013). In this formulation, curiosity is not inferred from affect alone; it is captured through the act of asking questions under immediate task conditions.

A second formulation appears in curiosity-driven question generation, where the distinction between immediate and broader context is explicit. Traditional question generation assumes the answer is present in the source text, whereas curiosity-driven question generation requires a question that is relevant to the source sentence $s_x$ but seeks information not directly contained in it. The paper formalizes this using $QA\_source = QA\_prob(q, s_x)$, $QA\_context = QA\_prob(q, P')$, and reward

$r(q, P, P') = QA\_context - QA\_source,$

so that curiosity is modeled as low answerability from the immediate source and high usefulness in the broader paragraph or conversation (Scialom et al., 2019).

A broader theoretical synthesis defines in-context curiosity through the local topology of a knowledge network. On this view, what becomes interesting depends on whether the current network contains gaps that can be filled, regularities that can be compressed, or constraints that can be reconfigured. The formal apparatus combines Betti numbers for topological cavities, rate–distortion compressibility, and conformational degrees of freedom, thereby treating curiosity as the process of filling, compressing, and reconfiguring knowledge networks rather than merely reducing uncertainty in a scalar sense (Patankar et al., 2022).

A rational-analysis perspective further reframes context as the anticipated distribution of future need. In that account, curiosity is the value of information for future action, with current knowledge value

$V = \sum_k p_k c_k$

and curiosity gradient

$\Omega_k = p_k \frac{dc_k}{dh_k}.$

This makes the curiosity–confidence relation contingent on environmental structure: when need probability is uniform, curiosity decreases with confidence, whereas when need probability increases with exposure, curiosity follows an inverted-U in confidence (Dubey et al., 2017).

2. Educational and collaborative learning settings

In classroom research, in-context curiosity has been operationalized most directly as question production during real-time interaction with disciplinary phenomena. In a Colorado School of Mines electromagnetics course, students used four interactive applets—electrostatic induction, moving charge, an inductance calculator, and a quantum harmonic oscillator visualizer—together with InkSurvey on pen-enabled tablet PCs. The open-format prompts were: “What did you learn from or observe in the simulation?”, “What does this simulation illustrate?”, and “What are you curious about after running the simulation?” Student questions were categorized as Incongruous, Congruous, Modifying, Generalizing/analogy, Causal/creative, or Informational. Every student present submitted at least one response per session, despite no extrinsic incentive. Across the four applets, most questions were Congruous, with notable Incongruous spikes in the moving-charge and quantum-harmonic-oscillator activities; for example, electrostatic induction produced 68 questions from 40 students, of which roughly 74% were Congruous and 12% Modifying, whereas the moving-charge applet produced 53 questions from 31 students, with 40% Incongruous and 25% Modifying (Kowalski et al., 2013).

Small-group learning studies shift the emphasis from individual elicitation to multimodal socio-cognitive dynamics. A continuous-time structural equation model over Rube Goldberg machine collaboration separated individual and interpersonal functions of knowledge identification, knowledge acquisition, and intensification. The strongest standardized loading from latent function to curiosity was Interpersonal Knowledge Identification & Acquisition at $2.612 \pm 0.124$, followed by Individual Knowledge Identification & Acquisition at $2.149 \pm 0.066$, with interpersonal functions overall exerting stronger influence than individual counterparts (Sinha et al., 2022). A related multimodal analysis of child–child collaboration identified 325 significant interpersonal influences versus 154 intrapersonal ones, described as roughly twice as many interpersonal causal influences as intrapersonal ones, and linked curiosity to sequential patterns involving justification, idea verbalization, argument, evaluation, task questioning, confusion, joy, and flow (Sinha et al., 2017).

AI-supported discovery environments extend these findings into human–AI coordination. In an Interactive Molecular Dynamics study on VMD using a Wizard-of-Oz AI tutor teammate, 11 high-school students completed four tasks of increasing complexity over about 60 minutes. Curiosity was indexed through the frequency and complexity of student-initiated questions, while AI behaviors were split into curiosity triggers and curiosity responses. High-performing teams asked more advanced questions and received more curiosity-triggering messages, and the diagonal recurrence rate from CRQA correlated strongly with curiosity at $r = .69$, $p < .001$ (Demir et al., 26 Jun 2025). Taken together, these studies position in-context curiosity as a real-time, socially mediated practice of noticing, questioning, and coordinated hypothesis formation.

3. Conversational systems and language-model prompting

In NLP, in-context curiosity first appeared as a supervised generation target rather than a pedagogical intervention. Curiosity-driven question generation defines samples $QA\_source = QA\_prob(q, s_x)$0, where the source sentence does not contain the answer and the broader context may. On QuAC and a constrained SQuAD derivation, a Transformer with copy mechanism was trained with maximum likelihood, pretraining, and reinforcement learning. The reinforcement objective mixed supervised learning with the reward $QA\_source = QA\_prob(q, s_x)$1, using a BERT-based QA model. On QuAC, RL generally increased $QA\_source = QA\_prob(q, s_x)$2—for example, base_beam5 had $QA\_source = QA\_prob(q, s_x)$3 and RL_beam5 had $QA\_source = QA\_prob(q, s_x)$4—but could also increase $QA\_source = QA\_prob(q, s_x)$5, and Self-BLEU remained very high, indicating low diversity (Scialom et al., 2019).

Task-oriented dialogue systems recast in-context curiosity as the insertion of short, vetted facts at moments of low cognitive load. In the Alexa TaskBot setting for cooking and DIY, 1,351 manually curated fun facts were matched to task title, step text, and ingredients or tools, with a timing policy that never offered curiosities at the beginning of a dialogue, during long steps of at least 200 words, or near the penultimate step. In an A/B test over 1,036 conversations, 71% of users in the curiosity-enabled condition accepted at least one curiosity. Conversations with at least one accepted curiosity averaged a rating of 3.94, compared with 3.62 when curiosities were disabled, and the paper reports an average relative rating improvement of 9.7% for the modal one-curiosity case. Semantic similarity retrieval achieved a rating of 3.99 across 344 conversations, compared with 3.86 across 192 conversations for the text-based method (Vicente et al., 2023).

LLM tutoring studies further treat curiosity as an interaction-level state that can be induced through linguistic operators. CURIOBOT operationalizes Berlyne’s collative variables—Novelty, Complexity, Conflict, Uncertainty, and Neutral—as adaptive tutor-response operators chosen from dialogue state. Across 270 tutoring conversations spanning three model families, three domains, and three topic complexity levels, the system produced up to 2.4x more conversational turns under fixed time budgets, with overall turns increasing from 6.66 for Baseline to 16.12 for CurioBot. Learner-side measures also improved: for example, L2 Productive Struggle rose from 1.94 to 2.86 for Gemini, 2.06 to 3.02 for Claude, and 1.86 to 2.68 for GPT. Operator success rates were highest for Conflict at 79.03%, followed by Novelty at 61.42% and Neutral at 60.75%, with the largest gains occurring on transitions out of Uncertainty to Novelty or Conflict (Ganganath et al., 21 Jun 2026).

A separate line of work uses curiosity prompting to improve answer quality rather than learner exploration. Self-critique-guided curiosity refinement first generates a curiosity-oriented “confusion output,” then an optimized answer, then a self-critique over explanation honesty, guidance helpfulness, and solution appropriateness, and finally a minimally edited refinement. On the 930-query HONESET dataset, evaluated with GPT-4o as judge, this produced relative gains in $QA\_source = QA\_prob(q, s_x)$6 scores over curiosity-driven prompting ranging from 1.4% to 4.3%; for example, GPT-4o improved from 8.627 to 8.748, while Llama 4 Maverick improved from 7.186 to 7.496 (Ho et al., 19 Jun 2025). This suggests that in-context curiosity can function both as exploratory elicitation and as a structured precondition for self-correction.

4. Knowledge networks, rational choice, and human questioning

A structural theory of in-context curiosity models knowledge growth as a node-ordered filtration

$QA\_source = QA\_prob(q, s_x)$7

with three associated measures: Betti curves for information gaps, compressibility for parsimonious representation, and conformational degrees of freedom for flexibility. On Wikipedia-derived data from 149 individuals in the KNOT study and 30 collective disciplinary subgraphs, empirical knowledge networks differed from degree-preserving nulls with $QA\_source = QA\_prob(q, s_x)$8 across all major curves. Individuals and collectives showed slower growth of $QA\_source = QA\_prob(q, s_x)$9 and $QA\_context = QA\_prob(q, P')$0 than nulls, faster growth of $QA\_context = QA\_prob(q, P')$1, increased compressibility, and greater-than-expected flexibility, supporting a pluralistic account in which curiosity fills low-dimensional gaps while also increasing compressibility and reconfiguration capacity (Patankar et al., 2022).

The kinesthetic theory of curiosity complements this by treating curiosity as movement through information space. It distinguishes Busybody, Hunter, and Dancer modes, generated by different balances of edge reinforcement and Lévy-flight exploration. In Wikipedia browsing, humans exhibited a step-length exponent of $QA\_context = QA\_prob(q, P')$2, consistent with Lévy-flight dynamics favorable for sparse, unpredictable targets. The resulting personal knowledge networks were characterized by hubs, modules, and compressibility measured via the map equation codelength $QA\_context = QA\_prob(q, P')$3 (Zhou et al., 2020). This formulation makes local context explicit: the next step depends on recent path history, local neighborhood structure, semantic similarity, and uncertainty gradients.

The rational-analysis account provides a decision-theoretic bridge between these structural theories and experimental findings. Because curiosity is weighted by need probability $QA\_context = QA\_prob(q, P')$4 and learning progress $QA\_context = QA\_prob(q, P')$5, it need not track novelty alone. Under uniform sampling, $QA\_context = QA\_prob(q, P')$6 implies monotone novelty-seeking. Under need probability proportional to exposure, $QA\_context = QA\_prob(q, P')$7 implies an inverted-U with peak at $QA\_context = QA\_prob(q, P')$8. A trivia experiment with 220 MTurk participants confirmed the contextual shift: in the bonus round, curiosity showed an inverted-U in the confidence-sampling condition but a decreasing relation in the uniform condition (Dubey et al., 2017).

Large-scale analysis of natural human questions shows that curiosity-driven inquiry is often causal. CausalQuest contains 13,500 questions from Quora, MS MARCO, Natural Questions, ShareGPT, and WildChat, with 5,701 causal questions, or 42% of the dataset. Causal questions were longer on average than non-causal ones, at 12.56 words versus 10.45, and were dominated by cause-seeking (36.3%), steps-seeking (27.8%), and recommendation-seeking (12.2%). An iterative prompt-improvement framework using GPT-4-turbo reached F1 88.7% for causal versus non-causal classification (Ceraolo et al., 2024). This suggests that in-context curiosity in natural language is frequently expressed as causal inquiry about prediction, intervention, and counterfactual structure.

5. Computational agents, multimodal reasoning, and exploration

In reinforcement learning, in-context curiosity is often instantiated as an intrinsic reward derived from the agent’s current predictive context. Latent Bayesian Surprise defines

$QA\_context = QA\_prob(q, P')$9

where surprise is the KL divergence between a latent posterior and prior over next-step dynamics. This avoids the expense of parameter-space Bayesian surprise and is more resilient to stochasticity than raw prediction error. In stochastic Mountain Car, LBS coverage dropped from 91.75% to 82.38% in the Frozen condition and to 87.0% in the Evolving condition, which was a smaller reduction than Disagreement, VIME, ICM, or RND; in wall-clock comparisons it was about 5–10% slower than ICM/RND, whereas VIME was about 200% slower than ICM/RND (Mazzaglia et al., 2021).

Active-inference work shows that curiosity can also be induced by the geometry of the agent’s internal world model. When policies are instantiated as group actions that change the agent’s internal frame, Euclidean rigid motions leave epistemic value invariant, whereas projective transformations produce depth-dependent magnification. With projective mapping $r(q, P, P') = QA\_context - QA\_source,$0 and Jacobian determinant $r(q, P, P') = QA\_context - QA\_source,$1, epistemic value increases as depth $r(q, P, P') = QA\_context - QA\_source,$2 decreases, so the agent selects approach behavior toward the object of interest. In the reported simulations, the Euclidean agent remained idle because no action was better than the initial state, while the projective agent consistently moved toward the target (Sergeant-Perthuis et al., 2023).

A related intuitive-physics framework treats curiosity as arousal at discontinuities in a “body-budget” over shape constancy, spatiotemporal continuity, and object permanence. Using YOLOv4, Kalman filtering, and occluder-aware reasoning on IntPhys-style events, the system computes a composite score

$r(q, P, P') = QA\_context - QA\_source,$3

and triggers curiosity when observed and predicted trajectories diverge beyond contextual explanation. Occluder presence determines whether an apparent discontinuity is judged “Possible” or “Not Possible,” making curiosity explicitly dependent on immediate scene context rather than on raw novelty alone (Gaikwad et al., 2021).

Multimodal LLM work extends this logic to selective sensing. Chain of Questions structures multimodal reasoning as

$r(q, P, P') = QA\_context - QA\_source,$4

so that generated questions activate object detection, speech-to-text, spatial detection, or other modality-specific components. On a 180,629-instance benchmark integrating WebGPT, ScienceQA, AVSD, and ScanQA, FLAN T5 XL achieved 137,701 matches and 42,928 mismatches for a 76.2% match rate, and asked modality-relevant questions on 80.0% of instances; Llama 2 7B achieved 43.9% match and 41.0% asked (Iji et al., 6 Aug 2025). This suggests that in-context curiosity can be realized as an explicit policy for deciding what sensory information to acquire before reasoning.

6. Measurement issues, controversies, and open problems

A major recent question is whether in-context curiosity can be distilled into pretrained decision models. The Prediction-Powered Transformer augments Decision-Pretrained Transformers with an auxiliary reward predictor and a curiosity vector

$r(q, P, P') = QA\_context - QA\_source,$5

which enters the policy loss through a curiosity-weighted inner product. On Gaussian multi-armed bandits, PPT moderated the performance degradation seen in DPT under higher test-time reward variance, especially with biased pretraining data; stable performance was reported for $r(q, P, P') = QA\_context - QA\_source,$6 in the range $r(q, P, P') = QA\_context - QA\_source,$7, while very large $r(q, P, P') = QA\_context - QA\_source,$8 could destabilize training (Yang et al., 30 Sep 2025). However, a complementary theoretical analysis proves that in general BAMDPs no finite-horizon reward constructed solely from an in-context learner’s prediction errors and context manipulations can unbiasedly identify Bayesian information gain. Positive results are recovered for Bayesian Experimental Design, where the proposed sum-of-future prediction gains and description-length reduction rewards bound and asymptotically converge to true learning progress (Elmoznino et al., 17 Jun 2026).

Measurement remains a persistent difficulty across human and machine settings. The engineering-physics study reports counts and category percentages but no control group, inter-rater reliability, Cronbach’s alpha, or standardized curiosity scale (Kowalski et al., 2013). Curiosity-driven question generation reports that automated metrics do not capture external knowledge well, and human evaluation shows that External Knowledge does not correlate with the automatic metrics used (Scialom et al., 2019). The Wizard of Curiosities acknowledges selection bias because only users who accepted the offer actually heard the curiosity and because users could not be tracked longitudinally due to privacy constraints (Vicente et al., 2023). CoQ, despite its interpretable question–task–sensor chain, does not define an explicit information-theoretic curiosity objective and evaluates primarily modality alignment rather than downstream task accuracy (Iji et al., 6 Aug 2025).

These limitations support a pluralistic reading of the field. In-context curiosity is not reducible to novelty, uncertainty, or question frequency alone. Depending on the setting, it may be the production of open-format questions during simulation, the selective asking of follow-up questions in dialogue, the closure of topological cavities in a knowledge graph, the choice of high-value information under future need distributions, the KL divergence between latent priors and posteriors, or the decision to activate a sensor or internal frame. A plausible implication is that “in-context curiosity” is best understood as a family of mechanisms for coupling local context to exploratory action, rather than as a single metric or theory.