CORE: Self-Training Framework for Virtual Agents

• CORE is a hybrid training framework that balances imitation-based learning with exploration-driven diversity for virtual agents.
• It employs Semantic Code Abstraction to automatically derive executable reward functions from expert demonstrations, eliminating manual reward design.
• Graph Expansion leverages strategy graph construction and trajectory-guided extrapolation to enhance both in-domain and out-of-domain behavioral diversity.

CORE, introduced in “Code-based Inverse Self-Training Framework with Graph Expansion for Virtual Agents” and published on 2026-01-05, is a training framework for Multimodal Virtual Agents that is positioned between imitation-based and exploration-based learning. The framework is presented as a response to a specific tension in agent training: Behavior Cloning is described as simple and effective through imitation but limited by low behavioral diversity, whereas Reinforcement Learning can discover novel strategies through exploration but depends heavily on manually designed reward functions. CORE is proposed to bridge these two regimes by promoting behavioral diversity while eliminating the reliance on manually reward design (Wang et al., 5 Jan 2026).

1. Concept and scope

CORE is defined as a “Code-based Inverse Self-Training Framework with Graph Expansion” for virtual agents (Wang et al., 5 Jan 2026). Its stated application domain is the training of Multimodal Virtual Agents in settings shaped by the integration of Multimodal LLMs. Within that setting, the framework is presented not merely as a reward-learning method or an exploration heuristic, but as a broader training paradigm intended to reconcile imitation from expert demonstrations with exploration beyond them.

The central design objective is dual. First, CORE seeks to preserve the practical advantages of demonstration-driven training. Second, it aims to recover the behavioral diversity commonly associated with exploration-driven methods, but without the usual requirement of manual reward engineering. This places the framework in the broader category of inverse or demonstration-derived training approaches, although the supplied record does not provide the formalism needed to specify its exact relationship to inverse reinforcement learning or other code-based supervision schemes. This suggests that CORE should be understood as a hybrid framework whose novelty lies in how it operationalizes demonstration-derived verification and search-space expansion, rather than in a simple interpolation between existing paradigms.

2. The training problem CORE addresses

The motivating problem is stated in explicitly comparative terms (Wang et al., 5 Jan 2026). Behavior Cloning is characterized as “simple and effective through imitation” but as suffering from “low behavioral diversity.” Reinforcement Learning, by contrast, is characterized as “capable of discovering novel strategies through exploration” but as depending “heavily” on “manually designed reward functions.” CORE is introduced specifically “to address the conflict between these two methods” (Wang et al., 5 Jan 2026).

This framing identifies two deficiencies that often coexist in virtual-agent training. One is under-diversification: a policy may reproduce demonstrated behaviors competently yet remain narrow in strategy space. The other is reward-specification burden: an exploratory learner may, in principle, discover richer behavior, but only if task rewards can be defined with sufficient precision. CORE’s proposal is that these limitations can be attacked together by deriving executable reward structure from demonstrations and then using graph-based and trajectory-based mechanisms to widen the behavioral distribution.

A plausible implication is that CORE treats expert demonstrations not only as trajectories to imitate, but also as a source from which task-validating computational structure can be extracted. Under that reading, demonstrations function as both behavioral exemplars and partial specifications of success conditions. The supplied source, however, does not include the formal definitions, algorithms, or ablations needed to determine how this trade-off is realized in optimization terms.

3. Semantic Code Abstraction and the Label Function

A core component of the framework is “Semantic Code Abstraction” (Wang et al., 5 Jan 2026). According to the supplied description, this mechanism is introduced “to automatically infers reward functions from expert demonstrations without manual design.” The resulting inferred reward function is termed the “Label Function,” and it is defined as “executable code that verifies one key step within a task” (Wang et al., 5 Jan 2026).

This is the most technically distinctive element visible in the available description. The use of executable code as the representation of an inferred reward function implies that reward evaluation is grounded in verifiable procedural structure rather than in manually authored scalar heuristics. The phrase “one key step within a task” is also significant: it indicates that the Label Function is not described as a complete end-to-end task specification, but as a verifier for a salient substep. This suggests a decomposition-oriented supervision regime in which partial correctness can be operationalized through code.

Within the limits of the supplied text, Semantic Code Abstraction can therefore be understood as a mechanism for transforming expert demonstrations into executable, step-level verification artifacts. What is not provided is the technical pathway by which such abstractions are derived: there are no formal schemas, program representations, synthesis procedures, or failure modes in the available record. The details section explicitly states that the supplied text does not contain the manuscript’s “motivation, definitions, algorithms, experiments, etc.,” and that the true paper text would be required for mathematically precise exposition (Wang et al., 5 Jan 2026).

4. Graph expansion and behavioral diversity

CORE’s second major axis is “Graph Expansion” (Wang et al., 5 Jan 2026). This is developed through two named mechanisms with distinct roles in expanding behavior.

The first is “Strategy Graph Expansion,” which is introduced “to enhance in-domain behavioral diversity” (Wang et al., 5 Jan 2026). It “constructs a multi-path graph called Strategy Graph that captures diverse valid solutions beyond expert demonstrations.” The terminology here is important. “Multi-path graph” indicates that the framework does not treat task completion as a single demonstrated route, but as a structured set of alternative valid paths. “Beyond expert demonstrations” further indicates that the graph is intended to generalize from, rather than merely replay, demonstrated behavior.

The second is “Trajectory-Guided Extrapolation,” which is introduced “to enrich out-of-domain behavioral diversity by utilizing both successful and failed trajectories to expand the task space” (Wang et al., 5 Jan 2026). This expands the scope of learning from merely generating alternative in-domain solutions to widening the effective domain of tasks considered during training. The explicit inclusion of failed trajectories is notable because it implies that failure is not treated solely as negative evidence, but as a resource for task-space expansion.

Taken together, these two mechanisms partition diversity into at least two forms. In-domain diversity concerns multiple valid strategies for solving tasks related to the demonstration distribution. Out-of-domain diversity concerns expansion of the task space itself through trajectory-derived extrapolation. This suggests a layered exploration strategy: first diversify solutions within a known domain, then leverage trajectory evidence to move beyond it. The source excerpt, however, does not specify the graph formalism, node or edge semantics, or how successful and failed trajectories are encoded in the extrapolation process.

5. Experimental setting and reported findings

The paper reports experiments “on Web and Android platforms” (Wang et al., 5 Jan 2026). In those experiments, CORE is said to “significantly improve both overall performance and generalization,” which is presented as evidence of its potential “as a robust and generalizable training paradigm for building powerful virtual agents” (Wang et al., 5 Jan 2026).

These claims indicate that the framework is intended for interactive agent environments rather than only synthetic reasoning benchmarks. The reference to both Web and Android platforms implies evaluation across at least two operationally distinct environments, which is relevant because multimodal virtual-agent competence often depends on cross-platform generalization, action verification, and robustness to varied interface structures.

At the same time, the supplied record does not include quantitative results, benchmark names, baselines, task definitions, or implementation details. No metrics, no experimental protocol, and no ablation outcomes are present in the available text. The details section explicitly states that the provided material is only a “LaTeX author-response template and bibliography stub” rather than the paper’s technical content, and that a full account of “semantic code abstraction, strategy graph expansion, trajectory-guided extrapolation, experimental results, etc.” would require the actual manuscript (Wang et al., 5 Jan 2026). Accordingly, the experimental significance of CORE can be characterized only at the level stated in the abstract: improved overall performance and generalization on Web and Android tasks.

6. Interpretive significance and documentation limits

Within the boundaries of the available description, CORE can be interpreted as a framework organized around three coupled ideas: deriving executable step-level reward structure from demonstrations, constructing a graph of diverse valid strategies beyond the demonstration set, and using both successful and failed trajectories to enlarge the effective task space (Wang et al., 5 Jan 2026). This combination suggests an attempt to replace manual reward design with code-level verification while preserving the strategic breadth usually associated with exploratory learning.

The framework’s significance therefore lies less in a single isolated module than in its proposed synthesis of imitation and exploration. If the abstract is taken literally, CORE’s novelty is not simply that it infers rewards, nor simply that it expands behavior, but that it does both in a way meant to remove the customary dependence on hand-authored rewards while increasing behavioral diversity. A plausible implication is that the framework is designed for settings where task validity can be partially checked through executable substep verifiers, making code a mediating layer between demonstrations and exploration.

Any stronger claim would exceed the available evidence. The present source does not provide formal definitions, pseudocode, equations, workflow details, benchmark protocols, or numerical results. As a result, CORE can presently be described with confidence at the level of its declared architecture and stated objectives, but not at the level of full algorithmic reconstruction. The terms “Semantic Code Abstraction,” “Label Function,” “Strategy Graph Expansion,” “Strategy Graph,” and “Trajectory-Guided Extrapolation” are therefore the authoritative conceptual anchors for understanding the framework as presently documented (Wang et al., 5 Jan 2026).