OneLife Framework Overview

• OneLife Framework is a comprehensive approach that defines robust learning and interpretable symbolic modeling under severe resource constraints.
• It integrates methods like single-life reinforcement learning using QWALE, demonstrating 20–60% performance improvements over conventional episodic approaches.
• The framework emphasizes sustainable research practices through FAIR-compliant LCA repositories, ensuring transparent lifecycle impact evaluations.

The OneLife Framework refers to a set of contemporary, technically distinct research efforts unified by their focus on robust learning, adaptive inference, sustainability, and knowledge management in the context of systems that operate under severe resource, interaction, or trial constraints. The term encompasses frameworks for single-episode reinforcement learning, interpretable symbolic modeling from minimal experience, and open-source infrastructures for sustainable scientific research. Central themes include autonomy in novel environments, utilization of limited prior data, effective knowledge transfer, and lifecycle assessment of resources.

1. Single-Life Reinforcement Learning: Motivation and Formulation

Single-Life Reinforcement Learning (SLRL) formalizes the setting in which an agent is allotted only one uninterrupted “life” to accomplish a complex task, without resets or real-time human intervention (Chen et al., 2022). This diverges from conventional episodic reinforcement learning, where repeated trials and resets allow iterative policy refinement. The SLRL paradigm models real-world deployments—such as disaster response robots—where opportunities for trial-and-error correction are infeasible. The critical focus is on leveraging offline prior data from a related but not identical “source” Markov Decision Process (MDP) while facing significant distribution shifts and unstructured novelty in the “target” MDP.

SLRL introduces the following principal challenges:

• Recovery from Novel States: There are no episodic resets; the agent must autonomously recover from state/action outliers.
• Sparse Reward Signals: Rewards are typically delayed or sparse, impeding learning from feedback.
• Distributional Shift: Offline prior data may be misaligned with the single-life operating context due to different environmental morphology or task structure.

2. Q-Weighted Adversarial Learning (QWALE) Algorithm

To address SLRL, the QWALE algorithm employs a distribution matching approach, advancing upon adversarial imitation learning by integrating Q-value-based task progress information (Chen et al., 2022). The target state-action distribution is formalized as

$$\rho^*_{\text{target}}(s, a) \propto \rho^{\beta}(s, a)\exp\left(Q^{\pi_{\text{target}}}(s,a) - V^{\pi_{\text{target}}}(s)\right)$$

where $\rho^{\beta}$ is the reference distribution from source data, $Q^{\pi_{\text{target}}}$ the Q-function for the target task, and $V^{\pi_{\text{target}}}$ the corresponding value function.

QWALE minimizes the Jensen-Shannon divergence between the agent’s rollout distribution and the Q-weighted reference. The loss is given by:

$$\min_{\pi} \max_{D} \mathbb{E}_{(s,a) \sim \rho^{\beta}} \left[w(s,a)\log D(s,a)\right] + \mathbb{E}_{(s,a) \sim \rho^{\pi}} \left[\log(1 - D(s,a))\right]$$

where $w(s,a) = \exp(Q(s,a) - b)$ and $b$ is a normalizing offset. The shaped reward incentivizes behaviors leading toward greater task completion, effectively guiding exploration back toward high-value regimes when novelty disrupts the agent’s progression.

3. Symbolic World Modeling from One-Life Exploration

The OneLife framework for symbolic world modeling targets learning executable, interpretable models of environment dynamics through probabilistic programming from a single episode of unguided exploration (Khan et al., 14 Oct 2025). The world transition function $p(s_{t+1}|s_t,a_t)$ is represented via a set of modular, programmatic “laws” with explicit precondition–effect structures.

Each law $(c_i, e_i)$ features:

• $c_i(s, a)$: Precondition function for law activation.
• $e_i(s, a)$: Effect function predicting state changes.

The world model dynamically assembles a computation graph for each $(s,a)$ tuple, composing the predictive distribution for each observable $o$ as

$$p(o = v | s, a; \theta) \propto \prod_{i \in \mathcal{J}_o(s,a)} \phi_i(o = v | s, a)^{\theta_i}$$

where $\mathcal{J}_o(s,a)$ indexes laws affecting $o$ for $(s,a)$. Credit assignment during learning is routed exclusively through activated laws, controlling the scalability with respect to world complexity and enabling efficient inference in highly stochastic, high-dimensional settings.

Evaluation protocols were introduced including:

• State Ranking (Rank@1, MRR): Probabilistic accuracy in distinguishing future state candidates.
• State Fidelity: Edit distance metrics between predicted and actual states.

In the Crafter-OO testbed, OneLife outperforms strong baselines (e.g., adapted PoE-World) in state ranking metrics (Rank@1 improvement from ~10.8% to 18.7%, MRR from 0.351 to 0.479) and demonstrates planning capability over extended hypothetical rollouts.

4. Sustainable Research Infrastructure and Life Cycle Assessment

The OneLife Framework also encompasses sustainable resource management in research infrastructure, as instantiated in the ORLCA (Open Research Life Cycle Assessment) repository (Wakeling et al., 10 Sep 2025). ORLCA establishes an open-source, FAIR-compliant LCA data repository tailored for the unique requirements of research domains such as particle and accelerator physics.

ORLCA is structured with:

• Standardized Data Formats: JSON-LD, ILCD for interoperability.
• Persistent Identifiers: Data records assigned DOIs via Zenodo, enabling versioning and citation.
• Community Curation: Guidelines, peer-review, and public curation policies for transparent data stewardship.

The repository supports comprehensive impact evaluation and eco-design (e.g., integration with digital twins for pre-prototype simulation). This addresses the barrier of inaccessible or domain-inappropriate commercial LCA datasets, facilitating lifecycle impact analysis for bespoke experimental components or large-scale scientific facilities.

5. Connections Among OneLife Paradigms: Themes and Implications

While the above manifestations of OneLife target distinct problem spaces, they share key conceptual and methodological commonalities:

• Resource Constraints: Each framework is constructed to function under tight limits: single-trial (SLRL), minimal interaction exploration (symbolic modeling), or limited access to sustainability data (ORLCA).
• Leveraging Prior Knowledge: Methods employ prior trajectories, offline data, or domain transfer to bootstrap successful operation when adaptation opportunities are inherently scarce.
• Interpretability and Modularity: Particularly in symbolic world modeling and life cycle assessment, modular structure and explicit activation of laws or data curation policies are prioritized, supporting transparency and human-in-the-loop validation.
• Autonomy in Novelty: Across OneLife instances, the ability to recover from unexpected deviations—whether environmental, task-based, or dataset-related—is a principal goal.

A plausible implication is that advances in any OneLife research direction inform developments in the others; for example, combining robust symbolic world models with single-life task learning could undergird safer agent deployment in non-resettable, high-stakes domains.

6. Empirical Results and Future Research Directions

Empirical validation in the respective OneLife lines includes:

• In SLRL, QWALE demonstrates 20–60% higher success rates than conventional episodic RL across several continuous control domains (Chen et al., 2022).
• In symbolic world modeling, OneLife outperforms baselines on 16 of 23 scenarios in structured environments (Khan et al., 14 Oct 2025).
• ORLCA’s practical efficacy is highlighted in case studies (e.g., ISIS-II Neutron and Muon Source), lowering the overhead and financial burden of bespoke LCA for scientific projects (Wakeling et al., 10 Sep 2025).

Identified research avenues include: strengthening success guarantees in the face of greater novelty, directly incorporating demonstration-only or suboptimal data, broader stochasticity in symbolic models, and the systematic integration of sustainability assessment into digital twin-based science infrastructure.

7. Significance and Outlook

The OneLife Framework, as reflected in these convergent research threads, is driving a shift toward learning and optimization paradigms anchored in practical autonomy, interpretability, and responsibility. Its technical contributions span robust single-trial adaptation, efficient symbolic knowledge acquisition from minimal data, and transparent, community-driven environmental impact management for science. The continuing evolution of OneLife, especially in cross-pollination between reinforcement learning, world modeling, and sustainable infrastructure, underpins the development of resilient, adaptable systems across both artificial agents and the ecosystems in which they operate.