Unsupervised Real-world Skill Acquisition (URSA)

• URSA is an unsupervised framework for robot behavior learning that acquires diverse skills without relying on task-specific rewards.
• It employs techniques like latent-conditioned reinforcement learning, Lipschitz constraints, and skill embeddings to enable robust zero-shot transfer and composition.
• The approach bridges simulation and real-world interactions, advancing methods for transferable, safe, and damage-adaptive robot skill acquisition.

In the cited literature, Unsupervised Real-world Skill Acquisition (URSA) denotes a research program aimed at learning reusable robot behaviors without task-specific reward engineering and, in stronger forms, without task labels, demonstrations tailored to the deployment task, or hand-authored skill boundaries. Across this work, a “skill” may be represented as a latent code, a goal in a learned outcome space, a temporally extended transition embedding, a factorized control variable, a dependency graph over state factors, or an archived behavioral descriptor. The unifying objective is to acquire behavior repertoires that are diverse enough to support transfer, composition, or zero-shot reuse, while remaining grounded in real sensor data, real-world robot interaction, or scalable weakly supervised data sources such as unlabeled videos and automatically organized repositories (Laversanne-Finot et al., 2019, Sharma et al., 2020, Grillotti et al., 26 Aug 2025).

1. Scope and historical emergence

A recurrent theme in URSA is the replacement of task-specific supervision with intrinsic objectives, learned goal spaces, or automatically induced structure. Early real-robot work on intrinsically motivated goal exploration showed that a 6-joint robotic arm could learn to manipulate a ball inside an arena by choosing self-generated goals in a latent space learned from images, rather than in an engineered outcome space (Laversanne-Finot et al., 2019). In parallel, video-grounded work began to treat passive observation as a source of reusable structure: "Adversarial Skill Networks: Unsupervised Robot Skill Learning from Video" learns a task-agnostic skill embedding from synchronized unlabeled multi-view videos and then reuses that embedding as a reward signal for downstream policy optimization (Mees et al., 2019).

The transition from simulation-centric unsupervised RL to physical deployment became explicit in "Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learning" (Sharma et al., 2020). That work converted DADS into an off-policy asynchronous variant, off-DADS, and demonstrated reward-free real-world training on a D’Kitty quadruped. More recent work widened the notion of URSA in two directions. One direction focused on direct on-hardware quality-diversity, exemplified by the framework explicitly named URSA in "From Tabula Rasa to Emergent Abilities: Discovering Robot Skills via Real-World Unsupervised Quality-Diversity," which learns a repertoire of safe, diverse, high-performing skills directly on a Unitree A1 in both simulation and the real world (Grillotti et al., 26 Aug 2025). The other direction expanded “unsupervised” to include weakly supervised or automatically supervised systems that avoid deployment-time teaching by relying on large offline corpora, automated annotation, or structured retrieval, as in "Uni-Skill: Building Self-Evolving Skill Repository for Generalizable Robotic Manipulation" (Xie et al., 3 Mar 2026).

This trajectory suggests that URSA is not a single algorithmic family but an umbrella over several related agendas: reward-free exploration, unsupervised skill discovery, learned goal spaces, real-world representation learning, repertoire discovery, and automatically expandable skill memory. The main line of continuity is the attempt to move from one-task-one-reward learning toward reusable behavioral competence.

2. Core objective formulations

One major line of work formulates URSA through latent-conditioned RL. In DADS and off-DADS, the policy is conditioned on a latent skill $z$, and learning maximizes a conditional mutual-information objective $\mathcal{I}(s', z \mid s)$, implemented through a variational skill-dynamics model $q_\phi(s' \mid s, z)$. The resulting intrinsic reward is high when a transition is likely under the current skill and unlikely under alternative skills, so different latents induce distinguishable, predictable state changes (Sharma et al., 2020). DIAYN-style formulations replace skill dynamics with a discriminator $q(z \mid s)$, but several later papers criticize plain mutual-information maximization for favoring skills that are statistically distinguishable yet behaviorally trivial.

A second line therefore redefines what counts as a useful skill. "Lipschitz-constrained Unsupervised Skill Discovery" introduces LSD, which rewards directional displacement in a learned 1-Lipschitz representation: $r_t = (\phi(s_{t+1})-\phi(s_t))^\top z$. Because latent displacement is constrained by state displacement, large reward is tied to actual movement in state space rather than to easily classifiable static postures (Park et al., 2022). "Controllability-Aware Unsupervised Skill Discovery" then replaces Euclidean state distance with a learned controllability-aware distance proportional to $-\log q_\theta(s' \mid s)$, so hard-to-achieve transitions become “farther” than easy ones under the current repertoire; this is intended to bias learning toward object interaction and other difficult behaviors that prior methods often ignore (&&&10&&&).

Several later methods further specialize the objective. CeSD partitions the state space into clusters using learned prototypes, assigns one skill to each cluster, maximizes local entropy inside each partition, and adds a state-distribution constraint to reduce overlap between skills (Bai et al., 2024). SD3 instead maximizes the deviation of one skill’s state density from regions explored by other skills and adds a latent-space novelty term derived from a skill-conditioned CVAE, explicitly targeting both inter-skill separation and intra-skill exploration in high-dimensional spaces (Xiao et al., 17 Jun 2025). RSD reframes skill discovery as a min–max game between a skill-conditioned policy and a regret-aware skill generator, using stagewise value improvement as a proxy for which skills remain under-converged and therefore deserve more exploration (Zhang et al., 26 Jun 2025).

A separate but related viewpoint casts unsupervised skill learning as curriculum design in goal-conditioned RL. "Variational Curriculum Reinforcement Learning for Unsupervised Discovery of Skills" rewrites variational empowerment as goal-conditioned RL with intrinsic reward $r_{\text{int}}(s,g)=\log q_\lambda(g\mid s)$, then proposes Value Uncertainty Variational Curriculum (VUVC), which samples goals according to value uncertainty and density skewing rather than from a fixed prior (Kim et al., 2023). In this formulation, the central question is not only what reward to maximize, but which goals or skills should be practiced at each stage.

3. Skill representations and inductive biases

URSA research diverges sharply in how it represents a skill. In ASN, a skill is not a whole trajectory or a symbolic primitive, but the embedding of two sequential video frames separated by a stride, $\mathbf{x}=(E(v),E(w))$. That design makes the skill representation explicitly temporally extended and allows downstream RL to use distances in the learned embedding as perceptual reward (Mees et al., 2019). In IMGEP-style real-robot exploration, by contrast, a skill is a parameterized outcome in a learned latent goal space, and the robot learns inverse mappings from latent goals to DMP parameters (Laversanne-Finot et al., 2019).

Other papers impose stronger structure. DIS learns skills incrementally, one after another, with each skill represented by its own independent policy network; old skills are preserved by freezing them rather than by regularization losses (Shafiullah et al., 2022). "Divide, Discover, Deploy: Factorized Skill Learning with Symmetry and Style Priors" factorizes both state and skill spaces, $S=S_1\times\cdots\times S_N$ and $Z=Z_1\times\cdots\times Z_N$, assigns different USD objectives to different factors, and adds symmetry priors plus a style factor for safety and deployability (Cathomen et al., 27 Aug 2025). SkiLD goes further by representing a skill as $\mathcal{I}(s', z \mid s)$0, where $\mathcal{I}(s', z \mid s)$1 is a target local-dependency graph over state factors and $\mathcal{I}(s', z \mid s)$2 is a diversity indicator; here the intended skill effect is not merely “move differently,” but “induce a different interaction pattern among factors” (Wang et al., 2024).

Repository- and planner-based systems use yet another notion of skill. In Uni-Skill, a skill exists simultaneously as a textual/API description, a node in a VerbNet-inspired hierarchy, a set of demonstration segments, a bundle of semantic constraints and trajectory references, and finally an executable sequence of 6-DoF poses (Xie et al., 3 Mar 2026). In the quality-diversity URSA framework, a skill is the expected feature vector $\mathcal{I}(s', z \mid s)$3, with $\mathcal{I}(s', z \mid s)$4 either manually specified or learned by a VAE; the achieved skill space is then the subset of descriptors that the skill-conditioned policy can realize safely (Grillotti et al., 26 Aug 2025).

These differences matter because they determine what reuse means. A skill library based on latent directions, cluster occupancy, or dependency graphs supports different kinds of downstream control than one based on API names, trajectory templates, or behavior descriptors. The literature therefore treats “skill” less as a fixed formal object than as a design choice linking unsupervised interaction to later transfer.

4. Real-world grounding and data regimes

URSA methods vary substantially in how directly they engage the real world. One class uses real sensory data without closing the robot-control loop physically. ASN trains on unlabeled synchronized multi-view demonstrations, including a real-world human video dataset with 60 multi-view videos per task and 24 minutes of interaction, and evaluates reward quality and transfer on real visual data, but the RL control experiments remain in simulation (Mees et al., 2019). Similarly, some newer methods pursue high-dimensional simulated settings intended to approximate real sensing or household complexity without direct deployment.

A second class learns from physical interaction but under carefully engineered assumptions. The IMGEP/VAE work uses a real 6-joint arm, overhead $\mathcal{I}(s', z \mid s)$5 images, and DMP-parameterized action primitives to autonomously expand ball-position coverage in the physical world (Laversanne-Finot et al., 2019). Off-DADS goes further toward online real-world URSA: training is conducted on a real D’Kitty quadruped for about 20 hours and approximately 300,000 samples, using off-policy replay, asynchronous data collection, and intrinsic reward recomputation; learned skills are then reused via MPC for goal-directed navigation without additional training (Sharma et al., 2020).

A third class combines simulation pretraining with real deployment. VUVC trains navigation skills in simulation and transfers them zero-shot to a Clearpath Husky A200, then improves long-range performance by coupling the learned local controller with an A* global planner (Kim et al., 2023). Factorized quadruped USD with symmetry and style priors is also trained entirely in simulation and transferred zero-shot to physical ANYmal-D hardware, where manually commanded latent factors yield behaviors such as forward/backward walking, pitching, crouching, rotating, and combinations thereof (Cathomen et al., 27 Aug 2025).

The strongest real-time on-hardware repertoire discovery in the supplied literature appears in the 2025 URSA framework and the 2026 MOD-Skill paper. URSA trains directly on a Unitree A1 for 5 hours of real-world data using an asynchronous DayDreamer-based architecture, a safe repertoire archive, and either heuristic or VAE-learned descriptors (Grillotti et al., 26 Aug 2025). "Diverse Skill Discovery for Quadruped Robots via Unsupervised Learning" evaluates MOD-Skill on the 12-DOF Unitree A1 in simulation and on the real robot, reporting an 18.3% expansion in state-space coverage compared to its baseline and attributing this to the OMoE policy plus multi-discriminator intrinsic reward (Cui et al., 10 Feb 2026). These results indicate that real-world URSA is no longer confined to passive visual grounding, although the strongest demonstrations still cluster around quadruped locomotion rather than open-ended manipulation.

5. Transfer, composition, and downstream reuse

A central criterion in URSA is whether unsupervisedly acquired behaviors are later useful. Many papers treat downstream reuse not as an auxiliary benefit but as the main justification for skill discovery. In ASN, the learned embedding becomes a perceptual reward $\mathcal{I}(s', z \mid s)$6 below threshold and $\mathcal{I}(s', z \mid s)$7 otherwise; PPO then uses this reward to solve unseen tasks that require interpolation of previously observed skills (Mees et al., 2019). In off-DADS, the learned skill-dynamics model $\mathcal{I}(s', z \mid s)$8 supports MPC directly in skill space, allowing goal-oriented navigation without extra reward-based training (Sharma et al., 2020).

Other methods make downstream control more explicit. LSD uses the learned representation $\mathcal{I}(s', z \mid s)$9 for zero-shot goal following by setting the skill direction to $q_\phi(s' \mid s, z)$0, repeatedly replanned from the current state (Park et al., 2022). DIS uses a high-level PPO controller over frozen incremental skills, showing faster downstream convergence than DIAYN and Off-DADS in hierarchical goal-conditioned Ant tasks, including skills learned in changing environments (Shafiullah et al., 2022). ATR learns robust low-level skills under automatically generated task distributions, then composes them with a symbolic planner to solve unseen sequential manipulation problems in simulation and on a Franka robot (Fang et al., 2022).

Repository-based systems shift the locus of reuse from low-level policy reuse to skill-memory reuse. Uni-Skill augments a base skill library whenever a sufficiency discriminator detects that existing APIs are inadequate, then retrieves demonstrations from SkillFolder to ground the new skill with semantic constraints and trajectory references (Xie et al., 3 Mar 2026). SkiLD evaluates transfer by training a high-level task policy over discovered interaction skills and reports superior performance on long-horizon sparse-reward household tasks such as thawing, cleaning, and cutting, with the strongest gains on tasks requiring bottleneck interactions (Wang et al., 2024). The URSA repertoire framework reuses its discovered descriptors for damage adaptation: in simulation it outperforms all baselines in 5 out of 9 damage scenarios, and on the real robot in 3 out of 5 scenarios, including recovery within 8 ITE iterations in one back-left leg damage setting (Grillotti et al., 26 Aug 2025).

Taken together, these results suggest that “reuse” in URSA takes at least four forms: perceptual reward reuse, low-level policy reuse, planning/model reuse, and repository/memory reuse. The breadth of these mechanisms is one reason the area remains methodologically heterogeneous.

6. Limits, misconceptions, and open problems

A recurring misconception is that all work labeled “unsupervised” eliminates strong prior structure. The cited literature does not support that reading. Several systems remain fully reward-free during pretraining yet still depend on privileged state, engineered action abstractions, or curated sensing setups. The 2019 real-robot goal-space work uses DMPs and nearest-neighbor inverse models rather than end-to-end control (Laversanne-Finot et al., 2019). ASN requires synchronized multi-view videos and does not close the loop on a physical robot (Mees et al., 2019). LSD explicitly warns that its Lipschitz constraint may cease to be semantically meaningful for pixel observations (Park et al., 2022).

Another recurrent issue is that many methods are only partially “real-world.” Simulation-only skill discovery remains common, and even strong robotics papers often stop at sim-to-real transfer rather than lifelong physical interaction. "Divide, Discover, Deploy" is candid that factorization is user-defined, symmetry maps are designed by the user, and safety relies on manually specified style and regularization terms (Cathomen et al., 27 Aug 2025). SkiLD assumes factored state spaces and, in harder domains, uses ground-truth local dependencies from the simulator instead of learned dependency inference (Wang et al., 2024). ATR automates task generation but still presupposes a predefined skill set, symbolic predicates, a procedural generator, and hand-specified success conditions (Fang et al., 2022).

The boundary between unsupervised, weakly supervised, and automatically supervised systems is therefore a substantive controversy rather than a terminological footnote. Uni-Skill explicitly depends on a manually designed base skill library, VerbNet priors, DROID demonstrations, VLM-based annotation, CLIP retrieval, AnyGrasp, top-down placement, and external foundation models; it reduces deployment-time supervision but is not unsupervised in the strict latent-discovery sense (Xie et al., 3 Mar 2026). Similar caveats apply whenever skill discovery depends on hand-chosen factors, base controllers, symbolic planners, or archived demonstrations.

Open problems follow directly from these limitations. The literature repeatedly identifies dependence on resets, safety constraints, privileged state, and curated data collection as bottlenecks. Several papers note the absence of explicit skill segmentation, symbolic composition, or persistent online consolidation into long-term memory (Mees et al., 2019, Shafiullah et al., 2022, Grillotti et al., 26 Aug 2025). Others expose scaling issues in high-dimensional observation spaces, feasibility estimation, or mechanically grounded retrieval (Fang et al., 2022, Xie et al., 3 Mar 2026). A plausible implication is that future URSA systems will need to integrate several currently separate threads: representation learning from raw perception, safe and sample-efficient online interaction, adaptive curricula over skill difficulty, structured memory for reuse, and mechanisms for turning newly executed behaviors into stable, reusable policies. The cited work establishes each of these ingredients in isolation or in partial combinations, but not yet as a single autonomous open-world system.