LeHome: A Simulation Environment for Deformable Object Manipulation in Household Scenarios

Abstract: Household environments present one of the most common, impactful yet challenging application domains for robotics. Within household scenarios, manipulating deformable objects is particularly difficult, both in simulation and real-world execution, due to varied categories and shapes, complex dynamics, and diverse material properties, as well as the lack of reliable deformable-object support in existing simulations. We introduce LeHome, a comprehensive simulation environment designed for deformable object manipulation in household scenarios. LeHome covers a wide spectrum of deformable objects, such as garments and food items, offering high-fidelity dynamics and realistic interactions that existing simulators struggle to simulate accurately. Moreover, LeHome supports multiple robotic embodiments and emphasizes low-cost robots as a core focus, enabling end-to-end evaluation of household tasks on resource-constrained hardware. By bridging the gap between realistic deformable object simulation and practical robotic platforms, LeHome provides a scalable testbed for advancing household robotics. Webpage: https://lehome-web.github.io/ .

• The paper introduces LeHome, a simulation environment that models household deformable object manipulation with realistic assets and causal action graphs.
• It integrates diverse physics engines such as PBD, FEM, and Eulerian Fluid Simulation to accurately replicate the dynamics of liquids, garments, and other non-rigid materials.
• Empirical results show enhanced sim-to-real transfer, with improved task performance in scenarios like garment folding and food cutting through combined simulation and teleoperation.

LeHome: A Simulation Environment for Deformable Object Manipulation in Household Scenarios

Motivation and Context

The paper introduces LeHome, a simulation environment specifically designed to model, evaluate, and benchmark deformable object manipulation tasks in household scenarios (2604.22363). Manipulation of deformable objects such as garments, food items, liquids, and other non-rigid materials is a significant challenge in both simulated and real-world contexts due to the complexities of their material properties, manifold shapes, and nonlinear dynamics. Existing simulation platforms largely focus on rigid or articulated objects and offer limited support for high-fidelity modeling or systematic evaluation of deformable objects in domestic settings. LeHome addresses this gap by providing an integrated suite of assets, simulation engines, and benchmarking tasks tailored to the requirements of household robotics.

System Architecture and Assets

LeHome is structured into three principal components: Assets, Engine, and Benchmark. LeHome Assets deliver realistic simulations encompassing a wide spectrum of robots, rigid and articulated objects, and particularly, high-fidelity deformable objects. Multiple household scenarios are represented, facilitating task diversity and physical realism. The Engine leverages Position-Based Dynamics (PBD), Finite Element Method (FEM), and Eulerian Fluid Simulation for category-specific physical modeling, enabling authentic interactive behavior for both soft and hard objects. The Benchmark component defines representative manipulation tasks, integrates domain randomization, and supports teleoperation data collection, providing a robust platform for policy evaluation and sim-to-real transfer studies.

Figure 1: LeHome system overview: assets for realistic simulation, engine with diverse physics, and benchmark for task construction and evaluation.

Figure 2: LeHome delivers visually and physically high-fidelity simulated assets across six deformable categories.

Modeling Deformable Objects

Deformable objects within LeHome are categorized into six classes based on their mechanical characteristics:

• Liquid: Modeled with PBD for efficient flow simulation and frequent contact dynamics.
• Gaseous Fluid: Simulated via Omniverse Flow using sparse voxel grids for realistic flame and gas behaviors.
• Granular Objects: Fine granules modeled with PBD; coarse aggregates represented as rigid bodies.
• Linear Objects: Supported by multi-rigid-body chain (for efficiency) and FEM-based approaches (for bending/stretching fidelity).
• Thin Shell: Wrinkling garments use PBD; posters and similar assets employ FEM for elastic deformation.
• Volumetric Objects: FEM volumetric discretization for elastic/elastoplastic modeling, enabling intricate manipulation (e.g., food items).

This multi-method approach ensures coverage of varied manipulation tasks, realistic state transitions, and supports challenging actions such as garment folding or food cutting.

Manipulation Mechanisms and Action Graph

A pronounced innovation is the Action Graph framework, which models manipulation tasks through modular event-driven nodes—On-Trigger, Computation, and State Update nodes. These nodes encode causal relationships between actions and resulting object states, enabling high-fidelity simulation of processes such as food cutting, splitting, or reconfiguration. The Action Graph ensures simulation results abide by real-world cause-effect dynamics, with explicit modularity, controllability, and extensibility. This mechanism allows simulation of complex interactions that involve both physical and logical transitions, supporting task generalization.

Figure 3: Action Graphs enforce causal consistency, enabling high-fidelity interaction modeling for manipulation tasks.

Figure 4: Action Graph workflow for sausage cutting, with modular nodes for event-triggering, mesh segmentation, and state update.

Robotic Embodiments and Teleoperation

LeHome integrates a range of open-source and commercial robotic arms, with particular emphasis on low-cost robots such as LeRobot, XLeRobot, and LeKiwi. These platforms are more apt for broad household deployment owing to their affordability, compact size, and ease of maintenance. Teleoperation modalities include keyboard, joystick, and leader-follower control, supporting flexible demonstration collection and facilitating scalable real-world algorithm validation.

Figure 5: Teleoperation with LeRobot family via joystick/keyboard and leader-follower systems.

Domain Randomization and Data Augmentation

To ameliorate sim-to-real transfer challenges, LeHome incorporates domain randomization across initialization, lighting, background textures, and material properties. Recorded trajectories can be replayed under randomized settings, preserving task geometry and contact while augmenting the dataset’s visual diversity. Only successful trajectories, filtered by task-specific detectors, are used for training, thereby improving robustness and reducing annotation cost.

Benchmark Tasks and Experimental Results

Six representative household tasks were validated: Fold Garment, Fling Garment, Assemble Burger, Cut Sausage, Pour Coffee, and Wipe Surface. These tasks span single-arm and bimanual manipulation, rigid/deformable interactions, and tool utilization, in realistic environments (bedroom, kitchen, living room, bathroom).

Key results:

• Simulation: SmolVLA achieved highest success on garment manipulation (Fold Garment: 70%), DP excelled at Cut Sausage (93%), ACT was strongest on Assemble Burger and Pour Coffee (78%, 80%). Fling Garment remained challenging for all methods.
• Real-world Transfer: Policies trained with combined LeHome simulation and real-world data ("Sim+Real Co-Training") realized substantial gains in low-data settings. For example, Fold Garment success improved from 2/10 (real only) to 5/10 (Sim+Real for ACT) and 2/10 to 4/10 (SmolVLA).

Figure 6: Garment folding task within LeHome.

Figure 7: Real-world execution across tasks using dual-arm LeRobot, validating sim-to-real fidelity.

Implications and Future Directions

LeHome establishes a comprehensive foundation for deformable object manipulation research in household settings. By providing category-specific modeling, high-fidelity physical assets, causal action graphs, and support for low-cost robots, LeHome enables benchmarking and policy training under realistic conditions. The simulation’s physical fidelity supports effective sim-to-real transfer, as evidenced by empirical results.

Practically, LeHome enables scalable evaluation and deployment of robotic policies for everyday manipulation tasks, lowering hardware, annotation, and interaction barriers. Theoretically, the Action Graph paradigm and systematic physical categorization facilitate generalizable algorithm design and reproducibility in manipulation research. Given the persistent difficulty of large-deformation tasks and complex nonlinear dynamics, LeHome presents avenues for integrating advanced model-based RL, differentiable simulators, and vision-language-action models to further improve generalization and robustness.

LeHome’s modularity, extensibility, and high-fidelity simulation capacities suggest it will serve as a persistent benchmark for deformable object manipulation and sim-to-real research. Extensions toward multi-modal policy learning, adversarial domain randomization, and real-world federated evaluation could further augment its impact.

Conclusion

LeHome advances the state-of-the-art in simulation environments for deformable object manipulation in household scenarios. By providing diverse, category-specific assets, robust simulation engines, causal action modeling, and integrated low-cost robotic embodiments, LeHome enables rigorous evaluation of manipulation policies, robust sim-to-real transfer, and practical adoption in real-world settings. Its methodological innovations and empirical results underline its utility as a foundation for advancing embodied intelligence in domestic robotics.