Learning to Fold: prizewinning solution at LeHome Challenge 2026 (1st place online, 2nd offline)

Abstract: I describe my solution to the LeHome Challenge 2026, an ICRA 2026 competition on bimanual garment folding. The system placed 1st of 62 teams in the online (simulation) round and 2nd in the real-world final. It improves a vision-language-action (VLA) policy with a reinforcement-learning loop. The policy is its own value function: the same network that predicts actions also predicts success, progress, and a few task-relevant future quantities, and those predictions drive advantage estimation, live failure detection, and candidate selection. The work mostly recombines existing RL ideas with engineering and optimization contributions that can be used together as one recipe or individually: AWR + RECAP combined for flow-matching VLA; an asynchronous distributed training / rollout pipeline through HuggingFace Hub; inference-time hyperparameters optimization via Thompson sampling; a sim-to-real recipe with camera-alignment tooling, heavy augmentation and DAgger-like HIL data collection.

• The paper introduces a modular, asynchronous RL pipeline using VLA models for bimanual garment folding with dense reward shaping.
• It leverages an AWR + RECAP methodology, specific token conditioning, and hyperparameter bandit optimization to enhance data efficiency and precision.
• The system achieved a 79.63% success rate in simulation and demonstrated strong sim-to-real performance on challenging hardware environments.

Learning to Fold: Technical Summary and Analysis of the LeHome Challenge 2026 Solution

Problem Setting and System Overview

The paper "Learning to Fold: prizewinning solution at LeHome Challenge 2026 (1st place online, 2nd offline)" (2606.27163) describes an end-to-end system for bimanual garment folding in the context of the LeHome Challenge 2026. The competition consists of deformable-object manipulation with a focus on four garment types—long-sleeved tops, short-sleeved tops, long pants, and shorts—using a dual-arm, 12-DOF robot under visual observation from three RGB cameras (overhead, and left/right wrists). The system targets both simulation and real-robot environments, with the latter imposing domain transfer challenges.

Figure 1: The four garment types, seen from the overhead camera in the original simulation behavior-cloning dataset released by the organizers.

The paper details a modular, asynchronous training pipeline for vision-language-action (VLA) models, leveraging reinforcement learning (RL), advantage-based prioritized sampling, conditional inference, and sim-to-real transfer protocols. The solution is highly engineering-driven, with practical considerations for data efficiency, sources of robustness, and real-time performance.

Asynchronous RL Pipeline and Flywheel Design

The architecture implements an asynchronous "flywheel" pattern decoupling the training worker, rollout workers (across distributed simulator instances), and a manual DAgger correction station. These components synchronize via the HuggingFace Hub, enabling continuous data collection, training, and human-in-the-loop corrections without bottlenecks.

Figure 2: System overview: the asynchronous training / rollout / DAgger flywheel. The three components share state only through the HuggingFace Hub.

The design’s key characteristics are:

• Training Worker: Continuously trains on accumulated rollouts, recomputes (generalized) advantages, and checkpoints weights.
• Rollout Workers: Collect episodes with the latest checkpoint, maximizing sim usage and avoiding synchronization stalls.
• DAgger Station: Asynchronously corrects failure/near-miss rollouts, further increasing robustness to off-nominal states.

This system maximizes simulation efficiency—a crucial constraint given ~30s per episode in Isaac Sim—and ensures continual policy improvement and dataset expansion.

RL Methodology: AWR + RECAP for VLA

The approach synthesizes Advantage Weighted Regression (AWR) and RECAP-style advantage conditioning, which are better suited for flow-matching VLA models than log-probability-based policy gradients (e.g., PPO).

• AWR Implementation: Policy update is driven by prioritized sampling proportional to exponentiated/clipped advantage, not by loss weighting, which improves data and compute efficiency:

$$P(\text{sample frame } i) \propto e^{\mathrm{clip}(A_i,\,-2,\,2)}$$

• RECAP Conditioning: Advantage is injected as a conditional input to the model, enabling classifier-free guidance during inference. This “best-of-both-worlds” design achieves refinement both via reweighting and targeted conditioning.

The AWR+RECAP combination provides favorable exploration-exploitation trade-offs for high-precision, low-exploration tasks such as garment folding.

Figure 3: AWR + RECAP toy illustration.

Model and Policy Architecture

The policy builds on a frozen SigLIP visual encoder, followed by a Gemma-2B prefix transformer, which encodes images, robot state, garment-type token, advantage token, and auxiliary signals. A lightweight Gemma-300M flow-matching action expert generates 30-step action chunks.

• Token Layout and Attention: The hierarchical attention mask enforces access patterns, e.g., auxiliary heads only receive image tokens (to avoid state-overfitting).
• Advantage and Garment-Type Conditioning: Special tokens and AdaRMS conditioning provide per-layer modulation by garment type and advantage signal.
• Exclusive Self-Attention (XSA): Prevents trivial self-copying in attention, allegedly improving generalization and loss metrics.
• Auxiliary Heads: Inline prediction of success probability, task completion, keypoint distances, and future/pseudo-Q values.

  Figure 4: Policy architecture. A frozen SigLIP encoder feeds the Gemma-2B prefix (images, current query, state, garment type, advantage, FAST tokens, FAST query); the Gemma-300M flow-matching action expert reads the cleaned prefix and emits a 30-step action chunk. The prediction heads (orange) read from the prefix; the FM query at the tail of the action expert carries the action-conditional future predictions.

  

  Figure 5: Attention mask.

Reward Engineering and Advantage Estimation

Reward densification centers on decomposing the binary episode success into per-garment geometric checkpoints and smooth intermediate signals using critical keypoint distances. The reward is further regularized by gradual allocation and withdrawal policies upon failure, ensuring a completely binary/sparse total return but dense temporal credit assignment.

• Success/Completion Heads: The same policy network predicts per-timestep success probability and completion ratio, providing the value function for RL rollout scoring and advantage estimation (success probabilities EMA-smoothed, debiased for endpoint underprediction).
• GAE and Blended Baselines: Generalized Advantage Estimation (GAE) is applied over both success and completion signals, then blended toward a “segment” outcome-only baseline as rollouts turn off-policy.
• Precision Boost: Additional advantage assigned to top-quantile rollouts by threshold margin incentivizes higher-quality folds.

  Figure 6: Dense reward on a successful long-sleeve-top episode: overhead frames at five milestones (top) and the cumulative reward (bottom). The first 0.5 is allocated gradually as the primary proximity distance closes --- the gradual first checkpoint --- and reaching full success brings the total to 1.0.

  

  Figure 7: AWR sampling weight as a function of advantage, $P \propto e^{\mathrm{clip}(A,-2,2)}$.

Inference-Time Optimization: Hyperparameter Bandit and Robustness

The system exposes multiple inference parameters: chunk length, execution/playback velocity, inpainting overlap, classifier-free guidance (CFG) scale, flow noise temperature, and best-of-N candidate selection. These are automatically tuned per garment type using Thompson sampling bandits, maximizing performance with minimal manual grid search.

• Candidate Selection & CFG: Multiple FM samples are scored via the action-conditional future head, with CFG scales notably landing high (7–9), indicating substantial gains from guidance.
• Chunk Overlap: Soft inpainting ensures continuity and consistency over chunked execution.
• Bandit Arm Convergence: The bandit systematically adapts control parameters to the evolving policy and data distribution.

  Figure 8: Thompson-sampling posteriors for short pants.

Data Collection, Rollout Strategies, and Simulation Augmentation

Data diversity and quality are enforced through multiple rollout strategies: uniform random sampling, curriculum (difficulty-targeted), success/failure-state replay, and human-in-the-loop correction via DAgger. Aggressive environment augmentations—spanning textures, lighting, geometric jitter, and physical parameters—are central, especially for sim-to-real generalization.

• Replay and Hard Mining: Key to multiplying useful data for rare garment states or recovering from failures.
• Per-Source Decay: Ensures on-policy data dominates, while old/off-distribution rollouts are progressively de-emphasized.

  Figure 9: A mid-episode frame from a rollout debug video. The three camera views combined. The overlay shows the per-frame success probability (S), advantage (A), reward (R), completion (C), and time-to-completion (T), with their traces over the episode.

Sim-to-Real Transfer and DAgger-Based Recovery

For deployment to real hardware, the regimen combines camera/base calibration, per-bucket motion rescaling, and, critically, heavy per-frame and environment augmentation. A three-way data mixture—organizer BC, in-house DAgger, and replayed/augmented sim rollouts—enables partial domain randomization.

Figure 10: The four garment types in the organizer's real-robot BC dataset, seen from the overhead camera. These are the physical garments the real policy folds—the same four types shown in their original simulation form.

Figure 11: The three cameras (top, left wrist, right wrist) for each training bucket: the organizer BC data, my own teleop/DAgger recordings, and the sim success-replays.

Figure 12: The same idea across the sim/real boundary: an Isaac Sim render of the same action sequence overlaid on the real cameras, used to tune the sim's camera offsets against the real rig.

Figure 13: The same real camera frame under independent draws of the augmentation stack—color, gain, gamma, blur, noise, crop/rotate/zoom, and cutout. Every training frame is perturbed this way so the model cannot rely on any fixed appearance.

• DAgger Weighting: Recovery demonstrations are prioritized via per-frame sampling proportional to their proximity to interventions, exploiting human corrections for robustification.
• Motion Synchronization: Per-source speed rescaling aligns the temporal action distribution, addressing statistical divergence in trajectory speed/scale between sim and real data.

  Figure 14: Per-source speed factors.

  

  Figure 15: Per-frame sampling weight across an episode with two human interventions and a return to autonomous control. Human-correction frames get the highest weight, autonomous frames far from an intervention sit at the lower weight, and the 5-second windows just before each takeover ramp to zero.

Results and Empirical Findings

Online Simulation Round: The approach achieved a 79.63% overall success rate, with strong performance across all garment types and a notable gap (6.1%) above the nearest competitor. Per-type breakdowns confirm the policy’s broad generalization rather than over-specialization.

Sim-to-Real Round: In real-robot evaluation, the system scored 2nd place (865/1080), exhibiting substantial transfer robustness despite limited access to the actual evaluation hardware, multiple domain adaptation hurdles, and reliance on partially outdated checkpoints.

Discussion and Implications

Several methodological aspects were highlighted as having significant practical impact:

• Unified Policy-Value Representation: Efficient and performant for tasks where world modelling and value estimation must be consistent and share perception embeddings.
• Reward Engineering and Densification: Dense intermediate rewards, grounded in geometric task definitions, combined with policy-driven auxiliary signals, were critical for efficient RL.
• Inference Parameter Bandit Optimization: Online bandit-based tuning of control hyperparameters improved final system performance, offering a template for future deployment pipelines in RL-driven robotics.
• Heavy Augmentation and Domain Randomization: Extensively randomizing appearance and dynamics was necessary but not always sufficient for sim-to-real generalization. Per-trajectory motion rescaling and robust recovery data collection (via DAgger) were also required.

Outstanding Challenges:

• Automated exploration and recovery in RL for flow-matching VLA policies remain open, as best-of-N or CFG-based techniques tend only to reinforce known modes rather than generate novel corrective strategies.
• Overfitting to synthetic rendering and distributional artifacts underscores the need for better augmentation/meta-learning or explicit disentanglement architectures.

Future Directions:

• Joint real/sim reward and value estimation pipelines could merge the respective strengths observed in simulation and real-system phases.
• Lightweight model variants and policy distillation could address the observed over-parameterization relative to task complexity.

Conclusion

The "Learning to Fold" system demonstrates the feasibility and competitiveness of a VLA policy trained with a hybrid offline/online RL pipeline, asynchronous engineering, and modular sim-to-real transfer protocols in the context of bimanual garment manipulation. Its success is driven by an overview of prior RL methodologies, novel per-episode hyperparameter adaptation, dense reward shaping, and extensive data augmentation. The work provides a high-water mark for practical robotic RL system integration, with open directions for recovery policy learning and improved sim-to-real transfer.