SimWeaver-Syn: Deterministic Trajectory Synthesis

• SimWeaver-Syn is a deterministic, topology-aware trajectory synthesis module that automates high-throughput, simulation-based demonstrations for deformable-object manipulation.
• The module eliminates teleoperation and post-hoc filtering by leveraging explicit mesh landmarks and geometric predicates for real-time grasp selection and trajectory generation.
• Empirical results demonstrate high success rates, efficient sample production, and robust sim-to-real transfer across diverse tasks like T-shirt folding and cloth manipulation.

SimWeaver-Syn is a deterministic, topology-aware trajectory synthesis module within the SimWeaver pipeline, designed to generate high-quality demonstrations for deformable-object manipulation tasks entirely in simulation. It operates without the use of teleoperation, learned generative models, or post-hoc filtering, producing directly usable, closed-loop-verified demonstrations from a single seed and a canonical mesh. The module targets efficient, generalizable, and replayable demonstration generation for a range of objects, including cloths, structured garments, and bags, addressing key challenges in sim-to-real transfer for deformable manipulation (Hu et al., 13 Jun 2026).

1. Design Goals and Motivations

SimWeaver-Syn is developed with the following primary objectives:

• Zero-teleoperation, zero-filtering: All demonstrations are synthesized algorithmically, dispensing with human-in-the-loop teleoperation or after-the-fact discriminator-based filtering. Input is restricted to a labeled canonical mesh and a single random seed.
• Topology awareness: Deformable objects lack fixed frames; thus, semantic manipulation must be encoded through mesh topology. All grasps and motion planning are defined in terms of relations between canonical mesh landmarks rather than discrete keypoints.
• Determinism: Both grasp selection and trajectory synthesis are purely deterministic functions of the simulator state and mesh labeling. This guarantees consistent, high-throughput data generation and reliable sim-to-real policy transfer.
• Generality: The same synthesis protocol applies to all supported deformable assets—including rectangular cloths, garments, and plastic bags—differing only in a one-time annotation of mesh landmarks, with no per-task calibration.

2. Inputs and Outputs

Inputs required by SimWeaver-Syn:

• Canonical mesh $M$ annotated with a set of semantic landmarks $V$ (e.g., corners for cloths, sleeve tips for garments).
• Topology adjacency graph $G = (V, E)$, encoding valid bimanual grasp pairs per asset. For regular shapes, $G$ may be autogenerated; for others, labeled per asset.
• Real-time observation $\mathrm{obs}_t = \{p_v \mid v \in V\}$ from SimWeaver-Sim, where $p_v \in \mathbb{R}^3$ are 3D positions.
• Task specification (goal predicate and stage definitions per manipulation task).

Outputs:

• A deterministic, closed-loop-verified trajectory $\tau$, structured as a sequence of joint-space actions (e.g., $q(0 \dots T)$, gripper signals), serialized in 50-step segments at 25 Hz for use in offline policy training.

3. Algorithmic Pipeline

SimWeaver-Syn employs a closed-loop pipeline interleaving three deterministic components each episode:

a) Topology-Adjacency Selection (tas):

All semantically valid landmark pairs for bimanual grasps form a graph $G = (V, E)$. At each planning cycle, a feasibility set $F(\mathrm{obs}_t) \subseteq E$ is computed using closed-form geometric predicates, ensuring each candidate grasp pair is physically realizable. Each feasible edge is scored by a task-specific function $V$0. The selected grasp pair is:

$V$1

b) Topology-Aware Feasibility Predicates:

For $V$2, the following are enforced:

1. Reachability: $V$3
2. Anti-cross-arm: $V$4
3. Safety margin: $V$5
4. Surface exposure: For the convex hull $V$6, $V$7
5. Occlusion-above:

$V$8

1. Layer separation: $V$9

Where $G = (V, E)$0 is the $G = (V, E)$1-radius geodesic mesh neighborhood, $G = (V, E)$2 the local XY-neighborhood, and $G = (V, E)$3 the canonical coordinate.

The final feasibility set:

$G = (V, E)$4

c) Deterministic Motion Planning and Closed-Loop Verification:

For each attempt:

1. Observe $G = (V, E)$5
2. Select optimal grasp pair $G = (V, E)$6
3. Synthesize a motion trajectory $G = (V, E)$7 to the grasp and execute post-grasp skill (e.g., fold, lift) using planners such as TOPP-RA and Pinocchio/cuRobo for inverse kinematics. The time-optimal trajectory objective:

$G = (V, E)$8

1. Verify grasp and goal predicate post-execution.
2. If unsuccessful, anchor already-verified grasp and retry with updated $G = (V, E)$9.

All decision points and trajectory synthesis steps are deterministic given inputs, supporting replay and robust correction of failed or ambiguous episodes.

4. Interface and Integration in the SimWeaver Pipeline

SimWeaver-Syn interacts as follows with the other SimWeaver modules:

Module	Role	Data Exchanged
SimWeaver-Asset	Supplies asset mesh and semantic labeling	Canonical mesh $G$0, landmarks $G$1, adjacency $G$2
SimWeaver-Sim	Simulation, observation, reset, execution	$G$3, low-level execution, image/RGB, control
SimWeaver-Real	Downstream data augmentation and learning	Demonstration trajectories $G$4

Demonstrations synthesized by SimWeaver-Syn are serialized in LeRobot format, consumed by SimWeaver-Real for ISP-aware photometric augmentation, domain randomization, and VLA policy training. The workflow is strictly unidirectional: Asset $G$5 Sim $G$6 Syn $G$7 Real $G$8 Deployment, with sim-level closed-loop checks at each stage.

5. Empirical Performance and Ablation

Trajectory Synthesis Quality:

• For "T-shirt flatten" ($G$9): pass rate $\mathrm{obs}_t = \{p_v \mid v \in V\}$0, with $\mathrm{obs}_t = \{p_v \mid v \in V\}$1 replay determinism; $\mathrm{obs}_t = \{p_v \mid v \in V\}$2 lift failure, $\mathrm{obs}_t = \{p_v \mid v \in V\}$3 stall failure.
• Ablations:
  • Without closed-loop retry: pass reduces to $\mathrm{obs}_t = \{p_v \mid v \in V\}$4 (all additional failures are lift-failures).
  • Without topology-adjacency selection: further reduction to $\mathrm{obs}_t = \{p_v \mid v \in V\}$5 pass.
• Cross-method comparison ("T-shirt fold", $\mathrm{obs}_t = \{p_v \mid v \in V\}$6): SimWeaver-Syn $\mathrm{obs}_t = \{p_v \mid v \in V\}$7 pass, $\mathrm{obs}_t = \{p_v \mid v \in V\}$8 replay; SIM1 pipeline $\mathrm{obs}_t = \{p_v \mid v \in V\}$9 pass, only $p_v \in \mathbb{R}^3$0 replay, $p_v \in \mathbb{R}^3$1 stall, $p_v \in \mathbb{R}^3$2 lift failure.

Real-world zero-shot deployment (five tasks, $p_v \in \mathbb{R}^3$3 trials/task):

• Mean per-task success: $p_v \in \mathbb{R}^3$4 ($p_v \in \mathbb{R}^3$5 per task) using $p_v \in \mathbb{R}^3$6 demonstration trajectories/task synthesized with SimWeaver-Syn.

Sample-efficiency and robustness (silk grasping):

• In-distribution: Sim-trained policy achieves $p_v \in \mathbb{R}^3$7 success at $p_v \in \mathbb{R}^3$8 demonstrations (vs. $p_v \in \mathbb{R}^3$9 for real-data baselines).
• Under substantial visual distribution shifts (texture, lighting, rotation): sim-to-real policy retains $\tau$0 success, while real-trained baselines drop to $\tau$1.

Cost efficiency:

• $\tau$2 usable trajectories/day on $\tau$3 RTX 4090, compared to $\tau$4 for SIM1, $\tau$5 for real-robot.
• Unit cost: \$0.03 per trajectory (1.1\% of real-robot cost, 37\% of SIM1 cost).

6. Significance and Implications

SimWeaver-Syn’s deterministic, topology-aware, closed-loop trajectory generation algorithm enables scalable and efficient data synthesis for deformable manipulation tasks, sidestepping prior dependencies on teleoperation and costly post-processing. The methodology achieves fully replayable, high-yield, and cost-effective demonstration collection, supporting reliable zero-shot sim-to-real transfer without per-task calibration. Empirical results demonstrate substantial improvements in success rates, replayability, sample efficiency, and robustness to visual domain shifts over prior methods (Hu et al., 13 Jun 2026).

A plausible implication is that deterministic, algorithmic demonstration synthesis using explicit topology and geometric predicates—rather than imitation or generative approaches—may become a standard paradigm for sim-to-real learning pipelines in deformable object manipulation, particularly where real-world data and extensive teleoperation are impractical.