2000 character limit reached
PanoVine: Whole-Body Visuomotor Control for Soft Growing Vine Robot
Published 22 Jun 2026 in cs.RO | (2606.22923v2)
Abstract: Vine robots, a class of soft, growing robots, are suitable for navigating complex and confined environments due to their compliant bodies and self-supporting growth mechanism. However, hysteresis, tether interactions, and deformations make them difficult to predict and model, which in turn limits the effectiveness of conventional planning and control approaches. In this work, we present a data-driven, vision-based control framework for the first autonomous vine robot system. Our system integrates 19 cameras distributed along the robot's body to provide comprehensive feedback of both the robot state and the surrounding environment. Using this rich whole-body vision feedback, we train an end-to-end visuomotor policy from demonstrations for closed-loop autonomous control in complex environments. The policy efficiently aggregates information from distributed sensing while maintaining robustness to inaccurate robot states and actuation. Experimental results demonstrate that the learned policy enables robust navigation and manipulation in challenging scenarios, including steering through branched structures, climbing up slopes, traversing unsupported terrain, reaching objects precisely, and maneuvering through confined spaces and obstacles. Project website https://panovine-bot.github.io
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Explaining “Pano Vine: Whole-Body Visuomotor Control for Soft Growing Vine Robot”
1) What is this paper about?
This paper introduces a special soft robot called a “vine robot” that grows forward like a plant vine and can move through tight, twisty spaces. The new system, called PanoVine, gives the robot “eyes” all along its body using 19 small cameras and teaches it to control itself by learning from human demonstrations. It’s the first vine robot that can autonomously navigate using only its own onboard sensors.
2) What questions were the researchers trying to answer?
In simple terms, they wanted to know:
- How can a soft, growing robot safely and accurately move through complicated places (like pipes and tunnels) even when its movements are hard to predict?
- Can a robot “see” with cameras along its entire body and use that to guide its actions in real time?
- Is it possible to learn good control directly from examples, instead of relying on exact physics models that are hard to build for soft materials?
3) How did they do it?
Think of the robot like a long, flexible tube that “turns itself inside out” at the tip to grow forward. Along this tube, the team placed 19 tiny cameras—like having GoPro-style eyes spread across its body. The robot can also bend at several “joints” to steer.
To control the robot, they used a learning approach similar to how you’d learn to play a video game by watching and then trying it yourself:
- First, skilled operators drove the robot through different courses using a joystick, providing demonstrations.
- The robot recorded what its cameras saw and what actions the human took (how fast to grow, how much to bend at each joint).
- A computer program (a “visuomotor policy”) was trained to map “what the robot sees and feels” to “what it should do next.” This is called imitation learning.
Some helpful ideas they used:
- Whole-body vision: Instead of just one camera at the front, the robot looks around with many cameras along its body. This helps it know where its body is and what’s around it (like a 360° awareness spread out along its length).
- Combining views smartly: The computer learns how different camera views relate to each other and to the robot’s actions, even as the robot grows and bends. You can think of it like the robot “paying attention” to the most helpful angles at each moment.
- Focusing on changes: The robot’s internal signals (like joint angles and length) are treated in a way that focuses on “how much has changed since last time,” making the controller more stable when the robot’s exact position is uncertain.
- Balancing the training: During demonstrations, the robot spends lots of time just growing straight and only sometimes needs sharp steering. The team made sure the training paid enough attention to those crucial steering moments so the robot wouldn’t “forget” how to turn when it really matters.
4) What did they find and why does it matter?
They tested the robot in real-world challenges, not just in simulation:
- Complex course navigation: The robot had to choose the right branch, climb a slope, cross a gap without support, avoid obstacles, and make sharp turns, all over a 6-meter course. The learned controller succeeded 80% of the time. Simple approaches—like replaying the same old commands—failed completely, because soft robots don’t repeat motions exactly and obstacles can change.
- Object reaching: The robot had to travel about 2 meters and steer precisely to touch and knock down different objects in different places. It succeeded 85% of the time. A version using just one camera failed, showing why whole-body vision is important.
Why this matters:
- Soft, growing robots are great for squeezing into places rigid robots or drones can’t go (like narrow pipes or cluttered underground spaces).
- But soft robots are hard to predict; the same command can bend them differently each time. This work shows a way to handle that unpredictability by learning from vision and examples.
- It’s a step toward autonomous inspection and exploration in places that are otherwise dangerous, hard to reach, or too cramped for people or traditional robots.
5) What’s the bigger impact?
This research shows that giving robots “eyes” across their entire body and teaching them from human demonstrations can make soft robots much more capable and independent. In the future, this could:
- Improve inspection of pipes, tunnels, and industrial equipment without needing humans to enter risky areas.
- Help in disaster response or search-and-rescue by moving through debris-filled or collapsed spaces.
- Encourage new designs that combine vision with other senses (like touch and force), making robots even safer and smarter in complex environments.
Overall, PanoVine demonstrates a practical path for controlling soft, growing robots in the real world, pointing toward robots that can safely explore tight, unpredictable spaces on their own.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
Below is a single, consolidated list of what remains missing, uncertain, or unexplored in the paper, phrased to guide concrete follow-on research.
- Sensing modalities and coverage: Only small-FOV RGB cameras are used; the system lacks depth, wide-FOV, low-light illumination, contact/force, and tactile sensing. It remains unknown how performance changes with higher-FOV lenses, integrated lighting, depth or event cameras, and distributed tactile/force sensors, especially in dark, dusty, reflective, or occluded environments.
- Robustness to vision degradation: The policy’s tolerance to camera occlusion, lens contamination, motion blur, auto-exposure artifacts, dropped frames, or outright camera failures was not evaluated; strategies for redundancy, sensor weighting, or graceful degradation are missing.
- Implicit multi-view correspondences: Camera extrinsics are not calibrated and poses drift as the body deforms, yet there is no ablation on robustness to camera pose changes, shuffling/missing camera streams, or varying camera counts; it is unclear how the policy handles inconsistent or re-ordered views.
- Temporal reasoning and memory: The policy uses a very short observation history (To=2) and no explicit memory or mapping. It is unknown whether recurrent architectures, longer visual histories, or learned/explicit maps would improve long-horizon behaviors with occlusions or delayed consequences.
- Data efficiency and scaling: Tasks were learned from tens of demonstrations; no sample-efficiency analysis, scaling laws, or curriculum strategies are provided. The effectiveness of semi-/self-supervised data, DAgger-style corrections, or synthetic/simulated data augmentation remains unexplored.
- Generalization across embodiments: Experiments use a single robot morphology (length, diameter, number/spacing of joints/cameras). There is no analysis of how policies transfer across different lengths, diameters, joint layouts, payloads, or stiffness/pressure settings.
- Generalization across environments: Evaluation is in controlled lab courses. Robustness to real industrial settings (pipes, burrows, moisture, debris, variable friction, narrow apertures, vertical climbs, overhangs), and complex 3D branch networks with poor visibility, is untested.
- Control rate and latency: The policy runs at 5 Hz with 19 cameras and ViT inference; the impact of perception/control latency on accuracy and stability (especially at higher growth speeds or more dynamic maneuvers) is not quantified.
- Ablations of architectural choices: Only a limited ablation (rebalancing) is shown. The benefits of relative action/proprioception representations, cross-attention over multi-view tokens, and image augmentations (e.g., ColorJitter/dropout) are not isolated or quantified.
- Evaluation metrics and statistics: Success/failure is reported on small numbers of rollouts (n=10, n=20) without confidence intervals or detailed metrics (e.g., tip pose error, path deviation, clearance to obstacles, contact forces, cycle time, energy). More rigorous benchmarking is needed.
- Safety, constraints, and failure recovery: The policy lacks explicit safety constraints (pressure, joint limits, self-collisions) and automated recovery strategies (e.g., unstick maneuvers, backtracking). Failure modes are mentioned but not systematically analyzed or mitigated.
- Communication and compute constraints: The system relies on USB 2.0 over body cables and a base computer; scalability to longer robots (>5 m cable limits), EMI noise environments, bandwidth contention with more cameras, and feasibility of on-body inference are not addressed.
- Lighting and power for vision: The paper does not specify active illumination; robustness to low/no ambient light (typical of pipes or underground settings) and the power/cable budget for distributed lighting are open issues.
- Actuator and sensor failure tolerance: There is no study of robustness to encoder drift, pressure regulation errors, joint stiction/backlash, or partial actuation failures, nor of controller strategies for reconfiguration when segments malfunction.
- Mechanical durability and maintenance: Long-term effects of repeated eversion on fabric welds, cable wear, camera mounts, and lens fouling are unreported; maintenance schedules and protective design for harsh environments are undefined.
- Retraction and bidirectional operation: Autonomous retraction/backtracking, managing tail tension during withdrawal, and planning to safely disentangle from obstacles are not considered.
- Forceful interactions and manipulation: Demonstrations focus on navigation and light contact (knocking objects). Tasks requiring controlled contact forces (e.g., anchoring, pushing past obstructions, valve turning) and integration of force/tactile feedback remain open.
- Multi-task learning and hierarchy: Policies are trained per task; there is no exploration of a single policy handling multiple tasks, task switching, or integration with high-level planners for goal-conditioned control over long deployments.
- Sensor-placement and hardware co-design: With 19 fixed cameras, the optimal number, placement, and orientation of sensors and joints are not studied; automatic co-design to trade off coverage, bandwidth, weight, and robustness is only suggested, not performed.
- Domain shift and appearance variability: Although CLIP ViT is used, robustness to domain shifts (texture, lighting spectra, camera models), and the efficacy of domain adaptation or normalization in field conditions are not evaluated.
- Teleoperation bias and corrective learning: Imitation learning from teleop demos may encode human biases and error distributions; no corrective learning (e.g., DAgger), active learning, or online adaptation to covariate shift is reported.
- Lack of ground-truth state for benchmarking: There is no ground-truth pose/shape estimation of the body or tip during experiments, limiting the ability to quantify deformation, contact, and control accuracy, and to compare methods fairly.
- Energy and autonomy: Power budgets, energy consumption with many cameras and compute, and feasibility for battery-powered untethered missions are unaddressed.
- Field deployment logistics: Practical issues such as RF/GPS-denied communication strategies, tether routing around sharp edges, contamination control, and human safety protocols in industrial or ecological settings are not discussed.
Practical Applications
Immediate Applications
The following applications can be pursued with the current PanoVine system and its methods (6 m soft growing body, 19-camera whole-body vision, end-to-end diffusion policy trained from demonstrations) or with straightforward engineering adaptation.
- Industrial asset inspection in confined spaces (energy, utilities, manufacturing)
- Use case: Autonomous or supervised navigation inside large pipes, process ducts, pressure vessels, and cable trays to visually inspect corrosion, fouling, blockage, or damage where GPS is unavailable and rigid robots struggle.
- Tools/products/workflows: “PanoVine Inspection Kit” comprising the vine robot, on-body camera array, ROS-based teleop and autonomy modes, and a report generation pipeline from recorded multi-view footage. Operator workflow: short teleoperated demonstration in a representative segment, quick policy fine-tuning, supervised autonomous pass, post-run defect tagging.
- Dependencies/assumptions: Sufficient bore diameter (~≥0.18 m), dry environments, auxiliary illumination for RGB cameras, ability to stage base unit and compressed air/power, compliance with site safety policies (pressure, ignition sources, contamination limits).
- Sewer and storm drain reconnaissance (municipal utilities)
- Use case: Navigate branches, slopes, and unsupported spans to reach and document blockages or structural failures in culverts and large drains without relying on wheel traction.
- Tools/products/workflows: Drop-in “branch selection” and “gap traversal” policies; GIS-linked logging of multi-view imagery for maintenance tickets.
- Dependencies/assumptions: Flow control (low water), debris that won’t entangle the body, ruggedized camera housings; regulatory permissions for confined-space entry robotics.
- HVAC and industrial duct inspection and light maintenance (construction, facility management)
- Use case: Grow through duct networks, take multi-perspective imagery past bends and T-junctions, and precisely reach target elements (e.g., dampers, sensors) for status checks.
- Tools/products/workflows: Add-on end-effector cap (e.g., touch probe, swab) for “object reaching” behaviors demonstrated in the paper; semi-autonomous path-following based on whole-body vision.
- Dependencies/assumptions: Duct diameters ≥ robot body, dry dust loads (protect optics), site access and cleaning protocols.
- Cable pulling and sensor deployment in buildings and tunnels (construction, telecom)
- Use case: Use self-supporting growth to route pull-lines or fiber through conduits and false ceilings; exploit precise reaching to hook, place, or knock-in markers.
- Tools/products/workflows: Replace tip cap with line fastener; scripted grow-steer actions with human-in-the-loop corrections; post-run visual verification.
- Dependencies/assumptions: Bend radii compatible with joints, manageable tether drag, tip fixture integration.
- Robotic exploration of crawlspaces and attics (building diagnostics, pest control)
- Use case: Safely traverse fragile, unsupported structures (rafters, insulation gaps) and avoid obstacles to document leaks, wiring issues, or infestations.
- Tools/products/workflows: Portable base unit with battery and compact compressor; operator joystick for initial demos and autonomy fallback.
- Dependencies/assumptions: Environmental debris and dust management, lighting; careful body material choice to avoid contamination.
- Environmental and wildlife burrow studies at larger scales (field ecology)
- Use case: Non-destructive entry into large burrows, lava tubes, or root cavities to gather visual data while minimizing friction and disturbance.
- Tools/products/workflows: Low-noise pressure regulation, soft body sleeves for bio-safety, time-stamped multi-view datasets for habitat mapping.
- Dependencies/assumptions: Diameter and curvature constraints, permissions/ethics approvals, sterilizable coverings.
- Nuclear and hazardous facility reconnaissance-lite (energy, decommissioning)
- Use case: Short-range, line-of-sight reconnaissance in cluttered, GPS-denied areas to inform human decision-making while keeping operators remote.
- Tools/products/workflows: Shielded base station, single-mission disposable sleeves, “steer-to-sign” visual behaviors (like the paper’s “stop at exit”).
- Dependencies/assumptions: Radiation tolerance of electronics (limited), contamination control, strict safety certification.
- Research platform for soft robotics and multimodal policy learning (academia, R&D labs)
- Use case: Benchmarking imitation learning from demonstrations on deformable, long-horizon tasks; studying multi-view cross-attention without explicit extrinsic calibration; experimenting with relative action representations and dataset rebalancing for sparse steering.
- Tools/products/workflows: Open-source ROS stack, dataset formats (multi-camera + proprioception), training scripts for diffusion policies, ablation baselines.
- Dependencies/assumptions: Access to GPU training, reproducible hardware segments, curated demo collection.
- Method transfer to other continuum/snake robots (robotics, inspection)
- Use case: Retrofit existing snake/continuum robots with distributed micro-cameras and adopt the paper’s cross-attention, relative action, and rebalanced training strategies for robust closed-loop control.
- Tools/products/workflows: “Whole-Body Vision SDK” and “Teleop-to-Autonomy Trainer” package (camera drivers, CLIP-ViT encoders, diffusion policy).
- Dependencies/assumptions: Mechanical mounting for cameras, adequate compute and bandwidth, domain-specific demo data.
- Operator training and certification modules (workforce development, policy)
- Use case: Train operators to collect high-quality demonstrations and supervise autonomy in regulated infrastructure environments.
- Tools/products/workflows: Simulator-free curriculum using the paper’s joystick interface, scenario libraries (branched structures, slopes, gaps), checklists for safety and data quality.
- Dependencies/assumptions: Access to mock courses, organizational buy-in, alignment with safety protocols.
Long-Term Applications
These applications build on the paper’s findings but require further research, scaling, ruggedization, sensing additions, or regulatory maturation.
- Fully autonomous subterranean search-and-rescue in rubble and caves (public safety, disaster response)
- Use case: Multi-kilometer navigation through collapsed structures, reactive steering around shifting debris, locating victims with on-body vision plus added thermal/audio sensors.
- Tools/products/workflows: Robustized body materials, wide-FOV/low-light cameras, multimodal sensing (tactile/force/thermal), learned recovery behaviors and self-retraction.
- Dependencies/assumptions: Environmental robustness (dust, water), comms relays beyond USB limits, extensive field data for training, incident command integration.
- In-situ pipe maintenance and remediation (energy, water utilities)
- Use case: After inspection, apply coatings, inject sealants, cut roots, or place patches using precise reaching and manipulation inside pipes.
- Tools/products/workflows: Interchangeable tool-tips, force/tactile whole-body sensing, compliance control on soft bodies, closed-loop visual servo with manipulation primitives.
- Dependencies/assumptions: Added actuation and sensing at the tip, certification for material application, predictive models for contact-rich manipulation in compliant structures.
- Standardized “whole-body vision” architectures for deformable robots (robotics, standards)
- Use case: Sector-wide adoption of distributed camera arrays with self-calibrating cross-attention policies for soft and continuum robots (medical, industrial, service).
- Tools/products/workflows: Open standards for sensor nodes, time sync, compression, and learned calibration-free fusion; reference datasets and benchmarks.
- Dependencies/assumptions: Community consensus, vendor support, long-term maintenance of models and datasets.
- Medical continuum devices inspired by whole-body vision (healthcare)
- Use case: Endoluminal tools (e.g., GI scopes, bronchoscopes) that use distributed micro-cameras along the shaft to maintain situational awareness and robust steering without precise kinematic models.
- Tools/products/workflows: Sterilizable micro-imagers, compliance with medical device regulations, domain-specific imitation learning from expert procedures.
- Dependencies/assumptions: Miniaturization, biocompatibility, regulatory approvals, clinical trials, rigorous fail-safe control.
- Planetary exploration of lava tubes and subsurface voids (space)
- Use case: Growth-based locomotion for low-friction traversal of tortuous subterranean environments on the Moon/Mars, mapping with whole-body vision and sparse tactile sensing.
- Tools/products/workflows: Radiation-hardened electronics, low-temperature materials, autonomy with extreme comms delay, hybrid energy/pressure systems.
- Dependencies/assumptions: Space qualification, mission integration, environmental survivability.
- Multi-robot “vine swarms” for rapid survey and mapping (construction, mining, security)
- Use case: Parallel deployment of multiple vine robots to different branches, sharing learned policies and maps for fast coverage.
- Tools/products/workflows: Fleet management, SLAM variants using distributed ego-views, inter-robot communication in GPS-denied environments.
- Dependencies/assumptions: Robust multi-robot comms, cross-robot policy generalization, conflict resolution in tight spaces.
- Co-design and auto-optimization of robot morphology and sensing (robotics, manufacturing)
- Use case: Automated selection of segment lengths, joint spacing, camera placement, and sensor mix (vision + tactile) for target tasks, as suggested by the paper’s future work.
- Tools/products/workflows: Differentiable design tools, task-driven objective functions, large-scale sim-to-real pipelines.
- Dependencies/assumptions: Accurate differentiable models for soft growth and contact, scalable optimization, real-world validation loops.
- Underwater and harsh-environment variants (offshore energy, aquaculture)
- Use case: Navigate flooded conduits or submerged structures using water-compatible materials and cameras, leveraging growth-based motion to avoid slippage.
- Tools/products/workflows: Waterproof everting skins, pressure-compensated camera pods, sonar/optical fusion with cross-attention.
- Dependencies/assumptions: Significant materials and sealing R&D, corrosion resistance, biofouling management.
- Data-driven infrastructure assessment and policy support (policy, city planning)
- Use case: Aggregate multi-view inspection data into risk scores and maintenance prioritization tools; inform standards for soft-robot inspections and record-keeping.
- Tools/products/workflows: Centralized data lakes, defect detection models trained on PanoVine imagery, audit trails and compliance reporting.
- Dependencies/assumptions: Data governance frameworks, interoperability with asset management systems, accepted inspection protocols.
- Entertainment, cinematography, and interactive exhibits (media, education)
- Use case: Safe, smooth camera motion through tight sets or exhibits; interactive “growing” robots in museums demonstrating soft robotics concepts.
- Tools/products/workflows: Quiet actuation, stabilized camera heads, operator-in-the-loop autonomy with safety interlocks.
- Dependencies/assumptions: Noise reduction, audience safety measures, aesthetic housing and lighting.
Notes on cross-cutting assumptions and constraints:
- Current system relies on RGB cameras with limited FOV and lighting; adding wide-FOV optics, IR/low-light, and tactile/force sensing will broaden applicability.
- Bandwidth and tethering: USB 2.0 distance constraints and tether management limit maximal reach; solutions include on-body compute, compression, and fiber/ethernet spines.
- Generalization: Policies trained from tens of demos per task may require on-site fine-tuning for new environments; robust domain adaptation and data scaling will help.
- Safety and compliance: Pressurized systems, contamination control, and failure modes must meet sector-specific regulations (industrial, medical, nuclear).
- Durability: Dust, moisture, chemicals, and abrasion necessitate ruggedization of fabrics, seals, and optics.
Glossary
- AdamW: An optimizer that decouples weight decay from gradient-based updates to improve generalization. "We use AdamW [43] with a cosine learning rate schedule (2000 warmup steps) starting at a learning rate of 3x10-4 for the diffusion model and 3x10-5 for finetuning the vision backbone, weight decay 1×10-6, and betas (0.95,0.999)."
- Buckling: Structural instability where a slender body suddenly deforms under compressive load. "because base buckling reduces the deployed length below what the replayed growth commands assume."
- Class token (CLS token): A special aggregate token in Vision Transformers used to represent an entire image. "we take the predicted class token for each image as a compact per-view embedding."
- CLIP: A vision-LLM used here as a pretrained visual encoder to embed images. "feed them into a pretrained CLIP Vision Transformers model [37, 38, 39]"
- Continuum robot: A robot with a continuously deformable body rather than discrete rigid links and joints. "Vine robots, a class of pneumatic soft growing continuum robots, are well suited to these scenarios [4, 5]."
- Cross-attention: An attention mechanism that conditions one sequence (e.g., actions) on another (e.g., observations). "This cross-attention learns task-specific semantic correspondences between multi-modal observations (multi-view images and proprioception) and robot actions."
- DDIM scheduler: A deterministic inference procedure for denoising diffusion models that accelerates sampling. "with a DDIM scheduler [42] using a squared-cosine ß schedule."
- Degrees of freedom (DoF): Independent controllable parameters of a system’s configuration. "The robot has seven controllable degrees of freedom, one from extension at the tip, and six from revolute joints."
- Diffusion Policy: A visuomotor policy class that models action distributions via denoising diffusion. "We use Diffusion Policy with a Diffusion Transformer backbone [36]."
- Diffusion Transformer (DiT): A transformer-based backbone for diffusion models used to predict actions. "The DiT uses embedding dimension 768, depth 7, 8 heads, and attention dropout 0.1."
- Eversion: Growth-by-turning-inside-out of a soft tube to extend from the tip under internal pressure. "They grow in length and extend at the tip by using internal fluid pressure to evert body material supplied from a fixed base."
- Exponential moving average (EMA): A smoothed average of model parameters to stabilize inference/training. "We maintain an exponential moving average (EMA) of the weights (power 0.75, max decay 0.9999)."
- Finite-element discretization: Numerical method that approximates continuous deformable bodies with discrete elements. "their reliance on finite-element discretization limits scalability to long, continuously growing robots."
- Hysteresis: Path-dependent behavior where outputs depend on history, common in soft materials and actuators. "The robot behavior is shaped by material compliance, hysteresis, buckling, body-environment interaction, and tension in the un-everted tail"
- IMU: Inertial Measurement Unit; a sensor that measures acceleration and angular rate for estimating motion. "limited onboard sensing (e.g., IMU, single camera) insufficient for reliable control [6, 7]."
- Jacobian-based adaptive control: Control using the Jacobian mapping between joint velocities and task-space motion, adapted online. "and Watson et al. introduced a Jacobian-based adaptive control framework to enable deployment in contact-rich environments."
- Kinematic model: A model describing motion without forces, relating joint configurations to positions and orientations. "Model-based methods typically rely on kinematic models, such as the piecewise constant curvature assumption [8, 9, 10, 11], to plan tip trajectories"
- Koopman-based approach: Learning/control method leveraging the Koopman operator to linearize nonlinear dynamics in lifted space. "Haggerty et al. use a Koopman-based approach to learn an explicit dynamical model that enables model-based control; however, it relies on carefully chosen state representations and does not scale well to long, multi-segment systems."
- Model predictive control (MPC): Optimization-based control that plans actions by solving a finite-horizon problem at each step. "Kalibala et al. proposed a learning-based model predictive control framework trained on simulated trajectories, but this was also unverified in the real world."
- Odometer: A device measuring distance traveled; here used to estimate deployed robot length. "as well as between the robot odometer and deployed robot length, using calibrated mapping functions."
- Piecewise constant curvature: A common continuum-robot assumption that segments bend with constant curvature. "piecewise constant curvature assumption [8, 9, 10, 11]"
- Proprioception: Internal sensing of a robot’s own state, such as joint angles and extension length. "q denotes proprioception, comprising the joint angles and a base length signal derived from the growth spool encoder."
- PWM (pulse-width modulation): A method to control motor power by varying the duty cycle of voltage pulses. "specifying pulse-width modulation (PWM) duty cycle and pulse duration."
- Revolute joint: A hinge-like joint allowing rotation about a single axis. "Placement of the 6 revolute joints and 19 RGB cameras distributed across the 7 segments of a 6 m long, 0.5 m diameter robot."
- RS-485: A differential serial communication standard used for robust multi-drop networking. "Full-duplex RS-485 communication is employed to enable bidirectional data exchange."
- SLAM: Simultaneous Localization and Mapping; building a map while estimating the robot’s pose. "However, these methods often depend on prior knowledge of obstacle geometry or onboard sensing for real-time SLAM"
- Teleoperation: Remote human control of a robot. "the operator teleoperates the multi-segment vine robot using a joystick (Logitech G F710)."
- Token-passing protocol: A communication scheme where a token grants permission to transmit on a shared bus. "a circular token-passing protocol is implemented, allowing only one local MCU to transmit at any given time."
- Visual servoing: Feedback control that uses visual features to drive robot motion toward a goal. "Greer et al. combined a kinematic model with visual servoing to regulate steering based on image-space error,"
- Vision foundation model: A large pretrained vision model used as a general-purpose feature extractor. "The 19 RGB images are represented by the class token of a vision foundation model."
- Vision Transformer (ViT): A transformer architecture for images that treats patches as tokens. "The observation encoder finetunes a CLIP-pretrained ViT-B/16 backbone 37 shared across all camera streams"
- Visuomotor policy: A policy that maps visual (and other) observations directly to motor commands. "we learn a visuomotor policy from demonstrations."
Collections
Sign up for free to add this paper to one or more collections.
