TeleopLab: Smartphone Teleoperation for Remote Labs

• TeleopLab is a phone-based teleoperation platform that enables hands-on remote STEM education using intuitive smartphone controls.
• The system maps natural phone motion to robot end-effector movements via ARKit/ARCore, low-pass filtering, and virtual safety fences to reduce latency and cognitive load.
• A pilot study demonstrated a 46.1% reduction in cycle time, manageable workload (NASA TLX 38.2), and above-average usability (SUS 73.8), underscoring its practical impact.

TeleopLab is a phone-based teleoperation platform for remote STEM education that allows students to control a robotic arm and operate lab equipment through a system comprising a robotic arm, an adaptive gripper, cameras, lab equipment for a diverse range of applications, a user interface accessible through smartphones, and video call software. It is designed to address two persistent challenges in remote laboratories—cost and intuitiveness—by avoiding specialized input hardware and mapping natural phone motion to robotic manipulation. In a pilot study, the reported outcomes include a 46.1% reduction in task completion time as users gained familiarity with the system, a NASA TLX workload score of 38.2, and a SUS usability score of 73.8 (Chen et al., 6 Sep 2025).

1. Scope, purpose, and educational role

TeleopLab targets hands-on remote learning in settings where interaction with real-world equipment is pedagogically important but difficult to provide at scale. Its stated objective is to bridge the gap between physical labs and remote education with scalable infrastructure, while maintaining real robot interaction rather than reducing laboratory experience to simulation alone. The system is explicitly framed around remote STEM education, with representative activities including placing specimens on a tensile tester and triggering measurements (Chen et al., 6 Sep 2025).

The platform’s design rationale is rooted in accessibility. Rather than relying on VR sets, haptic devices, or other specialized interfaces, it uses smartphones on both iOS and Android, leveraging real-time pose estimation from camera frames and onboard IMU data. A “start teleop” button sets the current phone pose as the origin, after which subsequent motion is interpreted as relative pose until release. This relative-origin mechanism is presented as a way to reduce cognitive load by avoiding management of global coordinates during extended sequences (Chen et al., 6 Sep 2025).

TeleopLab is therefore not merely a teleoperation interface in the narrow sense. It is a complete remote laboratory arrangement in which manipulation, perception, equipment control, and classroom workflow are integrated into a single session. Students teleoperate the robot, observe camera feeds, and use remote desktop access to interact with laboratory instrumentation, all within a common session mediated by standard video call software (Chen et al., 6 Sep 2025).

2. System architecture and interfaces

On the server side, TeleopLab is built around ROS Noetic on Ubuntu 20.04 and supports multiple manipulators. The demonstrated configurations are an ABB IRB-120 and a UR5e collaborative robot. For ABB control, the system uses Externally Guided Motion over UDP with Google Protocol Buffers, chosen for higher update rates and lower latency than ABB’s ROS-Industrial RWS TCP service. End effectors include the 3D-printed InstaGrasp adaptive gripper and a Robotiq two-finger gripper (Chen et al., 6 Sep 2025).

The perception layer combines two cameras with distinct roles. An Intel RealSense D405 3D camera is mounted on the end effector to provide a close-up, task-centric view, while a Logitech C920x HD Pro Webcam on a tripod provides wide-angle environmental context. Both feeds are live-streamed using Cheese on Ubuntu to the host computer and shared through the remote desktop session, allowing students to switch views in Zoom. TeleopLab’s laboratory equipment integration is exemplified by a Mark-10 F305-EM tensile tester with PC Control, connected through serial communication to the host computer and exposed through a minimalist interface containing two buttons, Start Test and Reset Tester (Chen et al., 6 Sep 2025).

On the student side, the smartphone interface is implemented as an iOS app built on ARKit and an Android app built on ARCore. The apps auto-connect to the server via hard-coded IP/port, and distribution is handled through Apple TestFlight for iOS and emailed APKs for Android. Camera feeds and desktop controls are delivered through Zoom, which also enables monitoring and collaboration. The interface additionally includes mechanisms for connecting, restoring lost connections, and enabling the robot (Chen et al., 6 Sep 2025).

Safety is handled primarily through software-defined virtual fences around laboratory equipment. These fences constrain robot motion to avoid collisions and guide users during task execution, such as stopping motion when approaching the tensile tester beyond a limit. The described deployment also includes on-site staff intervention for preloading and unloading specimens and for resolving rare robot errors. No specialized physical enclosure is described (Chen et al., 6 Sep 2025).

3. Teleoperation model and control pipeline

TeleopLab uses end-effector pose control in a waypoint-based form. Phone motion generates desired end-effector waypoints $x_d$, which are transformed into the robot’s coordinate frame and then mapped to joint configurations $q_d$ by TracIK. The paper reports typical TracIK solve times of less than $1$ ms, and motion execution is performed through EGM or the robot controller at high update rates (Chen et al., 6 Sep 2025).

The mathematical description given for the system is solver-based rather than Jacobian-servo-based. Let $u(t)$ denote the relative $6$D phone pose change since the last “start teleop” event. The desired end-effector pose is computed as

$x_d(t) = T \, u(t)$

where $T$ encodes the alignment between phone coordinates and robot or world coordinates, including scale, axes, and handedness adjustments. The inverse kinematics stage is expressed as

$q_d(t) = \mathrm{IK}(x_d(t)).$

To suppress jitter, the system applies low-pass filtering to the desired pose stream:

$$x_d^{\text{filt}}(t) = \alpha \, x_d^{\text{filt}}(t-\Delta t) + (1-\alpha)\, x_d^{\text{raw}}(t),$$

with rate limits imposed as

$$\|\Delta x_d\| \leq v_{\max}\, \Delta t,\quad \|\Delta q_d\| \leq \omega_{\max}\, \Delta t.$$

Although a standard Jacobian-based end-effector velocity law is presented for completeness in the source, the implementation is explicitly described as relying on TracIK rather than Jacobian servoing (Chen et al., 6 Sep 2025).

Several engineering choices are reported as important for responsiveness. Motion execution latency was reduced from approximately $q_d$0 ms to approximately $q_d$1 ms by increasing joint speed to approximately $q_d$2 deg/s, applying low-pass filtering to phone pose streams, and performing solution comparisons to avoid singularities and joint limits. Stability is described pragmatically rather than through formal passivity or time-delay analysis: low-pass filtering, rate limits, IK solution comparison, and virtual fences constitute the core stabilizing mechanisms. The paper explicitly states that it does not report formal passivity proofs or time-delay compensation models (Chen et al., 6 Sep 2025).

These design decisions define TeleopLab as a solver-centric teleoperation system: the human specifies motion through a relative smartphone pose, the server computes a filtered end-effector target, TracIK resolves that target into joint space, and the robot-side execution layer enforces the commanded motion subject to kinematic and workspace constraints (Chen et al., 6 Sep 2025).

4. Empirical evaluation and measured outcomes

The reported evaluation is a pilot study conducted with six students during class sessions over three weeks, amounting to approximately $q_d$3 person-hours. Participants connected via cellular data, home WiFi, or company WiFi. A staff member at the laboratory handled specimen preloading and unloading and assisted with occasional robot errors. The experimental task sequence consisted of connecting the smartphone app, teleoperating the robot to pick up a specimen, placing the specimen on the Mark-10 tensile tester, operating the tester through the Start and Reset controls, and returning to the reload location to repeat the cycle (Chen et al., 6 Sep 2025).

The primary performance measure was cycle time per measurement, defined from the start of specimen pickup to the start of the next pickup phase. The reported learning trend is descriptive rather than inferential. Early in use, over approximately the first ten trials, average cycle time was approximately $q_d$4 s. Around thirty trials, it had reduced to approximately $q_d$5 s, and by forty trials it had stabilized in the $q_d$6–$q_d$7 s range. The reported summary statistic is a 46.1% decrease in cycle time as familiarity increased, accompanied by decreasing error bars in the reported figure (Chen et al., 6 Sep 2025).

The subjective measures are summarized below.

Measure	Value	Remark
Cycle-time reduction	46.1%	As familiarity increased
NASA TLX overall	38.19 (SD 20.82)	Manageable workload
SUS	73.75 (SD 14.21)	Above-average usability

The NASA TLX subscale means reported are mental demand $q_d$8, physical demand $q_d$9, temporal demand $1$0, effort $1$1, performance $1$2, and frustration $1$3, each on a $1$4–$1$5 Likert basis, with the overall score converted to $1$6–$1$7. The interpretation given in the source is that workload was manageable, with low physical demand, effort, and frustration, and moderate temporal and mental demands typical of real-time manipulation. The SUS score of $1$8 is described as above-average usability and above the commonly cited threshold of $1$9 (Chen et al., 6 Sep 2025).

Qualitative survey results indicated improvements after use in immersiveness, helpfulness in learning, intuitiveness, and engagement, with intuitiveness showing the largest positive shift. At the same time, the paper notes that statistical significance tests, $u(t)$0-values, and formal learning-curve fits are not reported. Moderate issues reflected in SUS responses—including the need for technical support and perceived inconsistency—are attributed to occasional connection drops, packet loss, and phone-frame drift (Chen et al., 6 Sep 2025).

5. Applications, deployment, and limitations

TeleopLab is presented as applicable across manufacturing education, robotics and mechatronics, and material science. The exemplar use case is a remote tensile strength laboratory modeled after the Berkshire Manufacturing Academy curriculum, but the paper also identifies more general manipulation tasks such as moving, grasping, pushing, stacking, and de-stacking. The central pedagogical claim is that live camera feeds combined with phone-based actuation preserve sensorimotor engagement and allow students to confront real-world nuances, such as the effect of sample positioning on a measurement, that simulations may miss (Chen et al., 6 Sep 2025).

The deployment model is modular. The hardware stack comprises a ROS-supported manipulator, a gripper, an end-effector camera, a workspace camera, a host computer running Ubuntu 20.04, and optional laboratory equipment such as the Mark-10 tensile tester. The software stack comprises ROS Noetic, TracIK, ABB EGM with Protocol Buffers where relevant, Cheese for camera capture, Zoom for remote desktop and video, and smartphone apps based on ARKit or ARCore. Networking is organized through port forwarding from the lab’s public IP to the host machine, with Zoom providing authenticated sessions and remote desktop access. The paper states that port forwarding avoids direct public IP exposure of the host machine, but it does not report additional encryption or authentication layers on the control channel (Chen et al., 6 Sep 2025).

The system’s limitations are explicitly described. First, it is suited to standard pick-and-place and equipment interaction, while very high-precision teleoperation remains a target for improvement. Second, performance depends on internet conditions; occasional connection drops, packet loss, and phone-frame drift were observed. Third, the pilot still relied on human-in-the-loop support for specimen preloading, unloading, and rare recovery actions. Fourth, formal control guarantees are absent: safety is enforced through virtual fences, conservative motion constraints, and filtering rather than through passivity proofs or time-delay compensation models (Chen et al., 6 Sep 2025).

Planned improvements include high-precision teleoperation for dexterous tasks, adaptive feedback mechanisms to increase situational awareness, more intuitive interfaces and perception enhancements, and broader release of open-source resources. The project is described as intended for open-source release aligned with manuscript acceptance. The paper further notes that multi-user session management and autonomy assist are natural extensions for scaling, though not explicitly detailed (Chen et al., 6 Sep 2025).

6. Position within the teleoperation and remote-lab literature

TeleopLab occupies a specific position within a broader literature on remote robotics laboratories and shared teleoperation. Earlier virtual and remote laboratory systems allowed students to teleoperate a real mobile robot from home while using the same interface in simulation and on hardware, emphasizing authenticated remote access, code upload, and data download workflows (Chaos et al., 2024). TeleopLab differs in centering the interaction around smartphone-based end-effector teleoperation and integrated operation of laboratory equipment rather than remote algorithm deployment alone (Chen et al., 6 Sep 2025).

Within teleoperation research more broadly, TeleopLab addresses a different problem from studies of multi-robot burden, but it inherits a similar concern with human performance as a limiting factor. A web-based experiment on dual-robot teleoperation reported that the dual-robot condition took 34.3% more time on average than the single-robot condition, despite a simulated 50% dynamic advantage, and interpreted this as evidence that human coordination overhead can dominate raw dynamic capability (Wan et al., 2022). TeleopLab’s emphasis on relative pose origin, low-pass filtering, virtual fences, and simplified mobile-device interaction can be read as an attempt to keep that operator-side coordination burden manageable within an educational setting (Chen et al., 6 Sep 2025).

It also stands in contrast to interfaces that pursue richer shared autonomy or more complex authoring models. Task-level authoring systems allow remote operators to interleave exploration, specification, and execution by composing short sequences of high-level actions rather than continuously commanding the robot (Senft et al., 2021). Interfaces for highly redundant robots have incorporated autonomy modules, Behavior Trees, manipulability-aware whole-body coordination, and wearable haptic feedback to support dual-arm mobile manipulation and assistive tasks (Torielli, 12 May 2025). This suggests that TeleopLab belongs to a broader design space in which remote laboratory access, intuitive control, and varying degrees of autonomy are traded against cost, complexity, and deployment effort (Chaos et al., 2024, Senft et al., 2021, Torielli, 12 May 2025, Wan et al., 2022).

In that design space, TeleopLab’s distinguishing feature is not maximal autonomy or maximal immersion. It is the combination of a smartphone control paradigm optimized for educational intuitiveness and low latency, a complete lab integration of robot, equipment, cameras, and Zoom, and a pilot evaluation showing learning effects, manageable workload, and above-average usability in remote real-world tasks (Chen et al., 6 Sep 2025).