Self-supervised Learning Of Visual Pose Estimation Without Pose Labels By Classifying LED States (2509.10405v1)

Abstract: We introduce a model for monocular RGB relative pose estimation of a ground robot that trains from scratch without pose labels nor prior knowledge about the robot's shape or appearance. At training time, we assume: (i) a robot fitted with multiple LEDs, whose states are independent and known at each frame; (ii) knowledge of the approximate viewing direction of each LED; and (iii) availability of a calibration image with a known target distance, to address the ambiguity of monocular depth estimation. Training data is collected by a pair of robots moving randomly without needing external infrastructure or human supervision. Our model trains on the task of predicting from an image the state of each LED on the robot. In doing so, it learns to predict the position of the robot in the image, its distance, and its relative bearing. At inference time, the state of the LEDs is unknown, can be arbitrary, and does not affect the pose estimation performance. Quantitative experiments indicate that our approach: is competitive with SoA approaches that require supervision from pose labels or a CAD model of the robot; generalizes to different domains; and handles multi-robot pose estimation.

• The paper presents a self-supervised method that learns robot pose estimation via LED state classification, eliminating the need for pose labels.
• It leverages a Fully Convolutional Network with multi-scale inference to achieve real-time (153 Hz) performance on commodity GPUs.
• Results demonstrate competitive accuracy versus fully supervised methods, with robust generalization across diverse and multi-robot settings.

Self-supervised Visual Pose Estimation via LED State Classification

Introduction

This paper presents a self-supervised approach for monocular RGB relative pose estimation of ground robots, eliminating the need for pose labels or prior knowledge of the robot's shape or appearance. The method leverages the independent and known states of multiple LEDs mounted on the robot, along with approximate viewing directions and a single calibration image for metric depth disambiguation. Training data is autonomously collected by a pair of robots moving randomly, with each robot broadcasting its LED states. The core idea is to train a model to classify the state of each LED from an image, which implicitly forces the network to learn the robot's position, distance, and bearing relative to the camera. At inference, the LED states are not required, and the model generalizes to pose estimation in arbitrary environments and for multiple robots.

Figure 1: By solving the multi-LED state classification task, the model learns to estimate the robot's location, distance, and bearing from scratch.

Methodology

Model Architecture and Training

The proposed architecture is a Fully Convolutional Network (FCN) that outputs spatial maps for LED state prediction, robot localization, and relative bearing. Each output cell attends to a local receptive field (RF) in the input image. The model is trained using a multi-label binary classification loss (BCE) on the LED states, with spatial weighting via a localization map ($\hat{P}$) and further modulation by a bearing-dependent visibility function ($\hat{\Lambda}^k$). Multi-scale inference is achieved by processing rescaled versions of the input image, enabling distance estimation through the apparent size of the robot.

Figure 2: Overview of the approach: (a) input image is used to predict robot location and bearing; (b) multi-scale processing infers distance.

The final loss aggregates over spatial, LED, and scale dimensions:

$$\mathcal{L} = \frac{1}{K} \sum_{k=1}^{K} \sum_{s=1}^{S} \sum_{i=1}^{H'} \sum_{j=1}^{W'} \mathcal{L}_\text{ms}^{k,s} [i, j]$$

Inference Procedure

At inference, the robot's image location is computed as the barycenter of the normalized localization map, the bearing as the weighted average of the bearing map, and the distance as a convex combination of scale factors weighted by the multi-scale localization map. A single calibration image is used to convert the apparent size to metric distance. The model runs at 153 Hz on an RTX 4080, supporting real-time deployment.

Figure 3: Model's output maps for localization, bearing, and LED state prediction.

Experimental Evaluation

Data Collection and Baselines

Experiments are conducted on DJI Robomaster S1 robots equipped with four independently controlled LEDs. Data is collected in laboratory, classroom, gym, and break room environments, with 131K samples in the lab and 127K in out-of-domain (OOD) settings. Only 23% of lab samples contain a visible robot, highlighting robustness to sparse supervision.

Baselines include:

• Mean Predictor (returns mean pose)
• CNOS (CAD-based segmentation and template matching)
• Upperbound (fully supervised with pose labels)
• MegaPose (template matching with refinement; slow inference)

  Figure 4: Random training samples from laboratory and OOD datasets, with LED states annotated.

Quantitative Results

On the laboratory test set, the self-supervised model achieves competitive performance with the fully supervised upperbound, outperforming CAD-based methods in bearing and distance estimation. The model generalizes to OOD environments and supports multi-robot pose estimation. The LED state is not required at deployment, and fine-tuning on new environments is effective with limited data.

Model	$E_{uv}$ (px)	$E_\psi$ (deg)	$E_d$ (%)	$\Gamma^{45^\circ}_{1\text{m}}$ (%)	AUC (%)
Mean Predictor	141	86	34	10	50
Ours	17	17	24	70	98
Ours (OOD)	27	55	60	29	89
CNOS	13	72	35	25	N/A
Upperbound	18	14	11	93	99

The model discretizes distance into bins, resulting in a step function in predictions. Increasing the number of scales improves resolution at the cost of computation.

Figure 5: Predicted robot poses in various environments and multi-robot scenarios.

Analysis and Implications

Robustness and Generalization

The approach demonstrates robustness to sparse supervision, absence of pose labels, and varying LED states. It generalizes across environments without fine-tuning and supports multi-robot pose estimation by identifying multiple peaks in the localization map. Fine-tuning with small amounts of target domain data rapidly improves performance, outperforming training from scratch with limited samples.

Practical Considerations

• Computational Efficiency: Real-time inference is feasible on commodity GPUs.
• Data Collection: Autonomous, infrastructure-free data collection is enabled by LED state broadcasting.
• Deployment: No need for pose labels, CAD models, or LED state knowledge at inference.
• Scalability: The method can be extended to larger robot teams, with data efficiency scaling quadratically with the number of robots.

Limitations

• 2D Pose Focus: Extension to 3D pose is straightforward for position, but requires additional LEDs for full rotation disambiguation.
• Distance Discretization: Limited by the number of scales; finer resolution requires more computation.
• Occlusion Handling: Not explicitly addressed; further work needed for partial occlusions.
• Retraining for New Robots: Each new robot requires retraining, but data collection is simple and sim-to-real issues are avoided.

Future Directions

Potential avenues for future research include:

• Multi-scale architectures (e.g., U-Net) for improved distance estimation.
• Incorporation of temporal information (odometry, optical flow) for enhanced robustness.
• Extension to large-scale multi-robot systems for efficient data collection.
• Application to other sensors and actuators affecting robot appearance.

Conclusion

This work introduces a self-supervised method for visual pose estimation of ground robots, relying solely on LED state classification as a pretext task. The approach achieves competitive performance with supervised and CAD-based methods, generalizes across environments, and supports multi-robot scenarios. Its methodological simplicity, data efficiency, and real-world applicability make it a strong candidate for scalable robot localization in heterogeneous and dynamic settings. The framework is extensible to other modalities and actuators, provided their state affects observable appearance in sensor data.