RePL: Pseudo-label Refinement for Semi-supervised LiDAR Semantic Segmentation

Abstract: Semi-supervised learning for LiDAR semantic segmentation often suffers from error propagation and confirmation bias caused by noisy pseudo-labels. To tackle this chronic issue, we introduce RePL, a novel framework that enhances pseudo-label quality by identifying and correcting potential errors in pseudo-labels through masked reconstruction, along with a dedicated training strategy. We also provide a theoretical analysis demonstrating the condition under which the pseudo-label refinement is beneficial, and empirically confirm that the condition is mild and clearly met by RePL. Extensive evaluations on the nuScenes-lidarseg and SemanticKITTI datasets show that RePL improves pseudo-label quality a lot and, as a result, achieves the state of the art in LiDAR semantic segmentation.

• The paper introduces REPL, a framework that explicitly refines noisy pseudo-labels using a dedicated masked reconstruction module.
• It leverages a teacher-student EMA setup and adaptive error detection to improve segmentation accuracy, achieving up to 60.0 mIoU on nuScenes-lidarseg with 1% labels.
• Theoretical analysis shows that the refinement reduces conditional entropy, while ablation studies confirm its robustness and moderate computational cost.

Pseudo-label Refinement in Semi-supervised LiDAR Semantic Segmentation: An Analysis of REPL

Introduction

The challenge of semantic segmentation in outdoor LiDAR point clouds is exacerbated by the high cost and difficulty of acquiring dense, accurate human-annotated labels. Semi-supervised learning (SSL) methods, which combine scarce labeled data with abundant unlabeled data, therefore are of critical importance. However, SSD paradigms in LiDAR semantic segmentation are highly susceptible to confirmation bias and error propagation, largely due to the reliance on pseudo-labels generated by models that themselves are imperfect. The paper "RePL: Pseudo-label Refinement for Semi-supervised LiDAR Semantic Segmentation" (2604.06825) presents a novel SSL framework, REPL, which directly addresses the pivotal problem of noisy pseudo-labels by implementing an explicit refinement process. This essay details its architectural, theoretical, and empirical contributions, situates its innovations within current research, and discusses the implications for both the field and future AI systems.

Methodological Innovations

Framework Overview

REPL operates within the prevalent teacher-student paradigm. The teacher network, updated via exponential moving average (EMA) of the student weights, generates initial predictions for unlabeled data, which become the pseudo-labels for the student. The key advancement is the introduction of a separate pseudo-label refiner module, whose task is to detect unreliable voxels in teacher-generated pseudo-labels and reconstruct them through a masked reconstruction mechanism. Specifically:

• Error Detection: Voxels are deemed unreliable when the teacher and student disagree or their confidence scores—determined adaptively per scene—are below a certain percentile. The criteria require strict alignment in both prediction and confidence between the teacher and student to classify voxels as reliable.
• Masked Reconstruction: Unreliable voxels are masked and replaced with learnable tokens. The refiner network receives both the raw input and the masked predictions as input and reconstructs probable class assignments in the masked regions. This procedure is analogous to masked autoencoders but is tailored to voxel-wise 3D data, focusing on contextual learning rather than memorization.
• Training Regime: The refiner is optimized using (a) supervised reconstruction loss over masked regions in labeled data, (b) a negative learning loss suppressing unlikely classes in unlabeled data (restricting plausible outputs based on the teacher’s class distribution), and (c) a scene-mixing strategy (LaserMix) that ensures exposure to diverse error modes and spatial configurations by mixing labeled and unlabeled scenes at the voxel level.

This design ensures pseudo-label improvement is proactive and non-post-hoc, directly addressing teacher bias before pseudo-labels are consumed as training targets by the student.

Theoretical Analysis

REPL’s mask-based refinement approach is substantiated with information-theoretic and probabilistic analysis. Two main results are established:

• Task Difficulty Reduction: The pseudo-label refinement task—reconstructing true labels from inputs plus teacher predictions—is formally shown to be no harder than original segmentation from input alone, as measured by conditional entropy. Incorporating semantic cues from teacher predictions reduces uncertainty, yielding a fundamentally more tractable learning task.
• Improvement Condition: An explicit condition is derived for pseudo-label accuracy improvement post-refinement. If the refiner’s correction rate $q_j$ and error introduction rate $r_j$ in the error region $E_j$ satisfy

$T_j - \frac{r_j}{q_j + r_j} > 0$

where $T_j$ is the teacher’s error rate in $E_j$, then refinement yields a net gain. Empirically, the simple error detection scheme employed in REPL consistently satisfies this condition over a range of label ratios and datasets.

Comparative and Ablative Experimental Evaluation

Datasets and Architectures

All experiments utilize industry-standard datasets—nuScenes-lidarseg and SemanticKITTI—covering various semantic classes and large-scale scenes. Cylinder3D serves as the backbone architecture for both segmentation and the refiner.

Main Results

REPL achieves the highest or near-highest mean intersection-over-union (mIoU) across all label ratios and datasets compared with a wide range of state-of-the-art semi-supervised, weakly supervised, and representation-augmented competitors. Notably:

• nuScenes-lidarseg: With only 1% labeled data, REPL achieves 60.0 mIoU, outperforming strong baselines such as IT2 (+2.5 mIoU) and LaserMix++ (+1.5 mIoU).
• SemanticKITTI: REPL consistently achieves the best or second-best results, particularly excelling at lower label ratios, which are most challenging for SSL.
• In all settings, the average mIoU improvement over supervised-only baselines is substantial, demonstrating the efficacy of explicit pseudo-label refinement.

Ablation and Sensitivity

A comprehensive ablation reveals:

• All loss components (supervised, negative, and mixed-scene) for the refiner collectively yield maximal mIoU improvements.
• Mask quality is a central bottleneck: improvements are observed even with simple heuristic masks, but an oracle mask (from ground-truth) dramatically widens the gap, suggesting further research on advanced error detection will be rewarding.
• Random masking during refiner training reliably increases robustness and generalization.
• The computational cost of the refinement module is moderate relative to the substantial segmentation gains (+9.1 mIoU for a marginal latency increase).

Pseudo-label Dynamics

Longitudinal analysis indicates that the refiner’s benefit is greatest early in training, when teacher-generated pseudo-labels contain more errors. As the student improves and the teacher's EMA stabilizes, the marginal benefit of the refiner decreases, but the overall pseudo-label quality maintains superiority to teacher-only or non-refined settings.

Implications and Future Work

Practical Relevance

REPL’s explicit pseudo-label refinement directly addresses the primary bottleneck in SSL-based LiDAR segmentation: confirmation bias and error propagation from noisy labels. The plug-and-play nature of the refiner, which can theoretically be decoupled and improved independently of the backbone segmentation model, makes this framework highly adaptable to future advances in 3D perception and model architectures.

In safety-critical applications (autonomous driving, mobile robotics), any reduction in label error propagation can have significant downstream effects on deployment safety and reliability. REPL's robustness to label scarcity is particularly valuable in industrial-scale 3D semantic mapping where labeling costs are prohibitive.

Theoretical and Research Directions

The information-theoretic underpinning of the improvement condition gives this work conceptual generalizability: similar refinement strategies could be adapted beyond point cloud segmentation to other semi-supervised, dense prediction tasks where pseudo-labels are intrinsically noisy.

Future directions include:

• Improved error region identification using more sophisticated uncertainty estimation or learned uncertainty modules;
• End-to-end or multi-stage refinement strategies with iterative feedback between the refiner and the teacher/student;
• Integration with foundation models or multi-modal fusion pipelines, further expanding pseudo-supervision capacity;
• Theoretical exploration of calibration and trustworthiness of refined pseudo-labels in highly imbalanced or open-set settings.

Conclusion

REPL provides a rigorous, theoretically grounded, and empirically validated framework for pseudo-label refinement in semi-supervised LiDAR semantic segmentation (2604.06825). By explicitly identifying and correcting unreliable pseudo-labels via a masked reconstruction refiner, REPL advances the state of the art in mIoU while maintaining computational efficiency and generalizability. Its theoretical analysis specifies mild, practical conditions for guaranteed accuracy improvement, and its methodology presents a path forward for future robust SSL pipelines in 3D perception domains.