Semi-Automatic Labeling Technique

• Semi-automatic labeling is a hybrid method that leverages human guidance for initial seeding while algorithms automate large-scale label propagation.
• It utilizes varied techniques such as clustering, tracking, and projection-guided methods to enhance annotation precision and reduce manual intervention.
• This approach significantly cuts down annotation time and improves dataset quality across diverse domains like medical imaging, video analysis, and robotics.

A semi-automatic labeling technique is a methodological framework, algorithm, or system that operates at the interface between fully manual, human-driven annotation, and fully automated computational labeling. Such approaches leverage human expertise to define, seed, or quality-control labels, while delegating large-scale propagation, inference, and consistency to algorithms. These methods are now fundamental to domains with large, unstructured, or complex data (e.g., images, video, audio, documents, 3D point clouds), enabling scalable, high-quality training sets for supervised machine learning, benchmarking, and scientific analysis.

1. Principles and Taxonomy of Semi-Automatic Labeling

Semi-automatic labeling combines user interaction with algorithmic inference, typically following a two-stage or iterative workflow:

• Human-in-the-loop seeding or correction: Human operators annotate a small subset, draw ROIs, cluster regions, confirm automatically generated labels, or intervene on uncertain/rejected cases.
• Algorithmic propagation or batch inference: Machine learning, classical clustering, tracking, or rules extend annotations across large datasets, parameterize label distributions, or refine initial guesses.

Distinguishing characteristics relative to fully manual and fully automatic labeling:

Dimension	Manual	Semi-Automatic	Automatic
Human effort	High	Moderate—targeted intervention	Minimal or none
Algorithmic role	None	Propagation, consistency, pre-labeling, QC	Full generation
Quality control	Human annotated	Human checks mixed with algorithmic feedback	Optional or post hoc
Adaptivity	Labeled per instance	Generalizes via model/rules; correction loops	Relies on generalization only

Prominent categories include user-guided clustering (Bres et al., 2016), tracking-based propagation (Zhu et al., 2021, Ince et al., 2021), interactive map or document labeling (Klute et al., 2019, Bres et al., 2016), pseudo-labeling in SSL (Higuchi et al., 2021, Zhu et al., 2023), projection-based label propagation (Benato et al., 2020), and tool-augmented iterative correction (Lin et al., 10 Jun 2025, Lester et al., 1 Apr 2025).

2. Core Algorithmic Methodologies

The implementation of semi-automatic labeling varies with modality and problem class but shares common algorithmic patterns.

2.1 Clustering and Color Segmentation

In domains such as document analysis and microscopy, operator supervision is focused on initial clustering parameters and region selection:

• Color-plane segmentation in administrative documents (Bres et al., 2016):
  • User draws windows, specifies cluster count per window ($2 \leq k_i \leq 5$).
  • K-means in RGB for each window. Centroids $\{\mu_1, ..., \mu_M\}$ form a color model.
  • For new instances, pixels $x$ are assigned to nearest centroid by Euclidean distance, with out-of-distribution thresholds flagging for human review.
  • Competitive processing times: 0.5–1s per 2500$\times$3500px page, $\geq$90% acceptance rate.
• Microscopy image labeling (P. et al., 2022):
  • Species-specific choice between k-means (for uniform stains) and Otsu thresholding (for non-uniform).
  • Morphological closing removes noise. Human selects method and structuring element radius per species.
  • Generated masks become ground-truth for deep-learning segmentation.

2.2 Tracking and Propagation in Video

For temporal data, semi-automatic labelers typically combine manual initialization with automatic propagation:

• Kernelized Correlation Filter (KCF) tracking for text videos (Zhu et al., 2021):
  • User labels frame 1 (bounding boxes for all instances); unique IDs assigned.
  • KCF trackers initialized for each object, propagate boxes forward.
  • Failure detection based on drop in peak correlation; flagged segments are corrected manually.
  • Achieves IoU agreement of 88.5–93.0% with manual labels, 50$\times$ time reduction.
• Tracking-by-detection with MHT for object tracking (Ince et al., 2021):
  • Low-threshold detector generates candidates, grouped into temporally consistent tracklets by Multiple Hypothesis Tracking (MHT).
  • Human confirms or rejects each tracklet; labels propagated across frames.
  • Reduces annotation workload by 82–96% vs. naive framewise labeling.

2.3 Interactive Label Propagation

Feature- or embedding-based methods allow users to visualize high-dimensional structure and seed label/guidance:

• Projection-guided propagation (Benato et al., 2020):
  • Autoencoder extracts features; t-SNE projects to 2D.
  • Users label seeds, algorithm propagates to high-confidence neighbors (e.g., via graph energy minimization or Optimum-Path Forest).
  • Interactive thresholding allows selection between automatic and manual assignment; reduces manual annotation by 20–40% over iterative labeling.

2.4 Semi-Automatic in 3D, Point Clouds, and Medical Imaging

• LiDAR semi-automatic segmentation (SALT) (Wang et al., 31 Mar 2025):
  • Data alignment (zero-shot) projects 3D points as pseudo-images for foundation model inference.
  • 4D-consistent prompting tracks instances across time and scene structure.
  • 4D Non-Maximum Suppression and smoothing fuse spatially and temporally redundant regions.
  • Achieves up to 18.4% PQ gain over prior zero-shot methods and reduces annotation time by $\sim$83%.
• RECIST annotation for tumor response (Tang et al., 2018):
  • Cascaded networks: Spatial Transformer for lesion normalization, followed by Stacked Hourglass for endpoint detection.
  • Multi-task loss includes geometric constraints; only rough ROI input needed from annotator ($\sim$2–3s).
  • Outperforms radiologist inter-reader agreement on DeepLesion.

3. Interactive Systems and Annotation Tools

Modern semi-automatic labeling is increasingly tool-driven:

• BakuFlow (Lin et al., 10 Jun 2025):
  • Modular annotation framework combining IoU-based label propagation between frames, an enhanced YOLOE detector accepting multiple visual prompts, and a live magnification UI for pixel-precise edits.
  • Reduces human intervention by $\sim$60–75% over manual/CVAT baselines, 0.6–0.7s/box labeling speed.
• Augmented Reality Semi-automatic Labeling (ARS) (Gregorio et al., 2019):
  • Robotic camera and AR-pen define 3D object boxes in space; automatically projected to 2D for each frame and cross-view.
  • >400$\times$ speed-up over manual bounding box annotation workflows.
• SALT for RGB-D video (Stumpf et al., 2021):
  • Combination of 3D box parameterization/interpolation and GMM-based segmentation (GrabCut) allows interactive but low-effort dense labeling, including 6-DoF pose and mask consistency.
• EasyLabel for cluttered scenes (Suchi et al., 2019):
  • Incremental construction protocol: add one object at a time, detect depth difference, extract mask with no segmentation or outline drawing.
  • Produces dense, pixel-accurate labels even in high occlusion; minimal human correction.

4. Statistical and Empirical Evaluation

Semi-automatic labeling techniques are evaluated along several axes, often by comparing to full manual baselines:

• Label Acceptance Rate: e.g., $\geq90\%$ for color-plane maps in document segmentation (Bres et al., 2016).
• Human workload reduction: 50–97% lower manual contribution in frames/boxes/objects annotated (Ince et al., 2021, Gregorio et al., 2019, Wang et al., 31 Mar 2025, Lin et al., 10 Jun 2025).
• Agreement Metrics: Mean IoU (88–94%) with manual labels (Zhu et al., 2021, Lin et al., 10 Jun 2025), Cohen’s $\kappa$ for entity extraction (0.63–0.85 in various tags) (Bohn et al., 2021).
• Annotation throughput: Up to 450$\times$ increase in frames/hour compared to manual (Gregorio et al., 2019).
• Model performance: mAP, F1, WER, PQ, LSTQ improvements as final training sets grow, or as human-in-the-loop correction is applied (Lester et al., 1 Apr 2025, Higuchi et al., 2021, Zhu et al., 2023, Wang et al., 31 Mar 2025).

Typically, iterative or multi-pass approaches exhibit diminishing correction rates, e.g. human correction per iteration falls sub-linearly as initial seed and model improve (Lester et al., 1 Apr 2025, Ince et al., 2021).

5. Limitations, Adaptivity, and Future Directions

Despite their demonstrated efficiencies, current semi-automatic labelers have limitations:

• Generalization and parameterization: Manual selection of clustering parameters, structuring elements, or thresholds can introduce non-trivial operator workload (P. et al., 2022, Bres et al., 2016).
• Model drift and tracker failure: Scene changes, new objects, or tracking failures necessitate fallback to human review or correction (Zhu et al., 2021, Ince et al., 2021).
• Modality-specific weaknesses: Depth-based approaches fail for translucent or specular objects (Suchi et al., 2019), color-space clustering has difficulty with compressed or artifact-laden images (Bres et al., 2016).
• Tool limitations: Algorithmic interpolation degrades with rapid object motion, and GMM-based segmentation can fail on textureless or reflective surfaces (Stumpf et al., 2021).

Research directions include:

• Moving feature extraction and clustering into perceptually-uniform spaces (e.g., CIELAB), integrating spatio-temporal consistency post-processing (MRFs, CRFs), iterative active-learning loops, hybrid metric and pattern-based propagation, and full integration with foundation models for zero-shot generalization (Wang et al., 31 Mar 2025, Stumpf et al., 2021).
• Ensemble confidence-based triaging and dynamic weighting of human intervention (Shi et al., 2021).
• Open-source toolkits with modular pluggable interfaces supporting new model and user-feedback paradigms (Lin et al., 10 Jun 2025, Gregorio et al., 2019).

6. Application Domains and Cross-Disciplinary Impact

Semi-automatic labeling is central in diverse domains:

• Document analysis: Structured color-plane segmentation for extracting semantic regions from forms and administrative documents (Bres et al., 2016).
• Biomedical imaging: Rapid prototyping of segmentation ground-truth for bacteria, pathology, or medical imaging (P. et al., 2022, Tang et al., 2018).
• Autonomous robotics: Dense instance-level ground-truth in RGB-D scenes, LiDAR streams, and object tracking (Suchi et al., 2019, Wang et al., 31 Mar 2025, Zhu et al., 2021).
• Speech recognition: Self-training, pseudo-labeling cycles to leverage large-scale unlabeled data (Higuchi et al., 2021, Zhu et al., 2023).
• Natural language processing: Semi-automatic relation/entity extraction from structured or unstructured text (Bohn et al., 2021).
• Remote sensing and earth observation: Global-scale stratified sample selection and label consensus for validation and benchmarking (Shi et al., 2021).

This breadth demonstrates the foundational status of semi-automatic labeling in modern data-centric research, where annotation cost and scalability are limiting factors.

---

Key references:

(Bres et al., 2016, Zhu et al., 2021, Higuchi et al., 2021, Wang et al., 31 Mar 2025, P. et al., 2022, Stumpf et al., 2021, Suchi et al., 2019, Gregorio et al., 2019, Bohn et al., 2021, Lin et al., 10 Jun 2025, Benato et al., 2020, Lester et al., 1 Apr 2025, Ince et al., 2021, Zhu et al., 2023, Tang et al., 2018, Klute et al., 2019, Shi et al., 2021)