- The paper develops a smoke detector recognition pipeline using real and semi-synthetic images, showing that mixed training improves generalization while synthetic-only training fails under real-world testing.
- YOLOv11n achieved the strongest overall performance at 0.884 average [email protected] across normal and difficult test sets, outperforming larger models under limited-data conditions.
- The study demonstrates embedded deployment feasibility with 157 ms latency, or about 6.4 FPS, on a Raspberry Pi 5, while distortion-focused augmentation reduced performance across all evaluated detectors.
Motivation and problem setting
Smoke detectors are a mandatory component of fire safety systems, and their effectiveness depends on periodic inspection and testing. In industrial and storage facilities, detectors are frequently mounted on high ceilings or in otherwise hard-to-reach locations, making manual inspection hazardous and expensive. This paper addresses the visual recognition component of an envisioned autonomous drone-based inspection system: a UAV carrying a camera and an aerosol-based sensor-testing device would locate smoke detectors and verify their function without human intervention at height. The work focuses on single-class object detection under realistic deployment constraints — limited training data, embedded inference hardware, and image degradations such as motion blur and defocus expected during flight.
The study is framed around four research questions: how to efficiently augment existing data and select an optimal training strategy involving synthetic imagery; which object detector configuration best suits smoke sensor detection; how custom augmentation affects detection performance; and how a detector can be deployed on an embedded device. Code, pretrained models, and the dataset are released publicly.
Dataset construction and semi-synthetic generation
The authors collected over 2,700 photographs of smoke detectors in public environments (corridors, schools, garages, shops) using varied cameras and lenses to avoid generalization bias. After filtering for duplicates, blurring, and incomplete objects, 1,672 annotated images remained. Because this volume is marginal for detector training, they constructed semi-synthetic data by compositing 30 laboratory photographs and 384 rendered views of two detector models onto real empty backgrounds. Composites were constrained to the upper 40% of background height (reflecting ceiling placement), with adaptive brightness adjustment and mild Gaussian blur to blend insertion boundaries; annotations were generated automatically, yielding 3,840 additional images.
Data were split into five experimental configurations: R-R (real train/real validation), R-G (real train/generated validation), G-G (generated/generated), G-R (generated train/real validation), and M-R (mixed train/real validation). Evaluation used two held-out test sets: Test-Normal with standard appearance and Test-Difficult containing motion blur, partial occlusion, small resolution, and other distortions. This dual-test design is a strength of the evaluation protocol, as it explicitly probes robustness rather than only in-distribution accuracy.
Experimental setup
Three detector families were compared: YOLOv11 (N/S/M/X variants), SSD (MobileNetV3-L and VGG16 backbones), and the transformer-based RT-DETRv2 (S/M/L variants). All models trained for 120 epochs at batch size 16 on an RTX 4080 SUPER workstation; YOLOv11 and RT-DETRv2 used AdamW while SSD used SGD, with per-family learning rates and weight decay. Image sizes differed (640×640 for YOLO and RT-DETRv2, 300×300 for SSD). Model selection was based on maximum average mAP@0.5 across both test sets combined with minimal score difference between them, favoring consistent rather than peak performance.
Training strategy results
The G-G configuration produced near-perfect validation scores — [email protected] of 0.995 for YOLOv11m, 1.000 for SSD-MobileNetV3-L, and 0.989 for RT-DETRv2-M — yet collapsed on both test sets (e.g., RT-DETRv2-M scored 0.007 on Test-D). This is a clear demonstration that composited semi-synthetic data alone do not carry sufficient information for generalization, likely compounded by domain shift from uniform rendering conditions and single-camera studio captures.
The M-R configuration was optimal for YOLOv11 (average test [email protected] of 0.827 vs. 0.811 for R-R) and RT-DETRv2 (0.803 vs. 0.762), while R-R was best for SSD (0.749). Notably, RT-DETRv2 exhibited the largest improvement from mixed training (+0.041 average), supporting the authors' interpretation that transformer backbones, being more sensitive to training set size, benefit disproportionately from additional generated samples. Synthetic data therefore function effectively as a form of augmentation when blended with real data, but not as a replacement for it.
Detector variant comparison
| Model |
Val [email protected] |
Test-N |
Test-D |
Test Avg |
Test Diff |
| YOLOv11n |
0.942 |
0.940 |
0.828 |
0.884 |
0.112 |
| SSD (VGG16) |
0.914 |
0.909 |
0.843 |
0.876 |
0.066 |
| RT-DETRv2-L |
0.876 |
0.897 |
0.769 |
0.833 |
0.128 |
The headline result is that the smallest model, YOLOv11n, achieved the best average test [email protected] of 0.884, outperforming its larger siblings (YOLOv11s: 0.855; YOLOv11m: 0.834; YOLOv11x: 0.868). The authors attribute this to overfitting or data hunger in larger variants given the limited dataset — smaller models generalize better under blur and sub-optimal illumination, conditions directly relevant to drone operation. For SSD, backbone capacity proved decisive: VGG16 dominated MobileNetV3-L across all metrics, particularly on Test-D (0.843 vs. 0.664), though at substantially higher inference cost. For RT-DETRv2, the medium variant led on validation and Test-N, but the large variant won on Test-D and overall average — interpreted as underfitting paired with greater robustness to degraded inputs.
A secondary finding worth noting: post-hoc analysis showed most low-confidence YOLOv11 detections still localized correctly on the sensor with slightly imprecise bounding boxes, suggesting practical usability exceeds what raw mAP indicates.
Augmentation experiments
A counterintuitive result runs through this section: custom Albumentations augmentations simulating flight-induced degradations (Autocontrast, Illumination, MotionBlur, Defocus, ChromaticAberration, ISONoise at probability 0.2) failed to improve any detector. YOLOv11n degraded marginally (0.884 → 0.875 average), RT-DETRv2-L more noticeably (0.833 → 0.794), and SSD-VGG16 suffered the largest drop, losing 17 points on Test-D (0.843 → 0.671). Combining default and Albumentations pipelines for YOLOv11n performed worst of all (average 0.824). The authors hypothesize that aggressive photometric and blur augmentations disrupt learned features in the high-capacity VGG16 backbone and induce overfitting to standard appearance in the transformer case. The implication is that simulation of deployment-time distortions during training cannot be assumed beneficial and must be validated empirically per architecture.
Embedded deployment
A three-node ROS-2 pipeline (camera acquisition, YOLO inference, debug/synchronization) was implemented for future drone integration. Inference benchmarks on Raspberry Pi 4B and 5 at 320×320 px input show YOLOv11n achieving 157 ms total latency (≈6.4 FPS) on the Pi 5 versus 609 ms on the Pi 4B — roughly 3.5× throughput improvement. The authors judge this sufficient for low-speed fine navigation. SSD-VGG16 (1,179 ms) and especially RT-DETRv2-L (3,764 ms) fall into the multi-second regime and are deemed unsuitable for onboard inference on this class of hardware, reinforcing the conclusion that transformer detectors remain impractical for severely power-constrained platforms without accelerators.
Licensing is treated as a first-class deployment criterion: Ultralytics YOLO models carry AGPL-3.0 (with paid commercial licensing available), whereas PyTorch SSD (MIT) and RT-DETRv2 (Apache-2.0) permit unrestricted commercial integration. Given that SSD-VGG16 nearly matched YOLOv11n's accuracy with the most consistent test-set behavior, the authors argue it remains a legitimate deployment candidate despite its age.
Limitations and open questions
The paper is candid about several constraints. The training dataset is small — unavoidable given restricted industrial access — and likely caps the achievable performance of larger variants such as YOLOv11x and RT-DETRv2-L. Generated composites ignore realistic shadow geometry and illumination spatial distribution, introducing potential training bias. All data originate from European locations, so worldwide variation in detector appearance and facility aesthetics may limit transfer. Only one transformer-based detector was evaluated, a consequence of the scarcity of edge-viable DETR variants. Finally, the full inspection system — mechatronic aerosol dispensing, sensor activation verification, lidar-assisted indoor navigation, and SLAM in GPS-denied environments — remains outside the scope of this work and constitutes the principal open engineering question.
Conclusion
This deployment-oriented study demonstrates that competitive smoke detector detection is attainable with modest, realistically collectible datasets, provided training combines real and carefully composited semi-synthetic imagery. The strongest quantitative outcome is YOLOv11n's average test [email protected] of 0.884 with real-time-capable inference on a Raspberry Pi 5, while the most instructive negative results are the failure of purely synthetic training (G-G) and the universal degradation caused by distortion-simulating augmentations. The finding that a legacy SSD with a VGG16 backbone rivals modern architectures in consistency, coupled with favorable licensing, offers practitioners a pragmatic alternative. The work leaves open whether larger models would dominate with drone-collected data at scale, and whether more physically faithful synthesis via domain adaptation could further close the sim-to-real gap.
