Foreign Object Intrusion Detection

• Foreign Object Intrusion detection is a process that identifies unexpected objects or anomalies in operational environments through diverse sensing techniques and imaging modalities.
• It employs various methodologies—ranging from supervised YOLO-based detection to self-supervised reconstruction and segmentation—with high precision and recall reported across studies.
• Integration of real and synthetic data enables annotation-efficient training and robust performance under challenges such as occlusion, class imbalance, and varied environmental conditions.

Foreign Object Intrusion (FOI) detection denotes the identification of objects, materials, or anomalies that are not supposed to be present in an operational scene, inspected product, or restricted space. In the cited literature, the problem appears under related labels such as Foreign Object Debris (FOD) in airports and food inspection, retained foreign objects (RFOs) in chest radiographs, and operational “other” classes in robotic sorting. The technical formulations range from bounding-box detection and pixel-wise segmentation to anomaly localization, inpainting, tracking, and human-assisted verification, and the sensing stack spans RGB/UAS video, RGB-D, dual-energy and conventional X-ray, computed tomography, and capacitive electric-field sensing (Munyer et al., 2021, Munyer et al., 2022, Andriiashen et al., 2021, Zeegers et al., 2022, Rittscher et al., 2018, Wang et al., 16 Sep 2025).

1. Terminology, scope, and operational significance

The airport literature usually uses the term FOD rather than FOI. One paper adopts the FAA definition directly: FOD is “Any object, live or not, located in an inappropriate location in the airport environment that has the capacity to injure airport or air carrier personnel and damage aircraft” (Munyer et al., 2021). That definition is intentionally broad, and the same paper emphasizes that FOD may be any object of any material, size, or color, including fasteners, aircraft parts, tools, catering supplies, apron debris, runway/taxiway materials, construction debris, plastics, natural materials, and winter contaminants such as snow and ice. Airport-serving aircraft with more than 9 seats are required to perform FOD management and daily inspection to maintain certification, so FOI/FOD detection is framed there as both a safety and compliance problem (Munyer et al., 2021).

Other domains define the foreign object more operationally. In dual-energy food inspection, a foreign object is defined as a fragment of material with different X-ray attenuation properties than the food product, and the concrete validation is on bone fragments in chicken fillets (Andriiashen et al., 2021). In chest radiography, the literature distinguishes incidental or non-critical foreign objects from critical RFOs such as sponges, needles, wires, sutures, guidewires, rings, and surgical instruments left in the patient; the latter are clinically dangerous and treated as “never events” (Wang et al., 9 Jul 2025). In automated textile sorting, FOI recognition is implemented as a semantic “other” class that includes bottles, cans, EGAD objects, garments outside the target set, and unidentifiable garments (Ergun et al., 5 Mar 2026).

The application spaces are correspondingly heterogeneous. The cited works cover airport pavements, runways, taxiways, aprons, railway tracks, railroad transmission lines, high-voltage transmission corridors, food-processing conveyors, chest radiographs, aircraft fuel tanks, textile recycling cells, and confined tanks in marine vessels (Munyer et al., 2022, Qi et al., 2024, Liu et al., 12 Oct 2025, Wang et al., 11 Feb 2025, Andriiashen et al., 2021, Le et al., 2020, Vashist et al., 1 Dec 2025, Wong et al., 2022). This suggests that FOI detection is better understood as a class of inspection problems than as a single detection benchmark.

2. Data resources and annotation regimes

Recent FOI work is strongly dataset-driven. In airport inspection, a Millard Airport dataset was created from portable-camera and UAS video, converted into frames, resized to 400×400 while retaining original images, manually annotated in CVAT, exported in Pascal VOC format, and validated with a custom tool; the resulting dataset contains 14,260 object instances across 11 object classes, with a 75/25 training/validation split (Munyer et al., 2021). FOD-A extends this dataset-centric direction by providing 31 object categories and over 30,000 bounding-box annotation instances, with per-instance environmental labels for bright/dim/dark and dry/wet conditions (Munyer et al., 2021). AARFOD further scales airport-and-rail FOI benchmarking by combining RailFOD23 and FOD-A into 48,409 high-resolution images with 74,334 annotated objects and 35 object categories (Qi et al., 2024).

Several datasets encode domain-specific supervision beyond ordinary boxes. The airport pavement anomaly-localization framework uses 81,185 clean runway/taxiway training images and 447 manually annotated test patches, with the key design choice that training requires only clean images and no FOD labels (Munyer et al., 2022). In chest radiography, cheXphoto does not provide object-detection or segmentation labels, so annotations were created under a cross-validation review process in which each image must be reviewed by at least two people; if criteria are not satisfied, annotations are cleared and the image is re-queued. The annotation criteria include object shape, location within the box, presence in or partly in the lung region, bounding-box complexity of at least 6–8 vertices, and reviewer agreement on object count (Le et al., 2020). Hopkins RFOs Bench adds object-level and image-level supervision for critical chest X-ray RFOs, with 144 critical RFO cases from 144 distinct patients, 150 No RFO cases, and 150 No-critical RFO cases, split patient-wise into 70% training, 10% validation, and 20% testing (Wang et al., 9 Jul 2025).

Synthetic and hybrid data are increasingly prominent where real acquisition is expensive or access-constrained. FOD-S2R, designed for aircraft fuel-tank inspection, contains 6,250 high-resolution images: 3,114 real images from a controlled fuel-tank replica and 3,137 synthetic images generated using Unreal Engine, with 14 classes and systematic variation in field of view, distance, lighting conditions, color, and object size (Vashist et al., 1 Dec 2025). In industrial X-ray inspection, a CT-based workflow yields 1800 radiographs per scanned object and creates aligned 2D labels by segmenting the foreign object in 3D and then generating virtual projections (Zeegers et al., 2022). In textile sorting, the benchmark contains 223 inspection scenarios with class counts of 38 shirts, 64 socks, 43 trousers, 12 underwear, 65 “other,” and 4 empty scenes (Ergun et al., 5 Mar 2026).

Dataset or benchmark	Scale reported	Distinctive annotation or structure
Millard Airport FOD dataset	14,260 object instances, 11 classes	CVAT, Pascal VOC, custom validation, 400×400 standardized images
FOD-A	31 categories, over 30,000 instances	Bounding boxes plus bright/dim/dark and dry/wet labels
AARFOD	48,409 images, 74,334 objects, 35 categories	Combined airport and railway FOI/FOD benchmark
FOD-S2R	6,250 images, 14 classes	3,114 real + 3,137 synthetic fuel-tank images
Hopkins RFOs Bench	144 critical RFO, 150 No RFO, 150 No-critical RFO	Image-level and object-level chest X-ray labels

The annotation burden itself has become a research topic. The CT-based workflow argues that manual annotation of radiographs is tedious, subjective, and expensive, and reports that the mean Jaccard index between workflow-generated labels and direct-projection ground truth in simulation is 0.961, while in simulation 100 manually annotated training objects are needed to match the quality achieved by the workflow using only about 7 CT-scanned objects (Zeegers et al., 2022). A plausible implication is that FOI detection increasingly depends not only on model design but on annotation-efficient data generation.

3. Methodological families

A large portion of the literature uses supervised object detectors. The Millard Airport study trains YOLOv5m for 100 epochs and reports 95.2% precision, 99.3% recall, and 99.1% mean average precision on the validation set (Munyer et al., 2021). On AARFOD, an improved YOLOv5s integrating FasterNet/PConv and NAM attention reaches precision 0.979, recall 0.981, [email protected] 0.988, and [email protected]:0.95 0.874, while reducing parameters by 25.12% and GFLOPs by 10.63% relative to baseline YOLOv5s (Qi et al., 2024). High-voltage transmission-line inspection similarly modifies YOLOv8m with GAM, SPPCSPC, and Focal-EIoU, improving $mAP_{0.5}$ from 92.8% to 95.5%, $mAP_{0.5:0.95}$ from 76.4% to 80.4%, and recall from 85.9% to 91.9% on a Yunnan power-grid dataset (Wang et al., 11 Feb 2025). For railroad transmission lines, MRS-YOLO improves YOLO11n to mAP50 94.8% and mAP50:95 86.4%, while reducing parameters by 44.2% and GFLOPs by 17.5% (Liu et al., 12 Oct 2025).

Several works argue that closed-set object detection is structurally insufficient when the object vocabulary is open-ended. The airport pavement anomaly-localization paper explicitly states that supervised object detection is subpar for FOD because FAA-defined FOD can be arbitrary objects, and proposes self-supervised localization by reconstructing clean runway images and thresholding the difference map $D = |P' - P|$ with Otsu’s method (Munyer et al., 2022). Its best “Outer ViT” autoencoder reaches an 82.7% detection rate at IoU $> 0.3$, outperforming FOD-A SSD at 79.6% and FOD-A YOLO at 66.7% in the reported comparison (Munyer et al., 2022). Railway inspection adopts a related semi-supervised adversarial convolutional autoencoder trained only on normal track images; its best encoded-feature anomaly score $A_{\text{enc}}$ yields AUROC 85.66%, AUPRC 87.72%, EER 0.24, precision 79.26%, recall 76.04%, and F1 77.62% (Wang et al., 2021).

Pixel-wise segmentation and hybrid pipelines are also common. For chest X-ray foreign-object removal, one work uses a YOLOv4 detector, an EfficientNet-B0 U-Net-like segmentation network, and Fast Marching inpainting, achieving detection precision 76%, recall 70%, F1-score 73%, [email protected] 61%, Dice 0.8889, IoU 0.8045, pixel-level accuracy 0.9312, and 85% completely inpainted images (Le et al., 2020). The industrial CT-labeling paper trains U-Net and Mixed-Scale Dense networks as image-to-image segmenters using CT-derived masks rather than manually drawn radiograph labels (Zeegers et al., 2022). Dual-energy food inspection instead uses a quadratic thickness-correction model on the quotient $R(x) = M_1(x)/M_2(x)$, noise normalization by thickness bins, and Chan–Vese active contours, attaining 95% sample-level F1 and correctly identifying 97% of normal samples (Andriiashen et al., 2021).

Other domains move beyond conventional CNN detection. Textile sorting frames FOI recognition as semantic classification by visual-LLMs after robotic grasping and inspection, with the “other” class as the operational FOI category; Qwen3-VL:235B achieves 87.89% overall accuracy and 93.85% on “other,” while Gemma3:12B is the only reported model with 100.00% accuracy on empty scenes, though the sample count is only four (Ergun et al., 5 Mar 2026). Power-transmission monitoring combines YOLOv7 segmentation, a ConvNeXt feature extractor trained with triplet loss, and a feature-assisted IoU tracker; the enhanced pipeline improves precision from 0.58 to 0.80, recall from 0.47 to 0.82, [email protected] from 0.52 to 0.83, and F1 from 0.51 to 0.85 (Wang et al., 16 Sep 2025).

4. Canonical workflows

One recurring workflow is the video-first object-detection pipeline. In the airport dataset study, portable cameras and UAS record videos of common FOD objects from multiple angles, a custom tool splits the videos into frames, the frames are resized and manually annotated, and a validation utility checks box integrity after video-to-image conversion and resizing. The paper gives a concrete example: a 30.6-second video at 30 FPS produces 918 images (Munyer et al., 2021). FOD-A uses a closely related expansion toolchain: video is trimmed, frames are extracted, the trimmed video is annotated in CVAT, annotations are exported in Pascal VOC XML, and environmental labels are stored in CSV; frame rate is reduced to 15 FPS from an original approximate 60 FPS to limit near-duplicates (Munyer et al., 2021).

A second archetype is detect–segment–repair. In smartphone chest X-rays, foreign objects are first localized by YOLOv4, then segmented by a U-Net-like model that outputs binary masks with foreground value 255 and background value 0, after which Fast Marching fills the masked region and the cleaned ROI is merged back into the original image (Le et al., 2020). The paper evaluates inpainting indirectly through downstream lung abnormality classification and reports accuracy gains of 3.3% for VGG16, 3.8% for ResNet50, and 0.9% for EfficientNet-B2 on inpainted data (Le et al., 2020).

A third workflow is annotation generation by tomographic lifting. In industrial X-ray FOI detection, the process is: select representative training objects, acquire fast low-dose radiographs, perform 3D CT reconstruction with SIRT and 100 iterations, segment the foreign object in the reconstructed volume by thresholding, and create virtual 2D projections for supervised learning (Zeegers et al., 2022). The segmentation operator is written as

$$S(z_{ijk}) = \begin{cases} 1 & z_{ijk} \ge \theta,\ 0 & z_{ijk} < \theta, \end{cases}$$

and the paper reports that fewer than 10 representative CT-scanned objects are often sufficient to achieve adequate performance in its industrial-like setting (Zeegers et al., 2022).

A fourth workflow is edge-oriented detect–embed–track. In power systems, YOLOv7-seg produces masks and boxes, a ConvNeXt encoder outputs 1024-D embeddings optimized by triplet loss,

$$L = \max\left(0,\ \|f(a)-f(p)\|_2^2 - \|f(a)-f(n)\|_2^2 + \text{margin}\right),$$

and a feature-assisted IoU tracker resolves association under low overlap or occlusion (Wang et al., 16 Sep 2025). The system supports incremental updates by adding embeddings for previously unseen objects into a reference database without retraining, and on Jetson Orin Nano the total latency is 42–73 ms/frame with 14–24 FPS (Wang et al., 16 Sep 2025).

A fifth workflow is geometry-first anomaly detection with human confirmation. In marine-vessel confined spaces, a nominal point-cloud map is compared to an online query map via local Mahalanobis discrepancy,

$d_M=\sqrt{(x_i-x_j)^T \Sigma_j^{-1} (x_i-x_j)},$

candidate outliers are clustered, robot viewpoints are planned, and remote humans review close-up images (Wong et al., 2022). Simulation yields recall 0.944 with wheel odometry and 0.923 with visual odometry, but low photo-level precision of 0.381 and 0.414 respectively; in physical trials, human photo-labeling accuracy reaches 0.981 (Wong et al., 2022). The design explicitly trades precision for recall, then uses the human as the final semantic verifier.

5. Evaluation logic and deployment constraints

FOI evaluation is not uniform across domains. Supervised detection papers typically report precision, recall, AP, mAP, IoU, parameter count, and GFLOPs (Qi et al., 2024, Wang et al., 11 Feb 2025, Liu et al., 12 Oct 2025). Segmentation-centric pipelines add Dice, pixel-level accuracy, and IoU, while the chest X-ray paper also defines Mean Difference in Object Counting,

$$MDOC = \frac{1}{\sum_i^N C_i} \times \sum_i^N (C_i - \hat{C}_i),$$

and

$mAP_{0.5:0.95}$0

to quantify object counting and mask agreement (Le et al., 2020). Medical RFO benchmarking emphasizes AUC, ACC, FNR, and FROC because false negatives are clinically dangerous (Wang et al., 9 Jul 2025). The dual-energy food paper distinguishes pixel-level segmentation F1 from sample-level defect detection and argues that correct rejection of normal samples is the more industrially relevant criterion (Andriiashen et al., 2021). The airport anomaly-localization work uses detection rate under IoU $mAP_{0.5:0.95}$1, explicitly acknowledging manual-labeling noise (Munyer et al., 2022).

These metric choices reflect different operating priorities. Airport, railway, and power-line studies repeatedly emphasize small-target detection, occlusion, complex backgrounds, lighting changes, weather, and large scale variation as core difficulties (Qi et al., 2024, Wang et al., 11 Feb 2025, Liu et al., 12 Oct 2025). Fuel-tank inspection adds rivets, joints, reinforcements, irregular textures, reflections, cluttered surfaces, and small partly hidden objects (Vashist et al., 1 Dec 2025). Textile sorting introduces hallucination behavior and exact-label constraints; llama-family models are reported to hallucinate when the class changes after multiple instances of the same class and in sparse scenes where no object is present, which directly affects empty-scene FOI handling (Ergun et al., 5 Mar 2026). In airport inspection by UAS, practical issues include weather obscuration, aircraft sharing the airspace, the need to recognize aircraft to stay out of active operations, and the computational cost of running detection and navigation on onboard hardware (Munyer et al., 2021).

Deployment-oriented papers often trade raw accuracy against latency, power, and extensibility. The power-system edge framework uses TensorRT FP16 for YOLOv7-seg and ONNX FP32 for ConvNeXt, with Orin Nano consuming 9–15 W and AGX Orin 18–45 W (Wang et al., 16 Sep 2025). The improved YOLOv5 rail/air detector interprets FasterNet as an efficiency mechanism and NAM as an accuracy-recovery mechanism under lightweighting (Qi et al., 2024). The transmission-line YOLOv8m paper reports that the improved model is not the fastest, but is still second in speed among the compared models and prioritizes recall because missed detections are more operationally costly (Wang et al., 11 Feb 2025).

6. Limitations, misconceptions, and emerging directions

A common misconception is that FOI detection is always a closed-set object-detection task. The airport pavement and railway anomaly papers explicitly reject that assumption: both are trained only on normal images and localize foreign objects through reconstruction error rather than class enumeration (Munyer et al., 2022, Wang et al., 2021). Another misconception is that better pixel segmentation necessarily implies better operational FOI detection. The dual-energy food paper shows the opposite emphasis, noting that sample-level identification of normal products without false alarms is often the more meaningful industrial measure, and it reaches 97% correct identification of normal samples under its best detection-oriented setting (Andriiashen et al., 2021).

Dataset incompleteness and class imbalance remain persistent constraints. The Millard Airport dataset is explicitly incomplete relative to the full FAA FOD taxonomy, and the authors identify balancing object counts, expanding categories, and adding images under rain as next steps (Munyer et al., 2021). The Yunnan transmission-line dataset is highly imbalanced in its original form, motivating augmentation by rotations, brightness variation, salt-and-pepper noise, and occlusion (Wang et al., 11 Feb 2025). Textile FOI recognition treats “other” as a merged class containing both true foreign objects and garments outside the target set, so contamination type is not yet disentangled (Ergun et al., 5 Mar 2026). In chest radiography, the cheXphoto inpainting pipeline is multi-stage and explicitly “not lightweight,” with limitations for edge or mobile deployment and reduced support for large foreign objects (Le et al., 2020).

Synthetic data is a major current direction, but its effects are modality-dependent. FOD-S2R reports that synthetic pretraining followed by real fine-tuning improves real-world detection and narrows the Sim2Real gap in aircraft fuel-tank inspection; for RF-DETR, synthetic pretrain plus real fine-tune raises mAPS from 0.383 to 0.676 and mAP50:95 from 0.702 to 0.740, whereas the reverse order performs worse (Vashist et al., 1 Dec 2025). In critical chest X-ray RFO detection, physics-based DeepDRR-RFO augmentation generally improves performance, especially when about 2,000 synthetic images are added, but diffusion-based RoentGen-RFO does not consistently help and often degrades results (Wang et al., 9 Jul 2025). This suggests that synthetic data is most effective when the image-formation model preserves task-relevant structure.

Human assistance also remains a recurring design choice rather than a temporary crutch. The marine-vessel system relies on humans to resolve the precision problem after high-recall outlier generation (Wong et al., 2022). The textile VLM system uses strict prompting and exact-label enforcement because semantically plausible but verbose responses are counted as wrong (Ergun et al., 5 Mar 2026). In airport inspection, future work includes practical UAS experiments in real operational settings, automated navigation, and item retrieval if the long-term goal becomes autonomous FOD removal (Munyer et al., 2021).

Across the surveyed literature, the next research steps are explicit: better coverage of open-ended object taxonomies, stronger difficult-condition datasets, improved small-object handling, more robust empty-scene reasoning, lighter edge deployment, tighter Sim2Real transfer, more reliable synthetic generation, and broader multimodal integration of geometry, semantics, and sensor physics (Munyer et al., 2021, Vashist et al., 1 Dec 2025, Wang et al., 9 Jul 2025, Wang et al., 16 Sep 2025). FOI detection is therefore not converging toward a single dominant architecture; it is diversifying into domain-specific inspection pipelines that balance open-set uncertainty, sensing constraints, annotation cost, and operational risk.