SIXray: Security X-ray Benchmark

• SIXray is a large-scale security inspection dataset featuring over 1 million X-ray images for detecting prohibited items in complex, overlapping baggage scenes.
• It establishes benchmark protocols with defined partitions and annotation strategies to support image-level classification, weakly supervised localization, anomaly detection, and object detection.
• Key methodologies, including class-balanced hierarchical refinement and self-supervised pretraining, have demonstrated significant performance gains across varying imbalance regimes.

Searching arXiv for recent and foundational papers on SIXray to ground the article. SIXray is a large-scale security inspection X-ray benchmark for prohibited-item discovery in overlapping baggage images. Introduced in early 2019, it comprises 1,059,231 X-ray images, including 8,929 positive images and 1,050,302 negative images, and was designed to foreground three coupled difficulties: overlap caused by X-ray penetration, extreme class imbalance, and complex yet task-irrelevant baggage context (Miao et al., 2019). Across subsequent work, SIXray has served as a benchmark for image-level classification, weakly supervised localization, proposal-based recognition, end-to-end object detection, anomaly detection, and self-supervised pretraining, making it a reference dataset for studying prohibited-item discovery under severe occlusion and rare-positive regimes (Hassan et al., 2019, Webb et al., 2021, Zawar et al., 2022, Halat et al., 2023).

1. Definition, scope, and class taxonomy

The original benchmark release defines SIXray as a large-scale dataset of security inspection X-ray images for prohibited-item discovery in overlapping images, with 8,929 annotated prohibited-item instances distributed across five experimental classes: gun, knife, wrench, pliers, and scissors (Miao et al., 2019). The class counts reported for these five categories are gun (3,131 positive images), knife (1,943), wrench (2,199), pliers (3,961), and scissors (983) (Miao et al., 2019). The same source notes that hammer exists in the dataset with 60 samples but was omitted from the original experiments due to data scarcity (Miao et al., 2019).

Later papers often describe SIXray as a six-class suspicious-object dataset comprising gun, knife, wrench, pliers, scissors, and hammer, and several downstream studies restore hammer in their task definition (Hassan et al., 2019, Zawar et al., 2022). This class-taxonomy divergence is one of the most important interpretive points in the literature: SIXray as a data release contains six threat categories, whereas several benchmark protocols operate on five major classes because hammer is under-represented (Cani et al., 23 Jul 2025).

The modality is also task-dependent in subsequent usage. One detection study describes a subset of SIXray as 8,929 dual-energy, pseudo-colour X-ray scans of baggage and parcels (Webb et al., 2021), whereas a self-supervised study treats SIXray as a source of single-view grayscale baggage images after grayscale conversion in preprocessing (Halat et al., 2023). This suggests that the benchmark’s scientific role is less tied to a single presentation format than to its structural difficulty: cluttered baggage imagery with overlapping prohibited items and highly skewed label prevalence.

2. Annotation protocol and benchmark partitions

The original annotation pipeline combines image-level labels and localized test annotations. Image-level presence or absence labels for each class were taken from original security-inspector metadata, and, for the test split only, every prohibited-item instance receives a tight bounding-box manually drawn to support weakly supervised localization evaluation (Miao et al., 2019). Negative images carry no prohibited-item boxes (Cani et al., 23 Jul 2025).

To quantify the effect of class imbalance, the benchmark introduces three canonical subsets with increasing negative-to-positive ratio. The original benchmark uses a random 80%/20% train/test split, and later work describes the same policy as a four-to-one training/testing ratio (Miao et al., 2019, Hassan et al., 2019).

Subset	Composition	Total size
SIXray10	all 8,929 positive scans + 10× that many negatives	98,219
SIXray100	all positives + 100× negatives	901,829
SIXray1000	1,000 positives + all negatives	1,051,302

For these three subsets, the train/test counts reported in later work are 78,575/19,644 for SIXray10, 721,463/180,366 for SIXray100, and 841,042/210,260 for SIXray1000 (Hassan et al., 2019). Because object-level bounding boxes are available only in test sets in the original protocol, early localization studies on SIXray are explicitly weakly supervised rather than fully supervised (Miao et al., 2019).

Protocol variation appears in later comparative work. A 2025 cross-dataset evaluation uses a 90%/10% train/test split, approximately 953,308 training scans and 105,923 test scans, with no separate validation split and early stopping applied on training loss (Cani et al., 23 Jul 2025). This means that benchmark numbers reported across papers are not always directly commensurate, even when they share the dataset name.

3. Overlap model, imbalance regime, and benchmark difficulty

SIXray’s defining physical premise is the penetration property of X-ray imaging. In a typical baggage scan, objects are arbitrarily stacked, and penetration causes all items, foreground and background alike, to be visible simultaneously (Miao et al., 2019). The original paper formalizes this with a mixture-distribution assumption:

$$\mathbf{x}_n \approx \sum_{c=1}^{C} y_{n,c}\,\mathbf{x}_{n,c}, \quad \mathbf{x}_{n,c}\sim\mathcal{P}(\mathbf{x}\mid c).$$

Here, the observed image is approximately a superposition of sub-images drawn from class-conditional distributions, with $y_{n,c}\in\{0,1\}$ indicating whether class $c$ is present (Miao et al., 2019).

This formulation is paired with two other dataset properties. First, the positive/negative ratio in the full dataset is approximately $1:117$, or, equivalently, illicit items appear in under 1% of scans (Miao et al., 2019, Cani et al., 23 Jul 2025). Second, baggage scenes contain large amounts of unannotated everyday content, which the original benchmark describes as complex and meaningless contexts for the prohibited-item task because no background classes are annotated or consistently present (Miao et al., 2019).

These properties affect representation learning at multiple levels. The original CHR formulation argues that mid-level feature maps inherit additive mixtures from the input and can be confused by nuisance classes unless higher-level cues are used to refine them iteratively or hierarchically (Miao et al., 2019). Later work on anomaly detection reaches a related conclusion from a different angle by training only on normal baggage patches and treating poor reconstruction or feature mismatch as evidence of anomalous content (Zawar et al., 2022). A plausible implication is that SIXray is not merely an imbalanced dataset; it is a benchmark in which label rarity, physical superposition, and unannotated clutter interact directly in the feature hierarchy.

4. Foundational baseline: class-balanced hierarchical refinement

The baseline introduced with SIXray is class-balanced hierarchical refinement (CHR), which attaches a classifier head $h^{(l)}(\cdot;\xi^{(l)})$ and a refinement module $g^{(l)}(\cdot,\cdot;\tau^{(l)})$ to each stage of a backbone network (Miao et al., 2019). Refined features are propagated in a top-down manner:

$$\tilde{\mathbf{x}_n^{(l)}} = g^{(l)}\!\bigl(\mathbf{x}_n^{(l)},\,\text{upsample}(\tilde{\mathbf{x}_n^{(l+1)}});\tau^{(l)}\bigr),$$

and final prediction is the average over stage-level predictions,

$$\mathbf{y}_n = \frac{1}{L}\sum_{l=1}^{L}\mathbf{y}_n^{(l)}.$$

The associated class-balanced loss uses a binary mask $\mathbf{w}_n^{(l)}$ so that lower stages emphasize positives and hard negatives while easy negatives are pruned top-down (Miao et al., 2019).

Empirically, the original study reports that DenseNet-121 + CHR improves image-level mean Average Precision over plain DenseNet-121 from 77.36 to 79.56 on SIXray10, from 57.15 to 59.92 on SIXray100, and from 39.28 to 48.36 on SIXray1000 (Miao et al., 2019). Using CAM for weakly supervised localization, mean localization accuracy improves from 62.46 to 65.62 on SIXray10, from 44.70 to 50.31 on SIXray100, and from 34.61 to 43.87 on SIXray1000 (Miao et al., 2019). The gain is most pronounced in the heaviest imbalance regime, where only 1,000 positive images are retained against 1,050,302 negatives (Miao et al., 2019).

In the benchmark’s original framing, CHR is significant for two reasons. It treats overlapping X-ray imagery as a refinement problem rather than a standard single-pass classification problem, and it explicitly targets the trivial all-negative solution induced by extreme imbalance (Miao et al., 2019). This establishes a design pattern that recurs in later SIXray work: architectural modifications are typically justified either by overlap-aware feature refinement or by mechanisms that suppress the dominance of easy negatives.

5. Later methodological uses across detection, recognition, anomaly detection, and self-supervision

A prominent follow-on study integrates SIXray into a three-stage cascaded structure tensor plus ResNet50 pipeline. In that system, each full-size scan is first processed with adaptive histogram equalization; object proposals are then extracted by a cascaded structure tensor procedure that computes directional gradients at $N=4$ orthogonal orientations, forms $y_{n,c}\in\{0,1\}$0 structure tensors, selects the tensor with the greatest coherency via SVD-derived eigenvalues, binarizes and cleans the result morphologically, extracts closed contours, crops minimum-area bounding rectangles, subtracts extracted regions, reduces Gaussian scale, and repeats until no more objects remain; all proposals are finally classified by a single feedforward ResNet50 fine-tuned for 30 epochs (Hassan et al., 2019). On SIXray, that study reports subset mAP values of 0.9612 on SIXray10, 0.9297 on SIXray100, and 0.8894 on SIXray1000, and an overall full-split mAP of $y_{n,c}\in\{0,1\}$1, with IoU $y_{n,c}\in\{0,1\}$2, AUC $y_{n,c}\in\{0,1\}$3, training time 677 s, and test time 0.019 s per image (Hassan et al., 2019).

Operational object-detection work uses SIXray to evaluate end-to-end detectors that balance accuracy and latency. One study benchmarks FreeAnchor and Cascade R-CNN with ResNet50 backbones and FPN under the standard SIXray setup, training for 30 epochs in MMDetection with SGD, batch size 4, and ImageNet preprocessing (Webb et al., 2021). On the test set, Cascade R-CNN attains mAP 84.6 and FreeAnchor attains mAP 85.8 at IoU $y_{n,c}\in\{0,1\}$4; the corresponding throughput figures are 12.5 fps for Cascade R-CNN and 13.3 fps for FreeAnchor (Webb et al., 2021). The same study evaluates JPEG robustness and reports that moderate compression causes only modest degradation: at quality level 50, FreeAnchor drops from 85.8 to 84.5 and Cascade R-CNN from 84.6 to 82.9 (Webb et al., 2021).

SIXray has also been used as an anomaly-detection benchmark in settings where only normal data are used for training. A GAN-based study subdivides each image into non-overlapping $y_{n,c}\in\{0,1\}$5 patches and samples three sets of normal patches for training: 10,000, 100,000, and 500,000 patches (Zawar et al., 2022). The model combines a dense-skip-connected encoder-decoder generator, a self-attention-augmented PatchGAN discriminator with $y_{n,c}\in\{0,1\}$6 heads, spectral normalization on every convolutional layer, and an anomaly score

$y_{n,c}\in\{0,1\}$7

with $y_{n,c}\in\{0,1\}$8 and $y_{n,c}\in\{0,1\}$9 (Zawar et al., 2022). On SIXray, the reported AUC/Recall values are 0.983/0.79 for the 10k set, 0.998/0.76 for the 100k set, and 0.999/0.75 for the 500k set, compared against GANomaly and Skip-GANomaly baselines (Zawar et al., 2022).

In self-supervised learning, SIXray serves a different role: not as a labeled target task, but as a large unlabeled background pool. SegLoc uses approximately 1,050,302 unlabeled SIXray images as backgrounds, converts them to grayscale, erodes with a $c$0 kernel, thresholds at 50/255 to define authentic baggage regions, resizes cropped regions to width approximately 500 px, and pastes transformed PIDray foreground segments into these authentic regions to create positive pairs for contrastive pretraining (Halat et al., 2023). The method generates 200,000 synthesized images over approximately 30 epochs and modifies MoCo-v2 with one queue per class to avoid false negatives (Halat et al., 2023). On downstream PIDray evaluation, SegLoc improves over random initialization by 3% to 6% in the paper summary and, in the detailed results, by 6.3 percentage points in Easy-set segmentation AP@[0.50:0.95] and 9.6 percentage points in Hard-set detection AP@0.50:0.95.

6. Comparative evaluations, protocol variability, and continuing significance

A 2025 comparative evaluation situates SIXray within a broader cross-dataset landscape and tests ten detectors spanning CNN, custom CNN, transformer, and hybrid CNN-transformer schemes (Cani et al., 23 Jul 2025). On SIXray, the highest reported mAP@50 is 0.906 for D(YOLOv8, Next-ViT-S), while the highest mAP@50:95 is 0.794 for D(YOLOv8, CSPDarkNet53); D(RT-DETR, HGNetV2) reaches 0.901 mAP@50 and 0.789 mAP@50:95 with 9.32 ms inference time (Cani et al., 23 Jul 2025). The same evaluation reports that pure transformers such as DINO and Co-DETR attain mAP@50 around 0.90 but are much slower, at 159.85 ms and 187.00 ms, respectively, and that custom X-ray modules such as CHR, DOAM, and LIM underperform the vanilla YOLOv8 baseline on this protocol (Cani et al., 23 Jul 2025).

These later results do not invalidate earlier SIXray papers; rather, they demonstrate that benchmark performance is highly protocol-sensitive. The literature varies in class definition, with five-class and six-class formulations both in active use (Miao et al., 2019, Cani et al., 23 Jul 2025). It also varies in split construction, with 80%/20%, four-to-one, and 90%/10% partitions all reported in different studies (Miao et al., 2019, Hassan et al., 2019, Cani et al., 23 Jul 2025). Annotation availability is similarly task-dependent: the original release emphasizes image-level labels with tight test-set boxes for weakly supervised localization, whereas later anomaly-detection and SSL papers use subsets of publicly available boxes or the unlabeled majority as auxiliary resources (Miao et al., 2019, Zawar et al., 2022, Halat et al., 2023).

For research practice, the main significance of SIXray lies in its stability as a stress test for methods that claim robustness to overlap, occlusion, and positive scarcity. The recurrent reuse of the benchmark across CHR, CST-based proposal extraction, FreeAnchor-style detection, normal-only GAN anomaly detection, and synthetic SSL pretraining suggests that SIXray functions less as a single canonical leaderboard and more as a common substrate for studying how representation learning behaves when prohibited items are rare, heavily overlapped, and embedded in unannotated baggage clutter (Miao et al., 2019, Hassan et al., 2019, Webb et al., 2021, Zawar et al., 2022, Halat et al., 2023, Cani et al., 23 Jul 2025).