Objectomaly: Object-Centric Anomaly Detection

• Objectomaly is an object-centric anomaly detection framework that defines anomalies based on semantic and structural deviations at the object level.
• It employs a three-stage pipeline—coarse anomaly scoring, objectness-aware score calibration, and meticulous boundary precision—to enhance pixel-wise OoD outputs.
• Empirical results on urban road scenes show significant improvements in metrics like AuPRC and component-level F1, demonstrating its practical viability.

Objectomaly denotes an object-centric approach to anomaly detection in which the operational unit is a coherent object rather than an isolated pixel or an arbitrary dataset-level distribution shift. In its most specific usage, “Objectomaly” names an objectness-aware refinement framework for out-of-distribution segmentation in urban driving scenes; more broadly, the surrounding literature uses closely related ideas to define anomalies relative to object identity, object masks, object structure, or object-level priors (Song et al., 10 Jul 2025, Ahmed et al., 2019).

1. Object-centric anomaly as a research problem

A central precursor to Objectomaly is the reframing of out-of-distribution detection as semantic, context-dependent anomaly detection. In the context of object recognition, the primary semantic factor is foreground object identity, so an anomaly is an image whose main object belongs to a category not present in the training label set, or more generally whose semantic identity falls outside the set of “normal” objects defined by the task (Ahmed et al., 2019). This formulation explicitly separates semantic distributional shifts from non-semantic distributional shifts such as noise, compression, camera differences, background variation, or contextual changes that should not, by themselves, cause an object recognizer to become uncertain.

This object-centric view was motivated by a critique of standard dataset-level OOD benchmarks. When CIFAR-10 is treated as in-distribution and SVHN, LSUN, TinyImageNet, or synthetic noise as OOD, low-level image statistics are often sufficient to separate the datasets. The same critique applies when one object dataset is used as OOD for another: performance may reflect dataset-specific quirks and biases rather than meaningful semantic distinctions. The practical question is not whether a model can detect a change in collection pipeline, but whether it is sensitive to a novel object category or fine-grained subtype within the same domain (Ahmed et al., 2019).

The benchmarks proposed in that setting already anticipate later Objectomaly work. They include hold-out-class protocols on CIFAR-10 and STL-10, and fine-grained ImageNet subsets such as Dog (hound dog), Car, Snake (colubrid snake), Spider, and Fungus, where the anomalous class is a held-out fine-grained object category from the same semantic family. This directly measures detection of novel object categories or new species within a genus rather than detection of low-level distributional artifacts (Ahmed et al., 2019).

2. Objectomaly as an OoD segmentation framework

In the 2025 sense of the term, Objectomaly is a post-hoc refinement framework for out-of-distribution semantic segmentation that injects object-level reasoning and explicit boundary processing into otherwise pixel-wise OoD detectors. The setting is urban driving: a semantic segmentation model is trained on an in-distribution road-scene label set, while test images may contain unknown objects such as unusual obstacles, debris, strange vehicles, or animals. The model produces an anomaly score map

$S \in \mathbb{R}^{H \times W},$

and thresholding $S$ yields a binary OoD mask (Song et al., 10 Jul 2025).

The motivation is tied to three failure modes of existing OoD segmentation pipelines. First, boundary imprecision causes anomaly predictions to bleed into neighboring in-distribution regions or miss parts of the true anomaly near contours. Second, inconsistent anomaly scores within an object produce fragmented masks, with high scores in an object center but weak or low scores at edges or interior subregions. Third, background noise and false positives arise when textures, shadows, or clutter are spuriously assigned high anomaly scores. The core claim is that these problems persist because existing methods are not explicitly object-aware (Song et al., 10 Jul 2025).

The framework is organized into three stages.

Stage	Input/output	Role
Coarse Anomaly Scoring (CAS)	$I \mapsto S^{(0)}$	Generates a raw pixel-wise anomaly score map from an existing OoD backbone
Objectness-Aware Score Calibration (OASC)	$S^{(0)}, \{M_k\} \mapsto S^{(1)}$	Uses SAM-derived instance masks to impose object-wise consistency and suppress noisy background
Meticulous Boundary Precision (MBP)	$S^{(1)} \mapsto S^{(2)}$	Refines contours via Laplacian filtering and Gaussian smoothing

A defining property is model agnosticism. CAS can use max-softmax, max-logit, energy-based scores, feature-distance maps, or hybrid methods such as DenseHybrid, Maskomaly, PixOOD, and residual pattern learning. Objectomaly does not retrain the backbone and does not alter the underlying segmentation network; it acts purely as a post-hoc refinement layer (Song et al., 10 Jul 2025).

3. Objectness-aware calibration and boundary refinement

The objectness component is implemented through class-agnostic instance masks from the Segment Anything Model. Given an image $I$, SAM produces masks $\{M_k\}_{k=1}^{K}$, with each

$M_k \in \{0,1\}^{H \times W}.$

For mask $M_k$, the object pixel set is

$\Omega_k = \{(i,j)\mid M_k(i,j)=1\}.$

The description characterizes OASC as computing object-level statistics, especially the mean anomaly score

$S$0

and optionally the within-object variance $S$1, then using these statistics to normalize or homogenize scores inside each object mask (Song et al., 10 Jul 2025).

The intended effect is structural consistency. If an OoD object receives high scores in its center but only moderate scores near its edges, OASC lifts those edge pixels toward the object-level anomaly value. Conversely, if an in-distribution object contains a few spuriously high-scoring pixels, calibration pushes them down because the object-level score remains low. Outside reliable masks, background scores can be attenuated, which suppresses clutter-induced false positives. The paper describes this as transforming a noisy pixel-wise map into an object-consistent map more aligned with human perception and component-level metrics (Song et al., 10 Jul 2025).

MBP addresses contour quality rather than intra-object homogeneity. It applies Laplacian filtering to emphasize strong transitions and Gaussian smoothing to suppress high-frequency noise. The standard Laplacian kernels given are

$S$2

and the Gaussian kernel is

$S$3

The exact implementation detail is not fully spelled out in the excerpt, but the key idea is a Laplacian-plus-Gaussian pipeline that sharpens boundaries, suppresses leakage, and yields visually aligned contours (Song et al., 10 Jul 2025).

Objectomaly itself has no learnable parameters. It uses a pre-trained OoD backbone, a pre-trained SAM or variant, and a set of hyperparameters such as calibration strength, Gaussian kernel size, threshold $S$4, and area threshold for removing small blobs. This design is consistent with its intended role as a plug-in refinement stage (Song et al., 10 Jul 2025).

4. Evaluation protocol and empirical behavior

The framework is evaluated on road-scene OoD benchmarks: SMIYC (SegmentMeIfYouCan), specifically AnomalyTrack and ObstacleTrack, and RoadAnomaly. These datasets operationalize the distinction between in-distribution road-scene content and unknown objects not belonging to the training classes (Song et al., 10 Jul 2025).

The evaluation emphasizes both pixel-level and component-level criteria. Pixel-level metrics include Area under Precision–Recall curve (AuPRC), FPR$S$5, and AUROC, although AuPRC and FPR$S$6 are more emphasized for anomaly segmentation. Component-level evaluation uses component-level F1-score, where predicted and ground-truth connected components are matched by IoU and precision/recall are computed over components rather than pixels. This is consequential because a system that detects part of an object but misses its extent may achieve respectable pixel-level behavior while remaining operationally weak at the object level (Song et al., 10 Jul 2025).

Reported quantitative results are explicitly object-oriented. Objectomaly achieves AuPRC up to 96.99, FPR$S$7 down to 0.07, and component-level F1-score up to 83.44. The abstract and experiments section describe these gains as state-of-the-art on key OoD segmentation benchmarks, with especially notable improvements in component-level F1, which is consistent with the method’s emphasis on coherent regions and accurate boundaries (Song et al., 10 Jul 2025).

The ablations clarify how the three-stage design behaves. Without OASC, component-level F1 degrades because inconsistent scores inside objects lead to partial detections. Without MBP, boundaries become more fuzzy or irregular, especially for small or thin objects. Without SAM-based masks, structural consistency declines and false positives from background increase because there is no objectness prior. The full CAS + OASC + MBP pipeline provides the best overall performance across pixel- and component-level metrics (Song et al., 10 Jul 2025).

Qualitative results on driving videos further stress temporal robustness at the level of coherent objects: anomalies appear as consistent object-shaped regions rather than flickering pixel clusters. Some failure cases remain. When SAM merges adjacent objects or fragments a single object into many masks, OASC inherits that error; very small or thin anomalies can also be missed or overly smoothed by MBP (Song et al., 10 Jul 2025).

5. Relation to semantic anomaly detection in object recognition

Objectomaly’s segmentation formulation extends a principle already articulated for object recognition: anomaly detection should be defined relative to a task’s semantics and label space. In object recognition, the task context specifies which factors are semantic. A kitchen with a bed in the middle may be anomalous for scene classification, but not anomalous for recognizing “bed” as an object. Under this view, unusual background should not reduce confidence in an object class; the anomaly is the object identity, not incidental context (Ahmed et al., 2019).

The earlier object-recognition literature also provides a methodological contrast. On hold-out-class benchmarks for CIFAR-10 and STL-10, and on fine-grained ImageNet subsets, multi-task learning with auxiliary self-supervised objectives improved both classification and semantic anomaly detection. On CIFAR-10 hold-out classes, average precision improved from $S$8 to $S$9 for MSP and from $I \mapsto S^{(0)}$0 to $I \mapsto S^{(0)}$1 for ODIN when rotation prediction was added; on STL-10, MSP average precision improved from $I \mapsto S^{(0)}$2 to $I \mapsto S^{(0)}$3 (Ahmed et al., 2019). The important point is not the particular auxiliary task, but the claim that better semantic representations yield better semantic anomaly detection.

This earlier work also argued against relying on AUROC alone in imbalanced anomaly settings and instead focused on precision–recall and average precision. In practical terms, this aligns with later segmentation practice, where object-level coherence and the utility of a detected region matter more than a distribution-free scalar summary. A plausible implication is that Objectomaly inherits not only the object-level emphasis of semantic OOD, but also its insistence that evaluation must reflect deployment-relevant structure rather than merely dataset-level separability (Ahmed et al., 2019).

6. Broader object-level anomaly paradigms and limitations

The broader anomaly-detection literature provides a vocabulary for situating Objectomaly. A domain-independent typology describes anomalies through five dimensions: data type, cardinality of relationship, anomaly level, data structure, and data distribution, yielding 3 broad groups, 9 basic types, and 63 subtypes of anomalies (Foorthuis, 2020). Within that scheme, object-centric OoD segmentation is naturally tied to multivariate anomalies in spatial or spatio-temporal data and often sits at the boundary between atomic and aggregate levels, since a coherent anomalous object is detected as a connected region rather than a single pixel. This suggests that Objectomaly is best understood not as a special case of per-pixel thresholding, but as a structured anomaly detector whose unit of analysis is an object-shaped region.

An analogous object-level philosophy appears in high-energy physics. A framework for anomalous object detection at the LHC defines anomalous objects operationally as reconstructed objects that are not any of the standard reconstructed objects—isolated photons, isolated electrons, hadronic taus, or QCD jets. The method learns only standard objects, constructs vetoes that eliminate them up to predetermined acceptance rates, and treats whatever survives as anomalous. It is therefore an explicitly object-level anomaly detector rather than an event-level density estimator (Chakraborty et al., 2017). Although the sensing modality differs sharply from urban driving, the design principle is similar: objectness, calibration against standard categories, and downstream analysis of coherent anomalous objects.

A further analogue appears in anomaly detection for general object data in Hilbert spaces. Adaptive depth quantile functions map each observation to a real-valued function of $I \mapsto S^{(0)}$4, and anomaly detection is motivated by the presence of antimodes in the data-generating distribution. The method is applicable to Euclidean and non-Euclidean objects, including shapes and topological summaries via kernels, and uses the geometry of midpoints, holes, and low-density regions to separate anomalous objects from the bulk (Chandler et al., 2022). This reinforces a shared theme: object anomalies are often best characterized through structure, coherence, and relation to low-density or semantically novel regions rather than through isolated scalar deviations.

The limitations of Objectomaly, in the segmentation sense, are correspondingly structural. Its effectiveness depends on SAM or similar instance-mask quality; crowded scenes, over-segmentation, under-segmentation, and very small or thin anomalies remain difficult. Running SAM and additional filtering adds inference cost, even if faster SAM variants mitigate some overhead. The proposed future directions are more efficient instance segmentation or lightweight SAM, tighter integration between semantic segmentation, OoD detection, and objectness, temporal extensions such as SAM 2 for videos, and adaptation to domains such as 3D point clouds and multispectral imagery (Song et al., 10 Jul 2025). These limitations do not undermine the core formulation; they specify where the objectness prior is most likely to fail or require redesign.

Objectomaly therefore names both a specific refinement pipeline for OoD segmentation and a broader methodological orientation in anomaly detection. Across object recognition, road-scene segmentation, high-energy physics, and general object data, the common thesis is that anomalies of operational interest are often coherent objects defined relative to a task’s semantics or structure, and that methods designed at the object level can better align detection outputs with downstream decisions (Song et al., 10 Jul 2025, Ahmed et al., 2019).