HiddenObjects: Detection & Inference

• HiddenObjects is a research topic focused on detecting and localizing concealed, occluded, or unknown objects through multimodal and context-driven approaches.
• Key methods include multimodal fusion, open-world discovery, unsupervised localization, and spatial prior learning to address challenges like occlusion, camouflage, and label-space incompleteness.
• Recent advancements integrate dual-stream encoders, state-space architectures, and probabilistic reasoning to enhance detection accuracy across both 2D and 3D scenarios.

HiddenObjects denotes a family of research problems centered on objects that are concealed, unlabeled, invisible, out-of-frame, or only implicitly specified by scene context and task intent. In current arXiv usage, the term covers multimodal hidden object detection under occlusion and camouflage, open-world discovery of previously unseen instances, unsupervised object localization without annotations, inference of fully invisible functional objects, probabilistic detection of unobserved objects in 2D and 3D, and class-conditioned spatial priors for object placement (Song et al., 28 Aug 2025, Wang et al., 2021, Chen et al., 14 May 2026, Bhattacharjee et al., 2024, Schouten et al., 12 Apr 2026). Across these settings, the common technical question is not merely whether an object is present, but how to localize it when direct visual evidence is incomplete, missing, or semantically unmodeled.

1. Conceptual scope and task taxonomy

The literature uses “HiddenObjects” for several distinct formulations. Some concern appearance degradation: partial occlusion, semi-opaque clutter, camouflage, or lighting collapse. Others concern label-space incompleteness: objects are visible but belong to unknown categories and are therefore treated as background by closed-world detectors. A third group concerns spatial invisibility: the target lies within the scene or near the camera but is fully invisible in the current image. A fourth, adjacent direction reverses the problem and models where an object would plausibly belong if inserted into a scene (Song et al., 28 Aug 2025, Yang et al., 2021, Chen et al., 14 May 2026, Bhattacharjee et al., 2024, Schouten et al., 12 Apr 2026).

Formulation	Hiddenness regime	Representative work
Multimodal hidden object detection	Occluded, camouflaged, low-light, semi-opaque clutter	HiddenObject (Song et al., 28 Aug 2025)
Open-world / unknown object discovery	Visible objects outside the training taxonomy	UVO (Wang et al., 2021), semantic topology (Yang et al., 2021), UnSniffer (Liang et al., 2023), U3HS (Gasperini et al., 2022), MEPU (Fang et al., 2023)
Unsupervised object localization	Salient objects hidden from supervision	PEEKABOO (Zunair et al., 2024)
Invisible / unobserved object reasoning	Fully invisible or out-of-frame objects	SceneFunRI (Chen et al., 14 May 2026), “Believing is Seeing” (Bhattacharjee et al., 2024)
Spatial prior learning for placement	Plausible but not yet inserted objects	HiddenObjects placement priors (Schouten et al., 12 Apr 2026)

This taxonomy matters because the underlying uncertainty differs. In multimodal perception, the object is physically present but its RGB evidence is degraded. In OWOD and dense open-world segmentation, the object is visible yet absent from the annotation vocabulary. In invisible-region reasoning, the object has no directly visible pixels. In placement-prior learning, the target object is hypothetical, and the problem is to estimate a distribution over plausible insertions rather than recover an existing instance.

2. Multimodal concealed-object perception

The HiddenObject framework addresses hidden or partially concealed object detection as a multimodal fusion problem. The motivating regimes include fruit under foliage, pedestrians behind cars, people in the dark, and camouflaged targets whose color or texture is similar to the background. The proposed system fuses RGB with one additional modality—thermal, depth, or NIR—through a Mamba-based state-space architecture built around a dual-stream VMamba encoder, stagewise Multimodal Feature Fusion (MMFF), and a channel-wise Mamba decoder (Song et al., 28 Aug 2025).

The backbone uses two branches with shared weights, patchifies both inputs, and applies four hierarchical encoder stages. At each stage, MMFF concatenates RGB and X-modality token sequences, computes input-dependent state-space parameters, performs forward and reverse selective scans, splits the sequence back into modality-specific outputs, scales them, and projects them into fused features. The decoder again uses SS2D, but with a cross-level design in which lower-level features generate state matrices and higher-level features parameterize the output mapping. The underlying state-space formulation follows continuous-time and discrete recurrences such as

$$y(t) = C h(t) + D x(t), \quad \dot{h}(t) = A h(t) + B x(t),$$

with input-dependent selective scanning used to make the dynamics context-sensitive. “Modality-agnostic fusion” means that the same fusion mechanism is reused for RGB+T, RGB+D, and RGB+NIR, without redesigning modality-specific fusion blocks.

The training protocol uses AdamW with initial learning rate $6 \times 10^{-5}$, weight decay $0.01$, 500 epochs, batch size 8, and input resolution $640 \times 480$, with an ImageNet-1K–pretrained VMamba-S backbone and up to 4 × NVIDIA RTX 3090 GPUs. Evaluation spans ACOD-12K, MFNet, PST900, NYU Depth V2, and SUN RGB-D. On MFNet, HiddenObject achieves mAcc 73.8 and mIoU 61.5; on PST900, mAcc 93.3 and mIoU 88.5; on NYU Depth V2, 56.8 mIoU; on SUN RGB-D, 52.1 mIoU; and on ACOD-12K, $S_{\alpha}=0.87$, $F_{\beta}^w=0.81$, and $E_{\phi}=0.97$. These results are reported as state-of-the-art or competitive, with especially strong gains on the more occlusion-heavy RGB–thermal and concealed-crop settings. The same study also identifies unresolved issues: spatial misalignment between sensors, degraded or missing modalities at inference time, and the non-trivial compute and memory demands of Mamba/SS2D despite its linear-time advantages over quadratic attention.

3. Open-world and unknown-object discovery

A major branch of HiddenObjects research treats hiddenness as a taxonomy problem: objects are present and often clearly visible, but they are absent from the training label set and are therefore absorbed into background. UVO formalizes this regime for dense video segmentation by defining taxonomy-free, open-world instance masks and tracks, without requiring class labels at evaluation time (Wang et al., 2021). UVO contains 1200 test videos and 108k densely annotated frames, with 12.29 objects per video on average, and 57% of instances are non-COCO objects. This design makes the benchmark materially different from DAVIS, YTVOS, and YTVIS, whose labels are not exhaustive and whose fixed taxonomies penalize discoveries outside the annotated class set. The reported drops from COCO-trained and YTVIS-trained baselines to UVO quantify how strongly closed-world supervision suppresses generic objectness.

Within OWOD, semantic topology provides a unified mechanism for both unknown discovery and incremental learning by assigning each class, including “unknown,” to a fixed semantic anchor in an embedding space (Yang et al., 2021). RoI features are projected into that space and optimized with a semantic anchor loss

$$\mathcal{L}_{sa} = \left\| \hat{f_i} - \mathcal{A}_i \right\|,$$

combined with semantic-head classification, RoI-head classification, and bounding-box regression. On the standard OWOD benchmark, this reduces Task-2 A-OSE from 7832 in ORE to 2546, while also improving WI and mAP. The same paper shows that even randomly generated topologies outperform prior OWOD baselines, although CLIP-derived anchors produce the best reported results.

UnSniffer approaches the same hidden-background problem from objectness and post-processing rather than anchor geometry (Liang et al., 2023). Its generalized object confidence (GOC) score is trained only on known samples, avoiding explicit suppression of unknowns hidden in the background; a negative energy suppression loss drives pure background proposals toward low negative energy; and a graph-based determination scheme replaces NMS for unknown boxes. On COCO-OOD, UnSniffer reports U-AP 0.454 and U-F1 0.479, compared with 0.214 or lower U-AP for the best cited baselines. The method is explicitly motivated by the observation that standard supervision uses unlabeled unknown objects as negatives in objectness, classification, and OOD scoring.

At pixel level, U3HS introduces “holistic segmentation,” which jointly performs panoptic segmentation on known classes and clusters unseen unknowns into instances without any prior knowledge about them or any unknown labels during training (Gasperini et al., 2022). It derives per-pixel uncertainty from Dirichlet evidence, thresholds uncertain pixels, and clusters their instance-aware embeddings with DBSCAN. The uncertainty measure is

$$u_{(i,j)} = \frac{K}{\sum_{k=1}^K \alpha_{(i,j),k}}.$$

On Lost&Found, U3HS attains unknown PQ 7.94 versus 1.45 for OSIS; on held-out COCO classes it reaches PQ 9.62 and RQ 13.20, while learning all 117 classes rather than converting subsets into void.

MEPU addresses the label-bias problem in OWOD with an unsupervised discriminative model, REW, that scores pseudo-unknown proposals via reconstruction error and Weibull foreground/background modeling, and with ROLNet, a classification-free self-training loop built on OLN (Fang et al., 2023). The method is designed to recover unknowns that are semantically unrelated to known classes. On the label-bias split, MEPU-FS reaches Unrelated Unknown R@10 25.0, substantially above OpenDet and CAT, while also improving known mAP. This suggests that hidden-object discovery in the open world depends not only on novelty detection but on how foreground is modeled before semantic classification is imposed.

4. Unsupervised localization and placement priors

A different use of HiddenObjects arises when objectness must be learned without supervision. PEEKABOO frames unsupervised object localization as a masking-based consistency problem: the same frozen DINO ViT-S/8 encoder and a $1 \times 1$ convolutional decoder with 770 learnable parameters predict masks on both an original image and a heavily masked version, and the refined masks are constrained to agree at both pixel and shape levels (Zunair et al., 2024). The total loss is

$6 \times 10^{-5}$0

Training uses only DUTS-TR, with images resized to $6 \times 10^{-5}$1, batch size 50, and about 500 iterations. On single-object discovery, PEEKABOO reports 72.7% CorLoc on VOC07, 75.9% on VOC12, and 64.0% on COCO20K. The same work reports strong unsupervised saliency performance, including Acc 94.6, IoU 79.8, and max $6 \times 10^{-5}$2 on ECSSD. Its central claim is that hiding image regions forces the network to infer foreground from context rather than explicit labels.

The 2026 HiddenObjects framework addresses a different but related problem: learning explicit, class-conditioned spatial priors for object placement by distilling the implicit placement knowledge of text-conditioned diffusion models (Schouten et al., 12 Apr 2026). For each image–class pair, it evaluates exactly 1004 candidate boxes via Qwen-Image with ControlNet-Inpainting, validates insertions with Grounded-SAM-2, scores them with ImageReward, and stores a dense set of verified placements,

$6 \times 10^{-5}$3

The resulting dataset contains 27M annotated placements across approximately 27k scenes and 50 object categories, with 77.7 positive boxes and 926.3 negative boxes per instance on average. A DETR-style distilled model with frozen ResNet-50 backbone, 6-layer transformer encoder, 50 learned queries, and Hungarian matching is then trained on the top 20 boxes per image–class pair.

This placement-oriented HiddenObjects line departs from hidden-object detection in the strict sense, because the object is not already present in the scene. Nevertheless, it provides a technically precise model of where an object is likely to belong. On downstream image editing, the full pipeline achieves 3.90 average VLM-Judge on OPA versus 2.68 for human annotation, and the distilled model runs at 3.77 ms per image and class on H100 with 188.9 MB memory, corresponding to a reported 230,000× speedup over the full diffusion pipeline. On a shared 28-class benchmark subset, the HiddenObjects-trained distilled model reports mAP 56.6, IoU50@1 62.9, and IoU50@5 79.1, far above zero-shot VLMs and prior placement models trained on OPA.

5. Invisible-region reasoning and unobserved 2D/3D detection

SceneFunRI isolates a stricter notion of hiddenness: the target object is completely invisible in the image, lies within the image boundary, and must be inferred from context, task instruction, and commonsense (Chen et al., 14 May 2026). Derived from SceneFun3D through a semi-automatic pipeline, the benchmark filters 1,232 candidates down to 855 instances. Each instance includes an RGB image, a task instruction, and a projected 2D ground-truth box for the hidden functional object. The principal metric is containment accuracy,

$6 \times 10^{-5}$4

On this benchmark, the strongest baseline, Gemini 3 Flash, achieves CAcc@75 of 15.20, mIoU 0.74, and Dist 28.65, while the human baseline reaches CAcc@75 66.43 and Dist 13.39. Prompting studies are grouped into Strong Instruction Prompting, Reasoning-based Prompting, and Spatial Process of Elimination (SPoE). Hidden prompting improves Gemini 3 Flash from 15.20 to 25.03 CAcc@75, but SPoE further reveals that current VLMs are often better at eliminating implausible regions than at directly selecting the correct hidden location.

“Believing is Seeing” formalizes unobserved object detection as a spatio–semantic distribution problem in 2D, 2.5D, and 3D (Bhattacharjee et al., 2024). For each object label $6 \times 10^{-5}$5 and spatial domain $6 \times 10^{-5}$6, the model predicts

$6 \times 10^{-5}$7

interpreted as the likelihood that an instance of class $6 \times 10^{-5}$8 is at location $6 \times 10^{-5}$9 given image $0.01$0. The paper evaluates 3D diffusion (DFM), 2D diffusion outpainting with SDXL, and several VLMs on indoor scenes from RealEstate10k and NYU Depth V2. Because the target is a distribution rather than a single box, evaluation emphasizes normalized entropy, cross-entropy, nearest-neighbor distance between peaks, false negative rate, and 2D region-wise accuracy rather than mAP. SDXL is reported as the strongest non-oracle method in 2D, while DFM systematically outperforms SDXL + monocular depth in most 2.5D and 3D metrics except cross-entropy and some FNR cases. The same study also makes clear that 3D diffusion inference is extremely costly and that both DFM and SDXL rely on downstream detectors whose errors propagate into the SSDs.

These two lines address complementary aspects of invisible-object reasoning. SceneFunRI is task-driven and functional: the target may be a drawer, socket, or door handle needed to execute an instruction. “Believing is Seeing” is object-centric and probabilistic: the target is a class-conditioned spatial distribution over unobserved space. This suggests that future HiddenObjects systems may need to combine the task grounding of SceneFunRI with the distributional formalism of unobserved object detection.

6. Memory-based 3D search, recurring limitations, and research directions

A search-oriented formulation appears in work that extends semantic abstraction to hidden objects in household environments (Pais et al., 22 Dec 2025). Here hidden objects are defined as “objects that cannot be directly identified by a VLM because they are at least partially occluded.” CLIP relevancy maps are treated as “abstract object” representations, projected into 3D with camera intrinsics and extrinsics, aggregated into point clouds, and then modeled with Gaussian Mixture Models to capture where specific objects tend to be placed. The number of mixture components is selected by Bayesian Information Criterion,

$0.01$1

and search proceeds by sampling or selecting likely GMM modes. In asymmetric toy distributions such as $0.01$2, the learned model significantly outperforms naive random search in first-time search accuracy; under uniform $0.01$3 placement, the learned model performs about the same as random search. The same study reports transparent-object failures for CLIP, difficulty enforcing receptacle constraints, instability of EM with small sample sizes, and the practical limitation that RoboTHOR and semantic abstraction were not run on the same machine.

Across the literature, several recurring limitations appear. Multimodal hidden-object detection remains sensitive to spatial alignment and degraded modalities. OWOD methods still struggle when proposal generators fail to cover tiny or heavily occluded unknowns, or when unknowns are visually close to known classes. Unsupervised localization methods remain biased toward salient-object formulations and can merge multiple instances. Placement priors inherit biases from Places365, COCO, and diffusion pretraining, and underrepresent very small objects because diffusion models often fail to generate them. Invisible-region reasoning benchmarks show that VLMs possess some latent commonsense but do not reliably ground it into compact spatial predictions. Generative 2D/3D hidden-object detectors are computationally expensive and depend on downstream detectors and depth estimators.

The proposed extensions in the cited works are correspondingly diverse. HiddenObject suggests Coupled Mamba, self-supervised pretraining, domain adaptation, and extension to LiDAR, radar, event cameras, tracking, and 3D scene understanding (Song et al., 28 Aug 2025). Open-world benchmarks motivate hybrid top-down and bottom-up systems, stronger objectness models, and exhaustive annotation protocols (Wang et al., 2021, Fang et al., 2023). Semantic-topology methods suggest richer graph structures and stronger vision–language anchors (Yang et al., 2021). SceneFunRI points toward tighter integration of task intent, commonsense priors, spatial grounding, and uncertainty-aware search (Chen et al., 14 May 2026). Semantic-abstraction search proposes better cluster initialization, multi-camera systems, and tighter real-time integration with embodied agents (Pais et al., 22 Dec 2025). A plausible implication is that HiddenObjects research is converging on three complementary resources: multimodal sensing for degraded evidence, class-agnostic objectness for unknown entities, and explicit spatial priors for invisible space.