Instance Net: Instance-Centric Computation

• Instance Net is a design principle for neural architectures that treat individual instances as primary signals, enabling personalized context mining.
• It employs instance formation, instance-conditioned feature extraction, and instance-specific prediction across domains such as HOI detection, segmentation, biomedical imaging, and point-cloud analysis.
• Empirical results demonstrate that Instance Net architectures can outperform traditional methods by leveraging tailored context and eliminating the need for post-processing steps like non-maximum suppression.

Instance Net is a broad designation, across the cited literature, for neural architectures that operate at the granularity of individual instances rather than only global labels or semantic classes. In the most explicit formulation, the Instance-centric Context Mining Network (InCoM-Net) describes an “Instance Net” as a network that builds personalized context representations per instance and reasons progressively about interactions (Seo et al., 2 Apr 2026). Related formulations appear in point-cloud scene understanding, unified image segmentation, biomedical imaging, depth-based object recognition, and weakly supervised video modeling, where the shared computational unit is the instance itself: a detected human or object, a proposal-specific point cluster, a learnable kernel, a query, a pixel group, or a segment in a bag (Nie et al., 2020, Zhang et al., 2021, Prytula et al., 3 Aug 2025, Bonde et al., 2015, Hong et al., 2020). This usage makes Instance Net less a single architecture than a recurrent design principle: instance-conditioned computation.

1. Terminological scope and formal meaning

Within the surveyed papers, the word instance is used in several precise but related senses. In HOI detection, an instance is a detected human or object for which context is mined separately; InCoM-Net extracts intra-instance, inter-instance, and global contextual cues for each detected entity, then progressively aggregates them with detector features for interaction reasoning (Seo et al., 2 Apr 2026). In point clouds, RfD-Net defines an instance-level network as one that jointly predicts the instance’s semantic class, 3D pose, and a complete high-resolution mesh, using proposal-specific point clusters and proposal-conditioned reconstruction (Nie et al., 2020).

In unified segmentation, K-Net treats each kernel as responsible for generating a mask for either a potential instance or a stuff class, so instance-ness is encoded by learnable kernels rather than bounding boxes or RoIs (Zhang et al., 2021). In biomedical image segmentation, IAUNet uses learnable object queries, SymTC assigns each anatomical instance to its own output channel, and YOLO2U-Net predicts a separate 3D mask for the primary cell inside each 3D bounding box (Prytula et al., 3 Aug 2025, Chen et al., 2024, Ziabari et al., 2022). In weakly supervised video highlight detection, MINI-Net interprets each video as a bag of segment instances and learns an instance-level highlight scorer together with a bag classifier (Hong et al., 2020).

This suggests that Instance Net is best understood as an organizational principle rather than a single model family. The defining property is that the architecture makes individual instances the first-class signal.

2. Core computational pattern

Across domains, Instance Net architectures usually share three operations: instance formation, instance-conditioned feature extraction, and instance-specific prediction. The mechanism differs, but the abstraction is stable. K-Net makes this explicit through the mask-generation rule

$M = \sigma(K \ast F),$

where a set of learnable kernels generates masks directly from a backbone feature map, and each kernel is iteratively updated to become conditional on its current group in the image (Zhang et al., 2021). In InCoM-Net, the corresponding instance-conditioned representation is formed by separately aggregating global, intra-instance, and inter-instance context and then fusing them layer by layer with detector queries:

$$f_i^l = \mathrm{FFN}\!\left([\,f_{i,G}^l \,\Vert\, f_{i,R}^l \,\Vert\, f_{i,C}^l\,]\right).$$

The result is a tailored context-processing pipeline for each detected entity (Seo et al., 2 Apr 2026).

Query-based designs instantiate the same idea with attention. IAUNet refines learnable object queries against multi-scale mask features using standard scaled dot-product attention,

$$\mathrm{Attention}(Q,K,V) = \mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V,$$

so that each query gathers the pixels belonging to one potential instance (Prytula et al., 3 Aug 2025). SymTC does not use explicit object queries, but it still enforces instance separation by producing 12 output channels per image: one background channel and 11 anatomical instances, with no post-processing required to split touching structures (Chen et al., 2024).

A second recurring property is that instance reasoning often couples local evidence with broader context. RfD-Net decouples semantic instance reconstruction into global object localization and local shape prediction, then bridges them so that detection provides canonical alignment and informative priors for reconstruction, while reconstruction feeds back gradients and improves detection (Nie et al., 2020). PFFNet fuses semantic and instance features panoptically through a residual attention feature fusion mechanism, while Deep Affinity Net predicts dense affinities and grouping embeddings so that local pixel relations and non-local instance coherence can both contribute to the final partition (Liu et al., 2020, Xu et al., 2020).

3. Major architectural families

The literature exhibits several distinct realizations of the Instance Net principle.

System	Domain	Instance mechanism
InCoM-Net	HOI detection	Per-instance multi-context mining with ICR and ProCA
K-Net	Unified segmentation	Learnable kernels generate and update masks
Deep Affinity Net	Cityscapes instance segmentation	Dense affinities and Cascade-GAEC graph partitioning
RfD-Net	Point-cloud scene understanding	Proposal-specific detection and semantic instance reconstruction
IAUNet	Biomedical instance segmentation	Learnable object queries on a full U-Net
SymTC	Lumbar spine MRI	One output channel per anatomical instance
YOLO2U-Net	3D microscopy	2D detection, multi-view fusion, per-box 3D segmentation

One family is instance-centric context mining. InCoM-Net is the clearest example. A DETR-based detector with a ResNet-50 backbone provides instance-level detector embeddings, a CLIP visual encoder provides multi-layer visual features, and the network uses Instance-centric Context Refinement (ICR) plus Progressive Context Aggregation (ProCA) to build personalized context representations per instance (Seo et al., 2 Apr 2026).

A second family is kernel- and query-based mask generation. K-Net replaces proposals and NMS with a fixed set of learnable kernels that are refined by an iterative kernel update head, enabling semantic, instance, and panoptic segmentation within one mask-driven framework (Zhang et al., 2021). IAUNet adopts a full U-Net backbone but adds a lightweight convolutional Pixel decoder and a Transformer decoder that refines object-specific features across multiple scales; each query maps to one potential instance (Prytula et al., 3 Aug 2025).

A third family is affinity- or graph-based instance formation. Deep Affinity Net predicts semantic maps, local affinities for 4-neighborhood pixel pairs, and grouping embeddings, then uses Cascade-GAEC to greedily merge nodes or segments from coarse to fine (Xu et al., 2020). This contradicts the view that an Instance Net must be proposal-based.

A fourth family is proposal-conditioned reconstruction or segmentation. RfD-Net reconstructs only high-objectness proposals and conditions implicit occupancy prediction on aligned local points plus proposal features (Nie et al., 2020). YOLO2U-Net detects 2D boxes in XY, XZ, and YZ views with YOLOv2, fuses them into 3D boxes, and uses a 3D U-Net to segment the primary cell in each 3D bounding box (Ziabari et al., 2022). US-Net similarly couples a detection branch with a segmentation branch on a shared backbone and refines per-box masks with a lightweight refinement U-Net (Xu et al., 2019).

A fifth family uses instance-specific outputs without post hoc splitting. SymTC assigns one channel to each vertebral body and disc, while HAISTA-NET augments Strong Mask R-CNN by concatenating an RGB image with a 1-channel human attention map, so that sparse partial boundaries become a first-class input to instance segmentation (Chen et al., 2024, Korkmaz et al., 2023).

An earlier instance-specific formulation appears in TemplateNet for depth-based object instance recognition, where an intermediate template layer uses prior knowledge of an object's shape to sparsify the feature maps without any additional parametrization from the template layer (Bonde et al., 2015).

4. Supervision, losses, and optimization strategies

Because Instance Net architectures center the instance as the optimization target, their loss design typically couples instance assignment with instance quality. K-Net and IAUNet both use Hungarian or bipartite matching to align predictions with ground-truth masks, then optimize classification and mask losses without NMS or box supervision (Zhang et al., 2021, Prytula et al., 3 Aug 2025). In K-Net, one-to-one assignment is mask-driven; in IAUNet, the matching cost combines classification and mask similarity, and the network includes a “no object” class.

Other systems use supervision tailored to their instance definition. MINI-Net employs a max–max ranking loss that enforces a higher highlight score for the most likely positive segment in a positive bag than for the hardest segment in a negative bag, together with a bag-level binary classification loss (Hong et al., 2020). RfD-Net optimizes a combined objective in which the detection loss is augmented by a shape loss consisting of occupancy reconstruction, latent regularization, and foreground denoising, with gradients from the shape head flowing back to the detector (Nie et al., 2020).

Several biomedical systems add auxiliary mechanisms to align instance confidence with mask quality or boundary accuracy. PFFNet introduces a mask quality sub-branch so that the confidence score of each object aligns with the quality of the mask prediction, and also adds a consistency regularization mechanism between the semantic segmentation tasks in the semantic and instance branches (Liu et al., 2020). US-Net modulates the detection classification loss by the segmentation output through an adaptive weight in its enhanced focal loss, while HAISTA-NET retains the standard Mask R-CNN objective and uses the human attention map solely as a 4th input channel (Xu et al., 2019, Korkmaz et al., 2023).

InCoM-Net uses a different balancing strategy. Both DETR and the CLIP visual encoder are frozen during HOI training, and Masked Feature Training alternates three input configurations—full, detector-only, and VLM-only—to prevent feature-source bias (Seo et al., 2 Apr 2026). This is not a generic property of Instance Nets, but it shows how instance-centric models often require explicit mechanisms to prevent a single information source from dominating the instance representation.

5. Application domains and empirical performance

The empirical range of Instance Net formulations is unusually broad. In HOI detection, InCoM-Net reports state-of-the-art performance on HICO-DET and V-COCO. With CLIP ViT-L on HICO-DET, it reports Default Full 43.96, Rare 45.61, and Non-rare 43.46, and on V-COCO it reports $\text{AP}^{\text{S1}}_{\text{role}}=73.6$ and $\text{AP}^{\text{S2}}_{\text{role}}=75.4$ (Seo et al., 2 Apr 2026). The same paper reports that instance-centric context mining outperforms RoI Align and Image Crop alternatives, and that gains saturate at $L=3$ ProCA layers.

In unified image segmentation, K-Net reports PQ=55.2 on MS COCO test-dev for panoptic segmentation, mIoU=54.3 on ADE20K val for semantic segmentation, and instance segmentation performance that reaches 40.6 AP with ResNet-101-FPN while remaining NMS-free and box-free (Zhang et al., 2021). In Cityscapes instance segmentation, Deep Affinity Net reports 32.4% AP on val and 27.5% AP on test, and achieves the best single-shot result as well as the fastest running time among all affinity-based models discussed in the paper (Xu et al., 2020).

In point-cloud scene understanding, RfD-Net reports mAP@0.5 = 35.10 for 3D detection, compared with MLCVNet 33.40 and RevealNet 29.29, and reports 37.02 IoU for single object reconstruction at per-instance $16^3$ grid, which the paper describes as an improvement of over 11 of mesh IoU in object reconstruction (Nie et al., 2020).

Biomedical and medical-image variants show the same instance-centered logic in different regimes. Triple U-Net on CryoNuSeg reports AJI 67.41 and PQ 50.56, compared with a U-Net benchmark score of AJI 52.5 and PQ 47.7 (Ahmed et al., 2024). SymTC reports the best average DSC of 94.900 ± 2.095% on the original test set and the best average DSC of 95.171 ± 2.144% on the augmented test set, with the lowest average $\mathrm{HD}_{95}$ in both settings (Chen et al., 2024). IAUNet reports AP 45.3 and AP$_{50}$ 75.3 on LIVECell with a ResNet-50 backbone, and on the Revvity dataset reports AP 53.7 with Swin-B and $N=300$ queries (Prytula et al., 3 Aug 2025).

Human-assisted and pathology-specific systems widen the scope further. HAISTA-NET reports increases of +36.7, +29.6, and +26.5 points in AP-Mask metrics over Mask R-CNN, Strong Mask R-CNN, and Mask2Former, respectively, on the PSOB dataset (Korkmaz et al., 2023). MILD-Net-RTS is reported as ranked 1st on all measures across both GlaS test sets with rank-sum=6, and YOLO2U-Net reports mAP 0.367, mAR 0.39, and mAJ 0.263 on synthetic 3D microscopy volumes (Graham et al., 2018, Ziabari et al., 2022).

These results do not imply that all Instance Nets solve the same task. They show instead that instance-centered computation recurs wherever the output must preserve individuality rather than only class membership.

6. Misconceptions, limitations, and open directions

A common misconception is that an Instance Net must be a proposal-based instance segmentation model. The surveyed literature does not support that view. K-Net is explicitly NMS-free and box-free, Deep Affinity Net is affinity-based and uses graph partitioning, SymTC performs instance separation through fixed per-instance channels, and MINI-Net addresses weakly supervised video highlight detection through multiple-instance ranking rather than mask prediction (Zhang et al., 2021, Xu et al., 2020, Chen et al., 2024, Hong et al., 2020).

A second misconception is that instance-centered models are necessarily fully automatic. HAISTA-NET shows the opposite: sparse, human-specified partial boundaries can be concatenated as a 4th input channel and substantially improve segmentation of high-curvature, complex, and small-scale objects (Korkmaz et al., 2023). This suggests that “instance-aware” and “human-assisted” are compatible rather than contradictory.

The main limitations are similarly diverse but structurally related. InCoM-Net depends on detection quality because its masks come from the detector; RfD-Net reports failure modes for very thin structures and extremely sparse point clouds; K-Net can miss or merge objects when the number of instances exceeds the kernel budget; IAUNet can produce duplicates when object counts are low; Triple U-Net remains sensitive to watershed marker selection; YOLO2U-Net propagates fusion and detection errors into per-box segmentation; TemplateNet underperforms on heavily occluded scenes because it does not explicitly model occlusion (Seo et al., 2 Apr 2026, Nie et al., 2020, Zhang et al., 2021, Prytula et al., 3 Aug 2025, Ahmed et al., 2024, Ziabari et al., 2022, Bonde et al., 2015).

Future directions proposed in the papers are consistent with these weaknesses. They include end-to-end fine-tuning of detector and VLM encoders, dynamic text prompts for verbs and objects, adaptive gating of context contributions, stronger pose refinement, better priors and multi-view fusion with RGB, explicit temporal modeling, uncertainty-aware detection-reconstruction coupling, learned instance centers or boundary-aware energy functions, boundary-consistency losses, topology-preserving losses, and 3D extensions when data quality permits (Seo et al., 2 Apr 2026, Nie et al., 2020, Ahmed et al., 2024, Chen et al., 2024).

Taken together, these works define Instance Net not as a fixed blueprint but as a technical stance: the network is organized around the formation, refinement, and prediction of individual instances. Whether the instance is a human–object pair participant, a query, a kernel, a point-cloud proposal, a gland, a cell, an anatomical structure, or a video segment, the central claim is the same: instance-conditioned computation can preserve distinctions that uniform scene-level processing tends to blur.