COLER: Single-Pass NCut for Unsupervised Detection

• The paper presents COLER which couples a novel single-pass NCut with standard detector training to efficiently extract multi-object masks without costly clustering.
• COLER is a zero-shot unsupervised framework that uses density tuning, boundary augmentation, and feature ranking on DINO ViT features to generate pseudo-masks.
• The method improves detection performance with faster mask generation, reduced reliance on CRF post-processing, and demonstrated gains in AP metrics on benchmarks.

Searching arXiv for the COLER paper and related methods mentioned in the provided data. Cut-Once-and-LEaRn (COLER) is a zero-shot unsupervised framework for instance segmentation and object detection that combines a single-pass Normalized Cut mask generator, termed CutOnce, with a standard detector training and self-training pipeline (Feng et al., 4 Aug 2025). The method is positioned within the two-stage paradigm of pseudo-mask generation followed by detector learning, but departs from earlier pipelines by applying Normalized Cut only once, avoiding clustering-based multi-object extraction, and eliminating dependence on expensive mask post-processing such as CRF. In the reported formulation, COLER targets the discovery of object masks and bounding boxes without any manual annotations, using self-supervised features from DINO ViT-B/8 and a Cascade Mask R-CNN detector trained with off-the-shelf loss functions (Feng et al., 4 Aug 2025).

1. Problem setting and methodological context

Unsupervised instance segmentation and object detection aim to discover object masks and bounding boxes without manual annotations (Feng et al., 4 Aug 2025). Within this setting, two-stage pipelines based on pseudo-mask generation and detector training have been established as a practical design pattern. The provided context identifies TokenCut, MaskCut, and VoteCut as representative approaches that apply Normalized Cut on self-supervised features from DINO (Wang et al., 2022).

The central motivation for COLER is the observation that prior pipelines suffer from three concrete limitations: repeated NCut passes or clustering to obtain multiple objects, reliance on expensive post-processing to refine mask boundaries, and a limited number of detectable objects together with slow mask generation (Feng et al., 4 Aug 2025). COLER addresses these limitations through CutOnce, described as a single-pass NCut-based mask generator enhanced with local density tuning, boundary augmentation, and a feature-ranking filter, followed by detector training and self-training using standard losses only (Feng et al., 4 Aug 2025).

This design places COLER at the intersection of graph partitioning, self-supervised visual representation learning, and pseudo-label-based detector training. A plausible implication is that the framework is intended not merely as a new mask proposal heuristic, but as a reformulation of unsupervised object discovery in which the computational and algorithmic burden is shifted away from repeated graph partitioning and post hoc refinement toward a more structured single-pass extraction stage.

2. CutOnce: single-pass Normalized Cut for multi-mask extraction

CutOnce models an image as a graph $G=(V,E)$ of $N$ patch-nodes with edge weights $w_{ij}$ measuring similarity, and applies the Normalized Cut objective

$$J(S) = \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(S,V)} + \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(\bar S,V)},$$

where

$$\mathrm{cut}(S,\bar S)=\sum_{i\in S,\,j\in\bar S}w_{ij}, \qquad \mathrm{assoc}(S,V)=\sum_{i\in S,\,j\in V}w_{ij}.$$

The relaxed solution is obtained from the second smallest eigenvector $\mathbf{y}_1$ of

$$(\mathbf{D}-\mathbf{W})\,\mathbf{x}=\lambda\,\mathbf{D}\,\mathbf{x},$$

with $\mathbf{W}\in\mathbb{R}^{N\times N}$ containing $w_{ij}$ and $\mathbf{D}=\mathrm{diag}(d_{ii})$, $N$0 (Feng et al., 4 Aug 2025).

The pipeline begins by resizing the image to $N$1 and extracting “key” tokens $N$2 from DINO ViT-B/8 (Feng et al., 4 Aug 2025). From these features, CutOnce computes an affinity matrix, solves for $N$3, chooses its sign so that the foreground obeys object-centric priors, performs boundary augmentation to form $N$4, binarizes $N$5 at its mean, extracts connected components, and finally ranks the components by a feature score before selecting top regions until cumulative feature mass reaches a threshold $N$6 (Feng et al., 4 Aug 2025).

The methodological novelty lies in deriving multiple object masks from a single NCut pass. Rather than recursively partitioning the graph or applying an external clustering stage, CutOnce thresholds a single enhanced eigenvector map and decomposes the foreground into connected components. This suggests a reinterpretation of the NCut eigenvector: not simply as a binary foreground–background separator, but as a scalar field from which multiple object hypotheses can be recovered once saliency bias and boundary ambiguity are explicitly corrected.

3. Core modules: density tuning, boundary augmentation, and ranking

COLER attributes the effectiveness of CutOnce to three modules introduced specifically to make a single-pass NCut solution suitable for multi-object discovery (Feng et al., 4 Aug 2025).

First, the density-tune cosine similarity modifies affinity computation to account for local density variation. The method defines

$N$7

with

$N$8

The stated motivation is that a uniform temperature on cosine similarity ignores local density variations and can cause over-activation; by increasing $N$9 in dense interior regions, the method seeks to flatten similarity and improve intra-object uniformity (Feng et al., 4 Aug 2025).

Second, boundary augmentation is introduced because a single eigenvector tends to focus on the most salient object. The boundary map is defined as

$w_{ij}$0

and the enhanced map is

$w_{ij}$1

The claimed effect is to amplify weaker objects’ regions and sharpen separation between adjacent instances, without CRF post-processing (Feng et al., 4 Aug 2025).

Third, the ranking-based instance filter operates after thresholding $w_{ij}$2 and decomposing the foreground into connected components $w_{ij}$3. For each region,

$w_{ij}$4

and regions are sorted by $w_{ij}$5. The selected set is the smallest top-$w_{ij}$6 subset satisfying

$w_{ij}$7

The framework specifies a single hyperparameter $w_{ij}$8, set to $w_{ij}$9, to control total object coverage (Feng et al., 4 Aug 2025).

Taken together, these modules redefine the role of NCut in unsupervised discovery. The data explicitly states that CutOnce leverages the single-pass NCut’s global view but corrects its saliency bias via density-tuned affinities and boundary subtraction, thereby allowing automatic discovery of multiple objects without recursion or clustering (Feng et al., 4 Aug 2025).

4. Detector learning and self-training

After pseudo-mask generation, COLER trains a detector using Cascade Mask R-CNN in Detectron2 (Feng et al., 4 Aug 2025). Pseudo-masks and boxes generated by CutOnce serve as “ground truth,” and the training objective uses standard cross-entropy for classification, smooth-L1 for box regression, and binary cross-entropy for the mask branch, with no special pseudo-label loss (Feng et al., 4 Aug 2025). The main training schedule is 80 K iterations with copy-paste augmentation, batch size 8, learning rate $$J(S) = \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(S,V)} + \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(\bar S,V)},$$0, weight decay $$J(S) = \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(S,V)} + \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(\bar S,V)},$$1, and momentum $$J(S) = \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(S,V)} + \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(\bar S,V)},$$2 (Feng et al., 4 Aug 2025).

The self-training stage consists of one round. The detector is initialized from the trained model, run on ImageNet-val, and predictions with score at least $$J(S) = \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(S,V)} + \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(\bar S,V)},$$3, where $$J(S) = \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(S,V)} + \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(\bar S,V)},$$4 is the round index, are retained (Feng et al., 4 Aug 2025). These predictions are merged with previous pseudo-labels having IoU $$J(S) = \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(S,V)} + \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(\bar S,V)},$$5 in order to avoid duplication, after which training proceeds for 60 K iterations at learning rate $$J(S) = \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(S,V)} + \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(\bar S,V)},$$6 (Feng et al., 4 Aug 2025). The reported effect is improved fine-scale accuracy.

This training design is notable because the method does not introduce a pseudo-label-specific optimization scheme. In contrast to approaches that tailor losses to noisy supervision, COLER emphasizes that strong performance can be achieved with off-the-shelf losses if pseudo-mask generation is sufficiently structured and is subsequently refined via self-training (Feng et al., 4 Aug 2025). A plausible implication is that the pseudo-label quality, rather than detector-side loss engineering, is treated as the dominant bottleneck.

5. Implementation regime and evaluation protocol

The implementation details supplied for CutOnce are fixed and concrete: input size $$J(S) = \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(S,V)} + \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(\bar S,V)},$$7, DINO ViT-B/8 as the self-supervised model, and hyperparameters $$J(S) = \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(S,V)} + \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(\bar S,V)},$$8, $$J(S) = \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(S,V)} + \frac{\mathrm{cut}(S,\bar S)}{\mathrm{assoc}(\bar S,V)},$$9, $$\mathrm{cut}(S,\bar S)=\sum_{i\in S,\,j\in\bar S}w_{ij}, \qquad \mathrm{assoc}(S,V)=\sum_{i\in S,\,j\in V}w_{ij}.$$0, and $$\mathrm{cut}(S,\bar S)=\sum_{i\in S,\,j\in\bar S}w_{ij}, \qquad \mathrm{assoc}(S,V)=\sum_{i\in S,\,j\in V}w_{ij}.$$1 (Feng et al., 4 Aug 2025). The detector is Cascade Mask R-CNN trained for 80 K iterations, with learning rate $$\mathrm{cut}(S,\bar S)=\sum_{i\in S,\,j\in\bar S}w_{ij}, \qquad \mathrm{assoc}(S,V)=\sum_{i\in S,\,j\in V}w_{ij}.$$2 reduced to $$\mathrm{cut}(S,\bar S)=\sum_{i\in S,\,j\in\bar S}w_{ij}, \qquad \mathrm{assoc}(S,V)=\sum_{i\in S,\,j\in V}w_{ij}.$$3 in self-training, weight decay $$\mathrm{cut}(S,\bar S)=\sum_{i\in S,\,j\in\bar S}w_{ij}, \qquad \mathrm{assoc}(S,V)=\sum_{i\in S,\,j\in V}w_{ij}.$$4, momentum $$\mathrm{cut}(S,\bar S)=\sum_{i\in S,\,j\in\bar S}w_{ij}, \qquad \mathrm{assoc}(S,V)=\sum_{i\in S,\,j\in V}w_{ij}.$$5, batch size 8, and copy-paste augmentation (Feng et al., 4 Aug 2025).

Training is conducted on ImageNet-1K val, comprising 50 K images, and zero-shot evaluation is performed on COCO 20K, COCO val2017, LVIS, VOC, KITTI, OpenImages, and Objects365 (Feng et al., 4 Aug 2025). The evaluation setup therefore uses ImageNet-val as the sole training source while testing transfer to diverse downstream benchmarks.

The dataset protocol is significant because it frames COLER as a zero-shot transfer system rather than a benchmark-specific unsupervised learner. This suggests that the method is intended to assess object discovery capacity that generalizes across domains and label spaces, not merely to overfit the structure of a single target dataset.

6. Reported empirical results

The reported results cover pseudo-mask quality, zero-shot instance segmentation, zero-shot object detection, and ablations (Feng et al., 4 Aug 2025).

Evaluation setting	Metric	Reported result
COCO val2017 pseudo-mask quality	AP$$\mathrm{cut}(S,\bar S)=\sum_{i\in S,\,j\in\bar S}w_{ij}, \qquad \mathrm{assoc}(S,V)=\sum_{i\in S,\,j\in V}w_{ij}.$$6	CutOnce $$\mathrm{cut}(S,\bar S)=\sum_{i\in S,\,j\in\bar S}w_{ij}, \qquad \mathrm{assoc}(S,V)=\sum_{i\in S,\,j\in V}w_{ij}.$$7 (no CRF), $$\mathrm{cut}(S,\bar S)=\sum_{i\in S,\,j\in\bar S}w_{ij}, \qquad \mathrm{assoc}(S,V)=\sum_{i\in S,\,j\in V}w_{ij}.$$8 (+CRF)
COCO val2017 pseudo-mask quality	Mask generation time	$$\mathrm{cut}(S,\bar S)=\sum_{i\in S,\,j\in\bar S}w_{ij}, \qquad \mathrm{assoc}(S,V)=\sum_{i\in S,\,j\in V}w_{ij}.$$9 s/img (10× faster)
COCO 20K zero-shot instance segmentation	AP$\mathbf{y}_1$0	$\mathbf{y}_1$1
COCO val2017 zero-shot instance segmentation	AP$\mathbf{y}_1$2	$\mathbf{y}_1$3
LVIS zero-shot instance segmentation	AP$\mathbf{y}_1$4	$\mathbf{y}_1$5
Zero-shot object detection	AVG AP$\mathbf{y}_1$6	$\mathbf{y}_1$7
Zero-shot object detection	AP$\mathbf{y}_1$8	$\mathbf{y}_1$9

For pseudo-mask quality on COCO val2017, CutOnce achieves AP$$(\mathbf{D}-\mathbf{W})\,\mathbf{x}=\lambda\,\mathbf{D}\,\mathbf{x},$$0 without CRF and $$(\mathbf{D}-\mathbf{W})\,\mathbf{x}=\lambda\,\mathbf{D}\,\mathbf{x},$$1 with CRF; MaskCut/CRF is reported at $$(\mathbf{D}-\mathbf{W})\,\mathbf{x}=\lambda\,\mathbf{D}\,\mathbf{x},$$2, and VoteCut/CRF at $$(\mathbf{D}-\mathbf{W})\,\mathbf{x}=\lambda\,\mathbf{D}\,\mathbf{x},$$3 (Feng et al., 4 Aug 2025). The same section reports mask generation time of $$(\mathbf{D}-\mathbf{W})\,\mathbf{x}=\lambda\,\mathbf{D}\,\mathbf{x},$$4 s/img, characterized as 10× faster (Feng et al., 4 Aug 2025). This juxtaposition is important: although VoteCut/CRF exceeds CutOnce on that specific pseudo-mask metric, COLER’s broader claim is that the overall pipeline yields stronger final zero-shot segmentation and detection performance.

For zero-shot instance segmentation, the reported AP$$(\mathbf{D}-\mathbf{W})\,\mathbf{x}=\lambda\,\mathbf{D}\,\mathbf{x},$$5 is $$(\mathbf{D}-\mathbf{W})\,\mathbf{x}=\lambda\,\mathbf{D}\,\mathbf{x},$$6 on COCO 20K, compared with $$(\mathbf{D}-\mathbf{W})\,\mathbf{x}=\lambda\,\mathbf{D}\,\mathbf{x},$$7 for CuVLER, representing a $$(\mathbf{D}-\mathbf{W})\,\mathbf{x}=\lambda\,\mathbf{D}\,\mathbf{x},$$8 improvement; $$(\mathbf{D}-\mathbf{W})\,\mathbf{x}=\lambda\,\mathbf{D}\,\mathbf{x},$$9 on COCO val2017, a $\mathbf{W}\in\mathbb{R}^{N\times N}$0 gain; and $\mathbf{W}\in\mathbb{R}^{N\times N}$1 on LVIS, a $\mathbf{W}\in\mathbb{R}^{N\times N}$2 gain (Feng et al., 4 Aug 2025). For zero-shot object detection, the average AP$\mathbf{W}\in\mathbb{R}^{N\times N}$3 is $\mathbf{W}\in\mathbb{R}^{N\times N}$4, described as $\mathbf{W}\in\mathbb{R}^{N\times N}$5 over the previous state of the art, with AP$\mathbf{W}\in\mathbb{R}^{N\times N}$6, and notable gains on LVIS $\mathbf{W}\in\mathbb{R}^{N\times N}$7 and KITTI $\mathbf{W}\in\mathbb{R}^{N\times N}$8 (Feng et al., 4 Aug 2025).

The ablation results identify boundary augmentation as the largest single contributor, yielding $\mathbf{W}\in\mathbb{R}^{N\times N}$9, while self-training adds approximately $w_{ij}$0 and density tuning adds approximately $w_{ij}$1 (Feng et al., 4 Aug 2025). These figures support the interpretation that the principal advance lies in modifying the NCut-derived signal before detector learning, with self-training serving as a secondary refinement stage.

7. Interpretation, limitations, and future directions

The reported explanation for COLER’s effectiveness is explicit: single-pass NCut provides a global view, while density-tuned affinities and boundary subtraction correct its saliency bias, enabling the automatic discovery of multiple objects without recursion or clustering; standard detector training and self-training then refine coarse masks into strong zero-shot detectors (Feng et al., 4 Aug 2025). This positions COLER as both a computational simplification and a methodological argument about where the core representational difficulty lies in unsupervised object discovery.

A potential misconception is that COLER is primarily a detector innovation. The supplied evidence points instead to CutOnce as the central contribution, with the detector stage deliberately kept conventional: Cascade Mask R-CNN, standard losses, and a straightforward self-training loop (Feng et al., 4 Aug 2025). Another possible misconception is that the method eliminates all post-processing. More precisely, it removes reliance on mask post-processing such as CRF for its main formulation, while the reported pseudo-mask table still includes a $w_{ij}$2CRF variant of CutOnce for comparison (Feng et al., 4 Aug 2025).

The stated limitations are equally specific. COLER struggles on heavily overlapping or occluded instances and can occasionally fail to resolve adjacent objects in the detector stage (Feng et al., 4 Aug 2025). These failure modes are consistent with the method’s dependence on a single eigenvector-derived map and connected-component decomposition: when object boundaries are not well separated in the underlying feature geometry, a single-pass partition may remain ambiguous. This suggests that the current approach is strongest when self-supervised features already induce sufficiently separable object manifolds.

The future direction identified in the source is end-to-end unsupervised object discovery that integrates NCut-style cuts into differentiable architectures (Feng et al., 4 Aug 2025). A plausible implication is that COLER can be read as a bridge between classical graph partitioning and modern learned detection pipelines: it retains an explicit spectral partitioning step, yet points toward architectures in which the cut objective itself becomes part of trainable inference.