Debate-Enhanced Pseudo Labeling for Camouflaged Objects

• The paper introduces a debate-enhanced approach that refines pseudo masks with entropy sampling and multi-agent deliberation, significantly improving detection precision.
• It employs an adaptive entropy-driven prompt selection strategy to ensure that ambiguous object boundaries are accurately captured.
• Integrating scribble annotations for supervision, the method effectively mitigates background intrusion and over-fragmentation in weakly-supervised settings.

Debate-Enhanced Pseudo Labeling is a methodology for refining pseudo masks in weakly-supervised camouflaged object detection (WSCOD), specifically utilizing sparse scribble annotations. Existing segmentation models such as Segment Anything Model (SAM) exhibit deficiencies in generating reliable masks for camouflaged regions—often suffering from background intrusion, incomplete coverage, and over-fragmentation due to the lack of task-specific semantic representations. The Debate-Enhanced approach aims to convert highly sparse annotation signals into robust pseudo masks using entropy-driven prompt point selection and a multi-agent debate mechanism, thereby facilitating both increased precision and recall of camouflaged object detection. This process is the initial stage of the ${D}^{3}$ETOR framework (Ge et al., 23 Dec 2025).

1. Motivation and Rationale

The core challenge addressed by Debate-Enhanced Pseudo Labeling lies in the limitations of general-purpose segmentation models applied to WSCOD. Specifically, pseudo masks generated by models such as SAM, when encouraged by heuristic prompts or simple thresholding, tend to fail in three systematic ways: - Inclusion of background clutter in masks. - Omission of visually ambiguous or camouflaged regions. - Over-fragmentation and failure to represent object coherence.

These deficiencies are exacerbated by the sparsity and inherent bias of scribble annotations. The principal objective is to exploit scribble supervision to derive multiple high-quality pseudo masks that cover the full extent of camouflaged objects with minimal noise.

2. Adaptive Entropy-Driven Point Sampling

The initial submodule of Debate-Enhanced Pseudo Labeling focuses on selecting prompt points that maximize local uncertainty and improve spatial distribution, facilitating more informative mask proposals from SAM.

Entropy Metric

For each pixel $p$ within the scribble region $S$, a $k \times k$ local intensity histogram $\{h_i\}$ is constructed. Entropy is computed as:

$$H(p) = -\sum_n (p_i \log p_i), \quad p_i = \frac{h_i}{\sum_j h_j}$$

$H(p)$ identifies pixels where local texture ambiguity is highest.

Candidate Selection Algorithm

1. Compute $H(p)$ for every scribble pixel.
2. Let $H_\text{max} = \max_{p \in S} H(p)$. Retain pixels with entropy at least $\tau \cdot H_\text{max}$:

$$C_0 = \{p \in S \mid H(p) \ge \tau \cdot H_\text{max}\}$$

1. Enforce minimum pairwise distance $d_\text{min}$ in $C_0$ to prevent spatial clustering.
2. Apply Farthest-Point Sampling (FPS) to select exactly $N$ prompt points.

This sampling ensures that only the most ambiguous boundaries are queried, avoiding trivial regions and forcing SAM to resolve challenging object characteristics.

3. Multi-Agent Debate Mechanism

After entropy-driven sampling, the selected prompt points are fed into SAM to generate $K$ candidate masks $\{M_1, ..., M_K\}$. Each mask undergoes multimodal Chain-of-Thought (CoT) discourse among three agents:

• Affirmative Debater (A): Advocates for mask validity, citing features such as color coherence, plausible shape, and adherence to scribble coverage.
• Negative Debater (N): Argues against mask validity, referencing deficiencies such as holes, background leaks, and missed object parts.
• Judge (J): Evaluates arguments using support scores and computes a final mask confidence.

Mathematical Formulation

Chains of thought, $D^+_k$ (from A) and $D^-_k$ (from N), are scored by external critics (e.g., LLM-based scoring functions) $f_1$ and $f_2$:

$u_k = f_1(D^+_k), \quad v_k = f_2(D^-_k)$

Judge outputs:

$s_k = \sigma(\alpha u_k - \beta v_k)$

where $\sigma$ is the sigmoid and $\alpha, \beta > 0$ are tunable weights. Masks with $s_k \ge \delta$ are selected, with optional aggregation via union or pixel-wise voting for overlaps.

This mechanism substantially emulates human peer review, enabling the retention of complex cases and filtering of spurious proposals.

4. Incorporation and Regularization via Scribble Annotations

Integration of scribble semantics occurs at both the agent-prompting and mask selection stages.

Agent Prompting

Meta-prompts for each agent comprise a natural-language description of the annotation context (e.g., “these red strokes mark likely foreground”) and illustrative CoT dialogues. This grounds agent arguments in the semantics of the weak supervision.

Scribble Alignment Loss

Masks are penalized for deviation from scribble guidance using:

$$L_{\text{scr}}(M_k) = - \lambda_{fg} \sum_{p \in S_{fg}} \log M_k(p) - \lambda_{bg} \sum_{p \in S_{bg}} \log(1 - M_k(p))$$

where $S_{fg}$ and $S_{bg}$ are foreground and background scribble pixels. $L_\text{scr}$ can regularize mask selection either as a soft constraint on the judge’s decision or as a re-ranking criterion for borderline cases.

5. Algorithmic Workflow

The unified workflow operationalizes the above principles:

def DebateEnhancedPseudoLabeling(I, S): # I: input image, S: sparse scribbles 1. Compute local entropy H(p) for p∈S 2. P ← AEDPS(S, H, τ, d_min, N) # adaptive sampling 3. {M₁,…,M_K} ← SAM(I; prompts=P) # candidate masks 4. Y_pseudo ← ∅ 5. for k in 1…K: A_k, N_k ← Chain-of-Thought debates u_k = score_support(A_k) v_k = score_support(N_k) s_k = sigmoid( α·u_k – β·v_k ) if s_k ≥ δ: compute L_scr(M_k) if L_scr(M_k) ≤ λₛₜₕᵣ: add M_k to Y_pseudo 6. Merge overlapping Y_pseudo masks (optional) 7. return Y_pseudo # refined pseudo masks

Final output: refined pseudo mask set for downstream frequency-aware debiasing.

6. Benefits and Empirical Observations

The Debate-Enhanced Pseudo Labeling approach confers several technical advantages:

• High-entropy point sampling excludes trivial regions, focusing the segmentation model on ambiguous object boundaries.
• Multi-agent CoT debate mechanism advances interpretability and robustness, rescuing hard cases and mitigating against erroneous masks.
• Scribble semantics, integrated through agent meta-prompting and loss regularization, enforce strict compliance with annotation intent.

Empirical evaluation demonstrates lower mean absolute error and higher boundary F-score for mask quality, relative to baselines relying on SAM plus heuristic thresholding. The output pseudo mask set is thus markedly cleaner and more complete, directly improving the reliability of weakly-supervised camouflaged object detection (Ge et al., 23 Dec 2025).

7. Context and Implications

The Debate-Enhanced Pseudo Labeling methodology presents a salient paradigm for mask refinement in sparse annotation settings. By combining entropy-based exploration of the annotation space with agent-driven deliberative validation, this approach bridges longstanding performance gaps between weakly and fully supervised segmentation for highly ambiguous object classes. A plausible implication is the generalizability of multi-agent debate frameworks to other domains in weak supervision beyond COD, contingent on robust grounding in annotation semantics and task-specific mask validation strategies.