Cross Label Propagation (CLP) in RGBD Co-saliency

• CLP is a graph-based optimization method that cross-seeds inter- and intra-saliency maps in RGBD co-saliency detection.
• It builds superpixel graphs using color and depth cues with adaptive thresholds to select high-confidence seeds for refinement.
• The technique improves object boundaries and reduces false positives, yielding significant gains in F-measure and MAE on benchmark datasets.

Searching arXiv for the specified paper and closely related co-saliency work to ground the article in published sources. Cross-Level Propagation, abbreviated CLP, refers in the RGBD co-saliency literature of "Co-saliency Detection for RGBD Images Based on Multi-constraint Feature Matching and Cross Label Propagation" to Cross Label Propagation, a graph-based optimization scheme for jointly refining intra- and inter-saliency maps by cross-using them as seeds. Within that pipeline, intra-saliency denotes the saliency of each RGBD image independently, obtained from a single-image RGBD saliency detector, whereas inter-saliency denotes a co-saliency cue computed from multi-constraint feature matching across an image group. CLP operates after both maps have been computed at the superpixel level, and its defining property is cross-seeding: the inter-saliency map is used to refine the intra-saliency map, and the intra-saliency map is used to refine the inter-saliency map (Cong et al., 2017). The terminology matters: the method is explicitly cross label, not cross-level in the sense of multi-scale propagation, although the acronym CLP can invite that confusion.

1. Position within RGBD co-saliency detection

The underlying task is co-saliency detection, which aims at extracting the common salient regions from an image group containing two or more relevant images. The method in (Cong et al., 2017) is formulated for RGBD image groups and introduces depth information at multiple stages: in single-image saliency, in cross-image feature matching, and in graph propagation.

The full workflow consists of four stages. First, an intra-saliency map is generated for each image by a single-image RGBD saliency model, specifically DCMC [36] in the reported implementation. Second, an inter-saliency map is computed using multi-constraint feature matching across the image group. Third, CLP is applied as an optimization stage to refine intra- and inter-saliency in a cross way. Fourth, the original and optimized maps are integrated to produce the final co-saliency result (Cong et al., 2017).

This organization gives CLP a narrow but structurally important role. It does not create cross-image correspondence by itself; that coupling is encoded upstream in the inter-saliency maps. Nor is it the final decision rule; the output of CLP is fused with the original maps. Its function is instead to bridge the gap between within-image distinctness and across-image commonality by allowing the two saliency fields to regularize one another.

2. Graph construction and propagation model

CLP is defined on a per-image superpixel graph. For each image $I^i$ with depth $D^i$, SLIC segmentation yields superpixels

$$R^i = \{r_m^i\}_{m=1}^{N_i}, \quad N_i \approx 200.$$

A graph

$G^i = (V^i, E^i)$

is then constructed, where nodes correspond to superpixels and edges connect adjacent superpixels. The adjacency set of node $u$ is denoted by $\Psi_u^i$ (Cong et al., 2017).

The affinity matrix $W^i = [w_{uv}^i]_{N_i \times N_i}$ encodes local color and depth similarity: $w_{uv}^{i}= \begin{cases} \exp\!\left( -\dfrac{ \left\|c_{u}^{i}-c_{v}^{i}\right\|_{2} +\lambda_{d}^{i}\cdot\left|d_{u}^{i}-d_{v}^{i}\right| }{\sigma^{2} } \right), & \text{if } v\in\Psi_{u}^{i},\[1ex] 0, & \text{otherwise}, \end{cases} \tag{11}$ where $c_u^i$ is the mean L$^*$a$D^i$0b$D^i$1 color of superpixel $D^i$2, $D^i$3 is its mean depth, $D^i$4 is a depth confidence term, and $D^i$5 is fixed to $D^i$6. Similar neighboring superpixels therefore receive large weights, yielding a local RGBD smoothness prior.

The depth confidence term is

$D^i$7

with $D^i$8 the mean depth value, $D^i$9 the standard deviation, $$R^i = \{r_m^i\}_{m=1}^{N_i}, \quad N_i \approx 200.$$0, and

$$R^i = \{r_m^i\}_{m=1}^{N_i}, \quad N_i \approx 200.$$1

Large $$R^i = \{r_m^i\}_{m=1}^{N_i}, \quad N_i \approx 200.$$2 indicates reliable depth and increases the contribution of depth in the affinity definition (Cong et al., 2017).

The propagation itself is deliberately simple. Once an initialization $$R^i = \{r_m^i\}_{m=1}^{N_i}, \quad N_i \approx 200.$$3 has been specified from cross-seeds, the optimized intra-saliency values are obtained by

$$R^i = \{r_m^i\}_{m=1}^{N_i}, \quad N_i \approx 200.$$4

followed by min-max normalization to $$R^i = \{r_m^i\}_{m=1}^{N_i}, \quad N_i \approx 200.$$5. The inter-saliency refinement uses the same form. The paper does not derive an explicit energy or normalized Laplacian; the behavior is instead that of a one-step weighted diffusion on the superpixel graph (Cong et al., 2017).

A plausible implication is that CLP occupies an intermediate space between classical label propagation and lightweight random-walk smoothing. It inherits graph-based locality and boundary sensitivity through $$R^i = \{r_m^i\}_{m=1}^{N_i}, \quad N_i \approx 200.$$6, but it realizes cross-map coupling through initialization rather than through a jointly optimized objective.

3. Cross-seeding, thresholds, and refinement logic

The defining feature of CLP is its seed selection mechanism. To refine one saliency field, the method uses the other saliency field to create high-confidence foreground and background seeds, while preserving the current map as soft labels on uncertain nodes.

Given a saliency map $$R^i = \{r_m^i\}_{m=1}^{N_i}, \quad N_i \approx 200.$$7 on image $$R^i = \{r_m^i\}_{m=1}^{N_i}, \quad N_i \approx 200.$$8, the adaptive thresholds are

$$R^i = \{r_m^i\}_{m=1}^{N_i}, \quad N_i \approx 200.$$9

$G^i = (V^i, E^i)$0

with $G^i = (V^i, E^i)$1 and $G^i = (V^i, E^i)$2. Foreground seeds are superpixels with $G^i = (V^i, E^i)$3, and background seeds are superpixels with $G^i = (V^i, E^i)$4 (Cong et al., 2017).

For refining intra-saliency using CLP, the initialization is

$G^i = (V^i, E^i)$5

Thus, strongly inter-salient regions are treated as hard foreground seeds, strongly non-inter-salient regions as hard background seeds, and all remaining regions retain their original intra-saliency values as soft labels. Refinement of inter-saliency is defined symmetrically, with the roles of intra- and inter-saliency reversed (Cong et al., 2017).

The paper states that the inter saliency map is firstly optimized using the intra saliency map in CLP, and then the optimized inter saliency map is used to update the intra saliency map, since the inter saliency is generated by the intra saliency map. This yields the operational order:

1. optimize $G^i = (V^i, E^i)$6 using intra-saliency as seeds to obtain $G^i = (V^i, E^i)$7;
2. optimize $G^i = (V^i, E^i)$8 using $G^i = (V^i, E^i)$9 as seeds to obtain $u$0.

The comparison with alternative propagation schemes clarifies the intended semantics.

Scheme	Seed source	Refinement pattern
LP	each map itself	self-refinement
SLP	joint intra+inter map	shared-seed refinement
CLP	the other map	cross-coupled refinement

CLP is therefore not merely a smoothing stage. Its core assumption is that intra- and inter-saliency exhibit complementary failure modes: intra-saliency may retain image-specific distractors, whereas inter-saliency may suppress non-common background but miss object parts. Cross-seeding is designed to exploit this complementarity rather than refining each map in isolation (Cong et al., 2017).

4. Inter-saliency as the upstream source of cross-image information

Although CLP is local to each image graph, the information it propagates is conditioned by cross-image correspondence, because inter-saliency is computed from multi-constraint feature matching across the image group. This is the stage at which group structure enters the pipeline most explicitly (Cong et al., 2017).

For each pair of images $u$1, superpixels $u$2 and $u$3 are matched under three constraints. The first is a color+depth similarity constraint: $u$4 For each superpixel, the $u$5 nearest neighbors in each other image under this similarity form the set $u$6. The second is a saliency consistency constraint: $u$7 with $u$8. The third is a cluster-based constraint, in which each image is partitioned into $u$9 clusters by K-means++ over superpixel features and candidate matches are constrained through nearest cross-image cluster centers: $\Psi_u^i$0 The final binary match indicator is

$\Psi_u^i$1

Image-level similarity modulates the contribution of each paired image. Each image is represented by hybrid features: RGB histogram $\Psi_u^i$2, texton histogram $\Psi_u^i$3, semantic CNN feature $\Psi_u^i$4 from VGG16 fc7, GIST feature $\Psi_u^i$5, depth histogram $\Psi_u^i$6, and saliency histogram $\Psi_u^i$7. The combined similarity is

$\Psi_u^i$8

with

$\Psi_u^i$9

$W^i = [w_{uv}^i]_{N_i \times N_i}$0

and $W^i = [w_{uv}^i]_{N_i \times N_i}$1. Noisy depth therefore reduces the weight of the depth channel and shifts emphasis toward color and saliency features (Cong et al., 2017).

Given matches and image similarity, inter-saliency is computed as

$W^i = [w_{uv}^i]_{N_i \times N_i}$2

Inter-saliency is therefore the average intra-saliency of matched counterparts in other images, weighted by image similarity. In conceptual terms, CLP can be understood as a local graph diffusion whose seed prior is already informed by a cross-image consensus mechanism.

5. Fusion strategy, depth usage, and practical properties

After cross refinement, the final co-saliency map is obtained by fusing the original and refined maps: $W^i = [w_{uv}^i]_{N_i \times N_i}$3 subject to $W^i = [w_{uv}^i]_{N_i \times N_i}$4, with $W^i = [w_{uv}^i]_{N_i \times N_i}$5 in the experiments. The final result is thus a simple average of the initial intra-saliency, the initial inter-saliency, the refined intra-saliency, and the refined inter-saliency (Cong et al., 2017).

Depth information is used at three distinct points. It contributes to the single-image RGBD saliency model that produces intra-saliency. It enters cross-image matching through the color+depth similarity term $W^i = [w_{uv}^i]_{N_i \times N_i}$6 and through the image-level depth histogram distance $W^i = [w_{uv}^i]_{N_i \times N_i}$7. It also guides graph propagation through the affinity matrix $W^i = [w_{uv}^i]_{N_i \times N_i}$8. The influence of depth is moderated by the depth confidence $W^i = [w_{uv}^i]_{N_i \times N_i}$9 and the adaptive image-level weight $w_{uv}^{i}= \begin{cases} \exp\!\left( -\dfrac{ \left\|c_{u}^{i}-c_{v}^{i}\right\|_{2} +\lambda_{d}^{i}\cdot\left|d_{u}^{i}-d_{v}^{i}\right| }{\sigma^{2} } \right), & \text{if } v\in\Psi_{u}^{i},\[1ex] 0, & \text{otherwise}, \end{cases} \tag{11}$0, which reduces depth’s role when the depth maps are unreliable (Cong et al., 2017).

The reported properties of CLP follow directly from this design. Because propagation is local over adjacent superpixels, object continuity can be improved by filling gaps in noisy saliency maps. Because seed selection is cross-coupled, common background is less likely to be reinforced if it lacks support in the inter-saliency field. Because the propagation step is a single sparse matrix-vector multiplication per image, CLP itself is computationally light; the paper reports that CLP takes only 0.20% of the runtime per image, while most time is spent in inter-saliency matching. Parameter settings include $w_{uv}^{i}= \begin{cases} \exp\!\left( -\dfrac{ \left\|c_{u}^{i}-c_{v}^{i}\right\|_{2} +\lambda_{d}^{i}\cdot\left|d_{u}^{i}-d_{v}^{i}\right| }{\sigma^{2} } \right), & \text{if } v\in\Psi_{u}^{i},\[1ex] 0, & \text{otherwise}, \end{cases} \tag{11}$1, $w_{uv}^{i}= \begin{cases} \exp\!\left( -\dfrac{ \left\|c_{u}^{i}-c_{v}^{i}\right\|_{2} +\lambda_{d}^{i}\cdot\left|d_{u}^{i}-d_{v}^{i}\right| }{\sigma^{2} } \right), & \text{if } v\in\Psi_{u}^{i},\[1ex] 0, & \text{otherwise}, \end{cases} \tag{11}$2, 200 SLIC superpixels per image, $w_{uv}^{i}= \begin{cases} \exp\!\left( -\dfrac{ \left\|c_{u}^{i}-c_{v}^{i}\right\|_{2} +\lambda_{d}^{i}\cdot\left|d_{u}^{i}-d_{v}^{i}\right| }{\sigma^{2} } \right), & \text{if } v\in\Psi_{u}^{i},\[1ex] 0, & \text{otherwise}, \end{cases} \tag{11}$3 neighbors for matching, and $w_{uv}^{i}= \begin{cases} \exp\!\left( -\dfrac{ \left\|c_{u}^{i}-c_{v}^{i}\right\|_{2} +\lambda_{d}^{i}\cdot\left|d_{u}^{i}-d_{v}^{i}\right| }{\sigma^{2} } \right), & \text{if } v\in\Psi_{u}^{i},\[1ex] 0, & \text{otherwise}, \end{cases} \tag{11}$4 clusters (Cong et al., 2017).

A common misconception is to read CLP as a generic propagation method over multiple representation levels. In this formulation, however, the relevant distinction is not level but label field: the method couples two saliency maps with different semantics defined on the same superpixel graph.

6. Empirical evaluation and relation to adjacent methods

The method is evaluated on two RGBD co-saliency datasets: RGBD Coseg183, consisting of 183 RGBD images in 16 groups from indoor scenes, and RGBD Cosal150, consisting of 150 images in 21 groups from the NJU-1985 RGBD saliency dataset (Cong et al., 2017). Performance is assessed by Precision-Recall curves, F-measure,

$w_{uv}^{i}= \begin{cases} \exp\!\left( -\dfrac{ \left\|c_{u}^{i}-c_{v}^{i}\right\|_{2} +\lambda_{d}^{i}\cdot\left|d_{u}^{i}-d_{v}^{i}\right| }{\sigma^{2} } \right), & \text{if } v\in\Psi_{u}^{i},\[1ex] 0, & \text{otherwise}, \end{cases} \tag{11}$5

and MAE,

$w_{uv}^{i}= \begin{cases} \exp\!\left( -\dfrac{ \left\|c_{u}^{i}-c_{v}^{i}\right\|_{2} +\lambda_{d}^{i}\cdot\left|d_{u}^{i}-d_{v}^{i}\right| }{\sigma^{2} } \right), & \text{if } v\in\Psi_{u}^{i},\[1ex] 0, & \text{otherwise}, \end{cases} \tag{11}$6

Comparisons include single-image saliency methods HS, BSCA, RC, DRFI, ACSD, and DCMC; RGB co-saliency methods CCS, SCS, and LRMF; and ablations in which the same framework is paired with LP, SLP, or CLP. The principal findings are that inter-saliency alone already outperforms existing co-saliency baselines, that adding optimization improves performance further, and that CLP outperforms both LP and SLP. On RGBD Cosal150, CLP achieves $w_{uv}^{i}= \begin{cases} \exp\!\left( -\dfrac{ \left\|c_{u}^{i}-c_{v}^{i}\right\|_{2} +\lambda_{d}^{i}\cdot\left|d_{u}^{i}-d_{v}^{i}\right| }{\sigma^{2} } \right), & \text{if } v\in\Psi_{u}^{i},\[1ex] 0, & \text{otherwise}, \end{cases} \tag{11}$7 and $w_{uv}^{i}= \begin{cases} \exp\!\left( -\dfrac{ \left\|c_{u}^{i}-c_{v}^{i}\right\|_{2} +\lambda_{d}^{i}\cdot\left|d_{u}^{i}-d_{v}^{i}\right| }{\sigma^{2} } \right), & \text{if } v\in\Psi_{u}^{i},\[1ex] 0, & \text{otherwise}, \end{cases} \tag{11}$8, whereas SLP is slightly worse with $w_{uv}^{i}= \begin{cases} \exp\!\left( -\dfrac{ \left\|c_{u}^{i}-c_{v}^{i}\right\|_{2} +\lambda_{d}^{i}\cdot\left|d_{u}^{i}-d_{v}^{i}\right| }{\sigma^{2} } \right), & \text{if } v\in\Psi_{u}^{i},\[1ex] 0, & \text{otherwise}, \end{cases} \tag{11}$9. On RGBD Coseg183, CLP gives the best MAE, $c_u^i$0, and a strong F-measure of $c_u^i$1. Percentage gains are reported as up to 30.67% improvement in F-measure and up to 46.00% reduction in MAE in some comparisons (Cong et al., 2017).

Qualitative examples in the paper show a characteristic pattern: CLP suppresses false positives such as sky regions and clutter while filling missing object parts, leading to cleaner boundaries and more uniform saliency on common objects. This is consistent with the stated rationale that intra-saliency captures within-image distinctness but may include non-common distractors, whereas inter-saliency captures across-image commonality but may be brittle when correspondence is imperfect (Cong et al., 2017).

Relative to adjacent graph-based and refinement methods, the distinctive aspect of CLP is that it operates on two related label fields on the same graph. Standard label propagation or random walks typically refine a single field from its own seeds, and CRF-based refinement typically combines a unary saliency map with pairwise smoothness terms. CLP instead uses one saliency field as the seed prior for the other. This suggests a broader interpretation of the method as a lightweight coupled label propagation scheme in which mutual regularization is realized through cross-seeding rather than through an explicitly optimized joint energy.