---
title: Caption Set Equivalence Reward (CSER)
url: https://www.emergentmind.com/topics/caption-set-equivalence-reward-cser
type: topic
---

# Caption Set Equivalence Reward (CSER)

Searching arXiv for the cited CSER papers and closely related work to ground the article.
arxiv_search.query({"search_query":"all:\"Caption Set Equivalence Reward\" OR ti:\"SC-Captioner\" OR ti:\"OwlCap\"", "start": 0, "max_results": 10, "sort_by": "submittedDate", "sort_order": "descending"})
arxiv_search.search({"query":"Caption Set Equivalence Reward SC-Captioner OwlCap", "max_results": 10})
Caption Set Equivalence Reward (CSER) is a reward design for caption generation that treats captions as sets of semantic elements and evaluates how closely a generated caption matches a reference under a set-equivalence view. In the 2025 literature, the term is used in two closely related but task-specific settings: SC-Captioner formulates CSER for self-correcting image captioning by rewarding accurate additions and removals of objects, attributes, and relations across an initial caption and a corrected caption [2508.06125], while OwlCap formulates CSER for video captioning by decomposing predicted and ground-truth captions into semantic units and enforcing both correctness and completeness through bidirectional validation inside Group Relative Policy Optimization (GRPO) [2508.18634]. Across both formulations, the central objective is to discourage hallucinations and omissions while providing a finer-grained reward signal than sequence-level caption metrics.

## 1. Origin and problem setting

The name CSER is explicitly attached to the reward construction described in "SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning" [2508.06125]. There, the reward is introduced as a correction-based signal for a two-stage captioning process in which a model first produces an initial caption $y_1$ and then a self-corrected caption $y_2$. The reward is built to identify what was added and what was removed during self-correction, and then to determine whether those modifications improved factual alignment with the ground-truth caption $y^*$.

In "OwlCap: Harmonizing Motion-Detail for Video Captioning via HMD-270K and Caption Set Equivalence Reward" [2508.18634], CSER is introduced for video captioning under a different optimization regime. The stated motivation is that video captioning must jointly capture static details and temporal motions, whereas existing methods often overemphasize one aspect and neglect the other. In that formulation, CSER is designed to enforce two desiderata simultaneously: **Correctness**, meaning that every piece of the generated caption should be grounded in the video, and **Completeness**, meaning that every key unit of the ground-truth caption should appear in the generated caption.

A plausible implication is that CSER is best understood not as a single fixed formula but as a reward family organized around the same semantic principle: generated and reference captions should be equivalent at the level of constituent units, subject to the task structure of either self-correction or one-shot video caption generation.

## 2. Set-equivalence principle and semantic decomposition

Both formulations begin by replacing whole-sentence comparison with structured semantic decomposition, but they operationalize that decomposition differently.

In SC-Captioner, each caption $y$ is parsed by the FACTUAL scene-graph parser into three finite sets [2508.06125]:

- $O_y = \{o\}$, the set of object phrases in $y$
- $A_y = \{(o,a)\}$, the set of attribute pairs
- $R_y = \{(o_i, r, o_j)\}$, the set of relation triplets

Applying the same parser to $y_1$, $y_2$, and $y^*$ yields object, attribute, and relation sets for the initial caption, corrected caption, and ground truth. The reward logic then operates on these sets directly.

In OwlCap, the decomposition is performed by an LLM, specifically Qwen3-32B, which splits the predicted caption into units $U_1,\dots,U_n$ and the ground-truth caption into fact units $F_1,\dots,F_m$ [2508.18634]. The paper describes these as “indivisible semantic units,” with the example “A man in a red shirt” being decomposed into $\{$“man,” “red shirt”$\}$.

The two formulations can be contrasted as follows:

| Aspect | SC-Captioner | OwlCap |
|---|---|---|
| Task | Self-correcting image captioning | Video captioning |
| Decomposition | Scene-graph parsing into objects, attributes, relations | LLM-based decomposition into indivisible semantic units |
| Equivalence mechanism | Set-difference plus similarity-based bonuses and penalties | Unit-to-set matching plus bidirectional validation |

This suggests that CSER is agnostic to the particular decomposition mechanism as long as captions can be mapped into units over which equivalence-like constraints can be enforced.

## 3. SC-Captioner formulation: corrections as rewarded set transformations

In SC-Captioner, the reward is explicitly tied to the transition from $y_1$ to $y_2$ [2508.06125]. For objects, the model isolates the newly added and removed elements as
$$
\Delta_{\mathrm{add}}^O = O_{y_2} \setminus O_{y_1}, \qquad
\Delta_{\mathrm{rem}}^O = O_{y_1} \setminus O_{y_2}.
$$
The same pattern is applied to attributes and relations:
$$
\Delta_{\mathrm{add}}^A = A_{y_2}\setminus A_{y_1}, \qquad
\Delta_{\mathrm{rem}}^A = A_{y_1}\setminus A_{y_2},
$$
$$
\Delta_{\mathrm{add}}^R = R_{y_2}\setminus R_{y_1}, \qquad
\Delta_{\mathrm{rem}}^R = R_{y_1}\setminus R_{y_2}.
$$

Each added or removed object is then matched against the reference object set using phrase embeddings from a pretrained Sentence-Transformer and cosine similarity $s(o,o^*) \in [-1,1]$. For added objects, the reward computes the best match of each $o_a \in \Delta_{\mathrm{add}}^O$ in $O_{y^*}$, yielding
$$
S_a^O = \left\{ \max_{o^* \in O_{y^*}} s(o_a,o^*) \mid o_a \in \Delta_{\mathrm{add}}^O \right\}.
$$
For removed objects, it analogously defines
$$
S_r^O = \left\{ \max_{o^* \in O_{y^*}} s(o_r,o^*) \mid o_r \in \Delta_{\mathrm{rem}}^O \right\}.
$$

These scores are converted into a soft reward and a hard reward. With thresholds $\tau_a,\tau'_a$ for additions and $\tau_r,\tau'_r$ for removals, the soft bonus is
$$
B_O^{\mathrm{soft}}
=
\sum_{s \in S_a^O} [s-\tau_a]_+
+
\sum_{s \in S_r^O} [\tau_r-s]_+,
$$
where $[x]_+ = \max(x,0)$. This rewards additions whose similarity exceeds $\tau_a$ and removals whose similarity falls below $\tau_r$. The hard bonus is
$$
B_O^{\mathrm{hard}}
=
\sum_{s \in S_a^O} \mathbf{1}_{s>\tau'_a}
+
\sum_{s \in S_r^O} \mathbf{1}_{s<\tau'_r}.
$$
The total object bonus is then
$$
B_O = B_O^{\mathrm{soft}} + B_O^{\mathrm{hard}}.
$$
The same construction is applied to attributes and relations, with the note that for relations one typically concatenates the triplet into a single string before embedding.

SC-Captioner also introduces explicit mistake punishment. For objects,
$$
\Delta_{\mathrm{bad\_add}}^O = O_{y_2} \setminus (O_{y_1} \cup O_{y^*}),
\qquad
\Delta_{\mathrm{bad\_rem}}^O = (O_{y_1} \cup O_{y^*}) \setminus O_{y_2},
$$
and the penalty is
$$
P_O = |\Delta_{\mathrm{bad\_add}}^O| + |\Delta_{\mathrm{bad\_rem}}^O|.
$$
A corresponding penalty $P_A$ is defined for attributes, while the relation-penalty term is omitted “to avoid over-penalizing minor graph-parsing mismatches” [2508.06125].

The full reward is
$$
R_{\mathrm{CSER}}(y_1,y_2;y^*)
=
[B_O + B_A + B_R] - [P_O + P_A].
$$
More explicitly, the paper gives
$$
R_{\mathrm{CSER}}
=
\bigl(B_O^{\mathrm{soft}}+B_O^{\mathrm{hard}}\bigr)
+
\bigl(B_A^{\mathrm{soft}}+B_A^{\mathrm{hard}}\bigr)
+
B_R
-
\bigl(|\Delta_{\mathrm{bad\_add}}^O|+|\Delta_{\mathrm{bad\_rem}}^O|\bigr)
-
\bigl(|\Delta_{\mathrm{bad\_add}}^A|+|\Delta_{\mathrm{bad\_rem}}^A|\bigr).
$$

The significance of this construction is explicit in the paper: it “directly rewards adding missing objects/attributes/relations and removing hallucinations, and it punishes the reverse mistakes” [2508.06125]. A plausible implication is that SC-Captioner’s CSER targets *edit quality* rather than caption quality in the abstract, because the reward depends on the delta between two model outputs.

## 4. OwlCap formulation: unit-to-set matching and bidirectional validation

OwlCap defines CSER in terms of semantic units extracted from the predicted caption and fact units extracted from the ground-truth caption [2508.18634]. Given predicted units $U_1,\dots,U_n$ and ground-truth fact units $F_1,\dots,F_m$, it introduces a binary matching function
$$
\mathbb{I}(u \in C) \in \{0,1\},
$$
which returns 1 if unit $u$ is judged by the same LLM to be semantically contained in the caption string $C$, and 0 otherwise.

From this, OwlCap defines the **Correctness score**
$$
S_{\mathrm{correctness}}
=
\frac{1}{n}\sum_{i=1}^n \mathbb{I}(U_i \in C_{\mathrm{gt}}),
$$
and the **Completeness score**
$$
S_{\mathrm{completeness}}
=
\frac{1}{m}\sum_{j=1}^m \mathbb{I}(F_j \in C_{\mathrm{pred}}).
$$
It then adds a small format term $S_{\mathrm{format}}$ and sets the scalar reward to
$$
r = S_{\mathrm{format}} + S_{\mathrm{correctness}} + S_{\mathrm{completeness}}.
$$

The paper characterizes this as a unit-to-set matching and bidirectional validation mechanism [2508.18634]. The forward direction checks whether each generated unit $U_i$ is semantically contained in the ground-truth caption, corresponding to correctness. The backward direction checks whether each ground-truth fact $F_j$ is semantically contained in the predicted caption, corresponding to completeness. The paper states that together these checks approximate “the notion of two sets being equivalent up to semantic rewriting.”

The motivation is also stated with unusual clarity. Standard RL-style rewards in captioning—BLEU, CIDEr, or single-sided event-coverage scores—are described as either insensitive to whether all facts have been covered or limited to checking only one direction. CSER is intended to fill that gap by enforcing that every generated unit must match something in the ground-truth set and every ground-truth unit must match something in the predicted set [2508.18634].

A plausible implication is that OwlCap’s version is closer to a semantic coverage reward, whereas SC-Captioner’s version is closer to a semantics-aware edit reward. The commonality lies in the bidirectional pressure against both hallucination and omission.

## 5. Optimization regimes and learning dynamics

The two CSER formulations are embedded in different reinforcement-learning updates.

In SC-Captioner, CSER is used “as the scalar reward on the self-correction turn in a standard KL-controlled policy-gradient update” [2508.06125]. The procedure is:

- sample an initial caption $y_1 \sim \pi_\theta(\cdot \mid [I,x_1])$
- sample the corrected caption $y_2 \sim \pi_\theta(\cdot \mid [I,x_1,y_1,x_2])$
- compute $R := R_{\mathrm{CSER}}(y_1,y_2;y^*)$
- update $\theta$ by minimizing
$$
L(\theta)
=
-R \log \pi_\theta(y_2 \mid [I,x_1,y_1,x_2])
+
\beta\,D_{\mathrm{KL}}\bigl(\pi_\theta(\cdot \mid [I,x_1]) \;\|\; \pi_{\mathrm{ref}}(\cdot \mid [I,x_1])\bigr)
$$

The first term increases the probability of self-corrections that achieve high CSER, and the second term anchors the one-turn distribution to a frozen reference policy $\pi_{\mathrm{ref}}$.

In OwlCap, CSER is integrated into GRPO [2508.18634]. For each video $x$, the method samples a group of $G$ captions $\{y_1,\dots,y_G\}$ from the current policy $\pi_{\theta_{\mathrm{old}}}$. Each caption receives a scalar reward $r_i$ from CSER, and the within-group advantage is
$$
A_i =
\frac{r_i - \mathrm{mean}(\{r_1,\dots,r_G\})}{\mathrm{std}(\{r_1,\dots,r_G\})}.
$$
The parameter update maximizes a clipped surrogate objective plus a KL penalty to a reference policy $\pi_{\mathrm{ref}}$:
$$
\mathcal{J}_{\mathrm{GRPO}}(\theta)
=
\mathbb{E}_{x,\{y_i\}}
\Biggl[
\frac{1}{G}\sum_{i=1}^G
\min\Bigl(
\frac{\pi_\theta(y_i|x)}{\pi_{\theta_{\mathrm{old}}}(y_i|x)}A_i,\,
\mathrm{clip}\bigl(\tfrac{\pi_\theta}{\pi_{\theta_{\mathrm{old}}}},1-\epsilon,1+\epsilon\bigr)A_i
\Bigr)
-\beta\,D_{\mathrm{KL}}\bigl(\pi_\theta(\cdot|x)\|\pi_{\mathrm{ref}}(\cdot|x)\bigr)
\Biggr].
$$

These optimization choices reflect the tasks themselves. SC-Captioner optimizes a second-turn correction conditioned on the first-turn output, while OwlCap scores multiple candidate captions for the same video and normalizes rewards within each group. This suggests that CSER is compatible with both single-sample policy-gradient updates and group-relative updates, provided that the reward remains scalar at the policy interface.

## 6. Implementation details, empirical results, and scope

The implementation choices reported for OwlCap are unusually specific [2508.18634]. The base policy is Qwen2.5-VL-7B. Supervised fine-tuning uses one epoch on HMD-270K with learning rate $1\mathrm{e}{-5}$ and batch size 64. RL uses group size $G=8$, top-variance subset = 50% of staged samples, $\beta=0.001$ for the KL penalty, $\epsilon=0.2$ for clipping, RL batch size = 24, and training time of approximately 35 hours on $32\times \mathrm{H20}$ GPUs. The judge model for unit decomposition and matching is Qwen3-32B. To speed matching, ground-truth units are preprocessed once and cached, while predicted units and matching use GPU-accelerated LLM inference.

The empirical gains reported for OwlCap isolate the contribution of CSER [2508.18634]:

| Setting | VDC | DREAM-1K |
|---|---:|---:|
| SFT only | +2.1 Acc | +2.0 F1 |
| RL with only Completeness score | +1.4 Acc | +2.6 F1 |
| RL with only Correctness score | +1.0 Acc | +1.6 F1 |
| RL with both (CSER) on SFT-initialized model | +4.2 Acc | +4.6 F1 |

Under identical settings, the paper also reports that, compared with VideoCap-R1’s Event Score reward, CSER yields +1.7 Acc more on VDC and +1.4 F1 more on DREAM-1K [2508.18634]. The abstract summarizes the benchmark improvements as significant gains on the detail-focused VDC and the motion-focused DREAM-1K.

For SC-Captioner, the abstract states that applying the framework on large visual-language models can generate better image captions across various scenarios and “significantly outperform” the direct preference optimization training strategy [2508.06125]. The same paper also introduces a set of metrics refined from CAPTURE to alleviate incomplete precision evaluation and inefficient relation matching problems, and it reports the collection of RefinedCaps, a fine-grained annotated image caption dataset consisting of 6.5K diverse images from the COCO dataset [2508.06125]. These elements are not themselves part of CSER, but they define the evaluation and data context in which the reward was developed.

A common misconception would be to treat CSER as a single immutable objective. The available literature instead shows two task-adapted instantiations under the same name: one centered on set-difference over scene-graph elements in self-correction [2508.06125], and one centered on unit-to-set matching with bidirectional validation in GRPO-based video captioning [2508.18634]. What remains invariant is the core criterion of semantic equivalence under penalties for hallucinated additions and omitted reference content.

Source: https://www.emergentmind.com/topics/caption-set-equivalence-reward-cser