Caption Set Equivalence Reward (CSER)

• CSER is a reward mechanism that treats captions as sets of semantic elements, ensuring factual alignment through set-difference evaluation.
• It adapts task-specific designs in SC-Captioner and OwlCap to enforce both correctness and completeness in captioning processes.
• The framework integrates semantic decomposition methods—via scene-graph parsing or LLM-based extraction—with reinforcement learning for improved caption quality.

Searching arXiv for the cited CSER papers and closely related work to ground the article. arxiv_search.query({"^^^^^^^^4search_query4 Set Equivalence Reward\"^^^^^^^^4 OR ti:\4^^^^^^^^"SC-Captioner\" OR ti:\4^^^^^^^^"OwlCap\"", "start": ^^^^4search_query4^^^^, "max_results": ^^^^4all:\4search_query4^^^^, "sort_by": "submittedDate", "sort_order": "descending"}) arxiv_search.search({"query":"Caption Set Equivalence Reward SC-Captioner OwlCap", "max_results": ^^^^4all:\4search_query4^^^^}) Caption Set Equivalence Reward (CSER) is a reward design for caption generation that treats captions as sets of semantic elements and evaluates how closely a generated caption matches a reference under a set-equivalence view. In the ^^^^4 OR ti:\4search_query4 OR ti:\4^^^^5 literature, the term is used in two closely related but task-specific settings: SC-Captioner formulates CSER for self-correcting image captioning by rewarding accurate additions and removals of objects, attributes, and relations across an initial caption and a corrected caption (&&&^^^^4search_query4^^^^&&&), while OwlCap formulates CSER for video captioning by decomposing predicted and ground-truth captions into semantic units and enforcing both correctness and completeness through bidirectional validation inside Group Relative Policy Optimization (GRPO) (&&&^^^^4all:\4^^^^&&&). Across both formulations, the central objective is to discourage hallucinations and omissions while providing a finer-grained reward signal than sequence-level caption metrics.

^^^^4all:\4^^^^. Origin and problem setting

The name CSER is explicitly attached to the reward construction described in "SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning" (&&&^^^^4search_query4^^^^&&&). There, the reward is introduced as a correction-based signal for a two-stage captioning process in which a model first produces an initial caption PRESERVED_PLACEHOLDER_^^^^4search_query4^^^^ and then a self-corrected caption PRESERVED_PLACEHOLDER_^^^^4all:\4^^^^. The reward is built to identify what was added and what was removed during self-correction, and then to determine whether those modifications improved factual alignment with the ground-truth caption PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4^^^^.

In "OwlCap: Harmonizing Motion-Detail for Video Captioning via HMD-^^^^4 OR ti:\4sort_by4search_query4^^^^K and Caption Set Equivalence Reward" (&&&^^^^4all:\4^^^^&&&), CSER is introduced for video captioning under a different optimization regime. The stated motivation is that video captioning must jointly capture static details and temporal motions, whereas existing methods often overemphasize one aspect and neglect the other. In that formulation, CSER is designed to enforce two desiderata simultaneously: Correctness, meaning that every piece of the generated caption should be grounded in the video, and Completeness, meaning that every key unit of the ground-truth caption should appear in the generated caption.

A plausible implication is that CSER is best understood not as a single fixed formula but as a reward family organized around the same semantic principle: generated and reference captions should be equivalent at the level of constituent units, subject to the task structure of either self-correction or one-shot video caption generation.

^^^^4 OR ti:\4^^^^. Set-equivalence principle and semantic decomposition

Both formulations begin by replacing whole-sentence comparison with structured semantic decomposition, but they operationalize that decomposition differently.

In SC-Captioner, each caption PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4^^^^ is parsed by the FACTUAL scene-graph parser into three finite sets (&&&^^^^4search_query4^^^^&&&):

• $O_y = \{o\}$, the set of object phrases in $y$
• $A_y = \{(o,a)\}$, the set of attribute pairs
• $R_y = \{(o_i, r, o_j)\}$, the set of relation triplets

Applying the same parser to $y_1$, $y_2$, and PRESERVED_PLACEHOLDER_^^^^4all:\4search_query4^^^^ yields object, attribute, and relation sets for the initial caption, corrected caption, and ground truth. The reward logic then operates on these sets directly.

In OwlCap, the decomposition is performed by an LLM, specifically Qwen^^^^4 OR ti:\4^^^^-^^^^4 OR ti:\4 OR ti:\4^^^^B, which splits the predicted caption into units PRESERVED_PLACEHOLDER_^^^^4all:\4all:\4^^^^ and the ground-truth caption into fact units PRESERVED_PLACEHOLDER_^^^^4all:\4 OR ti:\4^^^^ (&&&^^^^4all:\4^^^^&&&). The paper describes these as “indivisible semantic units,” with the example “A man in a red shirt” being decomposed into PRESERVED_PLACEHOLDER_^^^^4all:\4 OR ti:\4^^^^“man,” “red shirt”PRESERVED_PLACEHOLDER_^^^^4all:\44^^^^.

The two formulations can be contrasted as follows:

Aspect	SC-Captioner	OwlCap
Task	Self-correcting image captioning	Video captioning
Decomposition	Scene-graph parsing into objects, attributes, relations	LLM-based decomposition into indivisible semantic units
Equivalence mechanism	Set-difference plus similarity-based bonuses and penalties	Unit-to-set matching plus bidirectional validation

This suggests that CSER is agnostic to the particular decomposition mechanism as long as captions can be mapped into units over which equivalence-like constraints can be enforced.

^^^^4 OR ti:\4^^^^. SC-Captioner formulation: corrections as rewarded set transformations

In SC-Captioner, the reward is explicitly tied to the transition from PRESERVED_PLACEHOLDER_^^^^4all:\4^^^^5 to PRESERVED_PLACEHOLDER_^^^^4all:\4^^^^6 (&&&^^^^4search_query4^^^^&&&). For objects, the model isolates the newly added and removed elements as

PRESERVED_PLACEHOLDER_^^^^4all:\4^^^^7

The same pattern is applied to attributes and relations:

PRESERVED_PLACEHOLDER_^^^^4all:\4^^^^8

PRESERVED_PLACEHOLDER_^^^^4all:\4^^^^9

Each added or removed object is then matched against the reference object set using phrase embeddings from a pretrained Sentence-Transformer and cosine similarity PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4search_query4^^^^. For added objects, the reward computes the best match of each PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4all:\4^^^^ in PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4 OR ti:\4^^^^, yielding

PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4 OR ti:\4^^^^

For removed objects, it analogously defines

PRESERVED_PLACEHOLDER_^^^^4 OR ti:\44^^^^

These scores are converted into a soft reward and a hard reward. With thresholds PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4^^^^5 for additions and PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4^^^^6 for removals, the soft bonus is

PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4^^^^7

where PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4^^^^8. This rewards additions whose similarity exceeds PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4^^^^9 and removals whose similarity falls below PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4search_query4^^^^. The hard bonus is

PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4all:\4^^^^

The total object bonus is then

PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4 OR ti:\4^^^^

The same construction is applied to attributes and relations, with the note that for relations one typically concatenates the triplet into a single string before embedding.

SC-Captioner also introduces explicit mistake punishment. For objects,

PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4 OR ti:\4^^^^

and the penalty is

PRESERVED_PLACEHOLDER_^^^^4 OR ti:\44^^^^

A corresponding penalty PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4^^^^5 is defined for attributes, while the relation-penalty term is omitted “to avoid over-penalizing minor graph-parsing mismatches” (&&&^^^^4search_query4^^^^&&&).

The full reward is

PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4^^^^6

More explicitly, the paper gives

PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4^^^^7

The significance of this construction is explicit in the paper: it “directly rewards adding missing objects/attributes/relations and removing hallucinations, and it punishes the reverse mistakes” (&&&^^^^4search_query4^^^^&&&). A plausible implication is that SC-Captioner’s CSER targets edit quality rather than caption quality in the abstract, because the reward depends on the delta between two model outputs.

4. OwlCap formulation: unit-to-set matching and bidirectional validation

OwlCap defines CSER in terms of semantic units extracted from the predicted caption and fact units extracted from the ground-truth caption (&&&^^^^4all:\4^^^^&&&). Given predicted units PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4^^^^8 and ground-truth fact units PRESERVED_PLACEHOLDER_^^^^4 OR ti:\4^^^^9, it introduces a binary matching function

$O_y = \{o\}$^^^^4search_query4^^^^

which returns ^^^^4all:\4^^^^ if unit $O_y = \{o\}$^^^^4all:\4^^^^ is judged by the same LLM to be semantically contained in the caption string $O_y = \{o\}$^^^^4 OR ti:\4^^^^, and ^^^^4search_query4^^^^ otherwise.

From this, OwlCap defines the Correctness score

$O_y = \{o\}$^^^^4 OR ti:\4^^^^

and the Completeness score

$O_y = \{o\}$4

It then adds a small format term $O_y = \{o\}$5 and sets the scalar reward to

$O_y = \{o\}$6

The paper characterizes this as a unit-to-set matching and bidirectional validation mechanism (&&&^^^^4all:\4^^^^&&&). The forward direction checks whether each generated unit $O_y = \{o\}$7 is semantically contained in the ground-truth caption, corresponding to correctness. The backward direction checks whether each ground-truth fact $O_y = \{o\}$8 is semantically contained in the predicted caption, corresponding to completeness. The paper states that together these checks approximate “the notion of two sets being equivalent up to semantic rewriting.”

The motivation is also stated with unusual clarity. Standard RL-style rewards in captioning—BLEU, CIDEr, or single-sided event-coverage scores—are described as either insensitive to whether all facts have been covered or limited to checking only one direction. CSER is intended to fill that gap by enforcing that every generated unit must match something in the ground-truth set and every ground-truth unit must match something in the predicted set (&&&^^^^4all:\4^^^^&&&).

A plausible implication is that OwlCap’s version is closer to a semantic coverage reward, whereas SC-Captioner’s version is closer to a semantics-aware edit reward. The commonality lies in the bidirectional pressure against both hallucination and omission.

5. Optimization regimes and learning dynamics

The two CSER formulations are embedded in different reinforcement-learning updates.

In SC-Captioner, CSER is used “as the scalar reward on the self-correction turn in a standard KL-controlled policy-gradient update” (&&&^^^^4search_query4^^^^&&&). The procedure is:

• sample an initial caption $O_y = \{o\}$9
• sample the corrected caption $y$^^^^4search_query4^^^^
• compute $y$^^^^4all:\4^^^^
• update $y$^^^^4 OR ti:\4^^^^ by minimizing

$y$^^^^4 OR ti:\4^^^^

The first term increases the probability of self-corrections that achieve high CSER, and the second term anchors the one-turn distribution to a frozen reference policy $y$4.

In OwlCap, CSER is integrated into GRPO (&&&^^^^4all:\4^^^^&&&). For each video $y$5, the method samples a group of $y$6 captions $y$7 from the current policy $y$8. Each caption receives a scalar reward $y$9 from CSER, and the within-group advantage is

$A_y = \{(o,a)\}$^^^^4search_query4^^^^

The parameter update maximizes a clipped surrogate objective plus a KL penalty to a reference policy $A_y = \{(o,a)\}$^^^^4all:\4^^^^:

$A_y = \{(o,a)\}$^^^^4 OR ti:\4^^^^

These optimization choices reflect the tasks themselves. SC-Captioner optimizes a second-turn correction conditioned on the first-turn output, while OwlCap scores multiple candidate captions for the same video and normalizes rewards within each group. This suggests that CSER is compatible with both single-sample policy-gradient updates and group-relative updates, provided that the reward remains scalar at the policy interface.

6. Implementation details, empirical results, and scope

The implementation choices reported for OwlCap are unusually specific (&&&^^^^4all:\4^^^^&&&). The base policy is Qwen^^^^4 OR ti:\4^^^^.5-VL-7B. Supervised fine-tuning uses one epoch on HMD-^^^^4 OR ti:\4sort_by4search_query4^^^^K with learning rate $A_y = \{(o,a)\}$^^^^4 OR ti:\4^^^^ and batch size 64. RL uses group size $A_y = \{(o,a)\}$4, top-variance subset = 5^^^^4search_query4^^^^% of staged samples, $A_y = \{(o,a)\}$5 for the KL penalty, $A_y = \{(o,a)\}$6 for clipping, RL batch size = ^^^^4 OR ti:\44^^^^, and training time of approximately ^^^^4 OR ti:\4^^^^5 hours on $A_y = \{(o,a)\}$7 GPUs. The judge model for unit decomposition and matching is Qwen^^^^4 OR ti:\4^^^^-^^^^4 OR ti:\4 OR ti:\4^^^^B. To speed matching, ground-truth units are preprocessed once and cached, while predicted units and matching use GPU-accelerated LLM inference.

The empirical gains reported for OwlCap isolate the contribution of CSER (&&&^^^^4all:\4^^^^&&&):

Setting	VDC	DREAM-^^^^4all:\4^^^^K
SFT only	+^^^^4 OR ti:\4^^^^.^^^^4all:\4^^^^ Acc	+^^^^4 OR ti:\4^^^^.^^^^4search_query4^^^^ F^^^^4all:\4^^^^
RL with only Completeness score	+^^^^4all:\4^^^^.4 Acc	+^^^^4 OR ti:\4^^^^.6 F^^^^4all:\4^^^^
RL with only Correctness score	+^^^^4all:\4^^^^.^^^^4search_query4 Acc	+^^^^4all:\4^^^^.6 F^^^^4all:\4^^^^
RL with both (CSER) on SFT-initialized model	+4.^^^^4 OR ti:\4^^^^ Acc	+4.6 F^^^^4all:\4^^^^

Under identical settings, the paper also reports that, compared with VideoCap-R^^^^4all:\4^^^^’s Event Score reward, CSER yields +^^^^4all:\4^^^^.7 Acc more on VDC and +^^^^4all:\4^^^^.4 F^^^^4all:\4^^^^ more on DREAM-^^^^4all:\4^^^^K (&&&^^^^4all:\4^^^^&&&). The abstract summarizes the benchmark improvements as significant gains on the detail-focused VDC and the motion-focused DREAM-^^^^4all:\4^^^^K.

For SC-Captioner, the abstract states that applying the framework on large visual-LLMs can generate better image captions across various scenarios and “significantly outperform” the direct preference optimization training strategy (&&&^^^^4search_query4^^^^&&&). The same paper also introduces a set of metrics refined from CAPTURE to alleviate incomplete precision evaluation and inefficient relation matching problems, and it reports the collection of RefinedCaps, a fine-grained annotated image caption dataset consisting of 6.5K diverse images from the COCO dataset (&&&^^^^4search_query4^^^^&&&). These elements are not themselves part of CSER, but they define the evaluation and data context in which the reward was developed.

A common misconception would be to treat CSER as a single immutable objective. The available literature instead shows two task-adapted instantiations under the same name: one centered on set-difference over scene-graph elements in self-correction (&&&^^^^4search_query4^^^^&&&), and one centered on unit-to-set matching with bidirectional validation in GRPO-based video captioning (&&&^^^^4all:\4^^^^&&&). What remains invariant is the core criterion of semantic equivalence under penalties for hallucinated additions and omitted reference content.