Complaint Retention (CR) in Complaint Generation

• Complaint Retention (CR) is a hybrid metric that assesses if generated complaint text retains core attributes like negativity, specific aspects, and emotional cues.
• It combines normalized Vader sentiment, Text2Emotion scores, and GPT-4 aspect checking to quantify fidelity in complaint generation.
• Empirical studies demonstrate that CR distinguishes complaint-specific outputs (CoD-V) from generic video descriptions, highlighting the metric's relevance.

Searching arXiv for the cited paper and closely related complaint-retention literature. arxiv_search(query="(Das et al., 24 Sep 2025) OR \"When Words Can't Capture It All\" complaint retention metric", max_results=5, sort_by="submittedDate") Complaint Retention (CR) denotes, in its most explicit recent formulation, a complaint-specific hybrid evaluation metric for generated complaint text. It was introduced in the CoD-V setting—Complaint Description from Videos—to measure whether a model preserves the nature of a complaint rather than merely producing a fluent, semantically plausible, or lexically similar description. In that formulation, CR is tied to the ComVID dataset, which contains 1,175 complaint videos with corresponding descriptions and emotional-state annotations, and it is explicitly proposed to discriminate CoD-V from standard video summary generation and video description tasks (Das et al., 24 Sep 2025). In adjacent complaint analytics and CRM literature, the same phrase is also used more loosely for operational persistence of complaints—such as delayed handling, unresolved backlog, recurrence, or complaint-driven retention risk—so the term spans both a formal NLG metric and a broader systems concept (Vaishnav et al., 2024).

1. Terminological scope and conceptual boundaries

In the strict metric sense, CR is a complaint-aware evaluation criterion for complaint generation. The underlying claim is that standard NLG metrics—BLEU, ROUGE, METEOR, BERTScore, perplexity, and readability—mainly capture lexical overlap, semantic similarity, fluency, or readability, but do not explicitly test whether the output still behaves like a complaint. The CoD-V paper therefore introduces CR, or “Complaint Nature Retention,” to preserve three complaint-specific properties: negative complaint tone or sentiment, emotional intensity or state, and the specific complaint aspects such as defect, functionality, or quality (Das et al., 24 Sep 2025).

Outside that formal usage, recent work treats complaint retention more operationally. In CFPB complaint analytics, no explicit CR metric is defined; instead, “timely response” and company_response categories such as “In progress,” “Untimely response,” and “Closed without relief” function as proxies for complaint persistence, delayed handling, or weak closure quality (Vaishnav et al., 2024). In municipal complaint systems, backlog, defer/batch actions, recurrence, average close time, unresolved burden, and complaints left in lower-priority queues play a similar role, again without a single scalar CR definition (Aldridge et al., 7 May 2026). In CRM-oriented reviews, complaints are explicitly part of the customer database and customer life-cycle record, which places complaint handling inside the broader customer-retention problem rather than outside it (Farooqi et al., 2012).

This suggests that contemporary usage partitions CR into two non-equivalent but related notions. One is content retention: whether generated text retains complaint-specific semantics. The other is case retention: whether a complaint persists in operational systems through delay, recurrence, or unresolved handling. The literature is precise about the first and largely proxy-based about the second.

2. Formalization of CR as a complaint-generation metric

The formal CR metric is defined through three sub-scores: normalized Vader sentiment, an emotion score derived from Text2Emotion, and an aspect-retention score derived from GPT-4 aspect checking. The paper gives sentence-level and sample-level equations for the sentiment term, plus explicit definitions for the emotion and aspect terms (Das et al., 24 Sep 2025).

For sentence $i$, normalized Vader sentiment is

$$S_{\text{N Vader}, i} = \frac{S_{\text{Vader}, i} + 1}{2}$$

which maps the usual Vader range $[-1,1]$ into $[0,1]$. For sample $k$ with $N_k$ sentences, the paper defines

$$S_{\text{N Vader}, k} = \frac{1}{N_k} \sum_{i=1}^{N_k} S_{\text{N Vader}, i}.$$

For the emotion term, Text2Emotion extracts Happy $(H)$, Angry $(A)$, Surprise $(S)$, and Fear $$S_{\text{N Vader}, i} = \frac{S_{\text{Vader}, i} + 1}{2}$$0, each in $$S_{\text{N Vader}, i} = \frac{S_{\text{Vader}, i} + 1}{2}$$1, and the paper defines

$$S_{\text{N Vader}, i} = \frac{S_{\text{Vader}, i} + 1}{2}$$2

For aspect retention, GPT-4 is used as a yes/no aspect-presence classifier. The aspect score is

$$S_{\text{N Vader}, i} = \frac{S_{\text{Vader}, i} + 1}{2}$$3

where $$S_{\text{N Vader}, i} = \frac{S_{\text{Vader}, i} + 1}{2}$$4 is the total number of ground-truth aspects for sample $$S_{\text{N Vader}, i} = \frac{S_{\text{Vader}, i} + 1}{2}$$5, $$S_{\text{N Vader}, i} = \frac{S_{\text{Vader}, i} + 1}{2}$$6 is the indicator function, and $$S_{\text{N Vader}, i} = \frac{S_{\text{Vader}, i} + 1}{2}$$7 is the predicted text.

Component	Definition	Operational role
$$S_{\text{N Vader}, i} = \frac{S_{\text{Vader}, i} + 1}{2}$$8	Normalized Vader sentiment	Complaint tone retention
$$S_{\text{N Vader}, i} = \frac{S_{\text{Vader}, i} + 1}{2}$$9	Mean of Happy, Angry, Surprise, Fear	Emotional intensity/content
$[-1,1]$0	Gold-aspect presence in predicted text	Complaint aspect retention

The final CR expression is typographically malformed in the paper. The accompanying explanation states that CR “combines the three metrics (normalized sentiment, emotion, and aspect presence) into a single metric,” which suggests an equal-weight average of the three sub-scores:

$[-1,1]$1

The paper does not fully formalize whether this combination is computed per sample and then averaged, or built from dataset-level aggregates. That aggregation ambiguity is part of the metric’s published definition rather than an external criticism (Das et al., 24 Sep 2025).

3. Computational structure and annotation dependencies

CR is computed primarily from generated text, not directly from the source video. The required inputs are the generated complaint text, ground-truth aspect annotations, and sentence segmentation of the generated text. Generated complaint text is needed for Vader sentiment, Text2Emotion extraction, and GPT-4 aspect checking. Ground-truth aspect annotations are required for $[-1,1]$2. Sentence segmentation is needed because Vader normalization is performed sentence-wise before averaging (Das et al., 24 Sep 2025).

Several inputs commonly used in other evaluation settings are not strictly required by CR itself. The reference complaint text is not directly used in the stated formula. Video-derived evidence is not used directly. Retrieved exemplars or multimodal retrieval context are part of the model family, not the metric. The dataset’s emotion labels are also not directly compared with generated emotion; instead, emotion content is inferred from the generated text using Text2Emotion (Das et al., 24 Sep 2025).

Methodologically, CR is a hybrid metric. It is partly reference-based because aspect retention depends on gold aspect labels; partly rule-based or lexicon-based because Vader is used for sentiment; partly tool-based because Text2Emotion provides emotion scores; and partly LLM-judged because GPT-4 is used to decide whether each aspect is present in the generated text. It is not retrieval-based, and it is not learned end-to-end as a metric (Das et al., 24 Sep 2025).

The aspect-presence step is implemented through a GPT-4 prompt of the form: “Is the aspect {predicted_aspect}' present in the given{text}'? Answer with only Yes' orNo'.” The paper then maps Yes/No to 1/0. The surrounding description states that the checked aspects are manually annotated ground-truth aspects, so the prompt string and the prose are not perfectly aligned. The most defensible reading is that each gold aspect is checked for presence in the predicted text. No threshold is given for what constitutes “high” CR, and no sentiment, emotion, or classifier confidence cutoff is specified; CR is treated as a continuous score (Das et al., 24 Sep 2025).

4. Experimental role in CoD-V and empirical behavior

CR is used primarily to answer the question of how CoD-V differs from summary generation (SG) and video description (VD). In the main task-difference comparison, CR is consistently higher for CoD-V outputs than for SG or VD across all compared model families. This is the principal validation argument for the metric: it separates complaint generation from generic description-style outputs more clearly than standard lexical metrics do (Das et al., 24 Sep 2025).

Model	Task	CR
VideoLLaVA2-7b	SG	0.41
VideoLLaVA2-7b	VD	0.48
VideoLLaVA2-7b	CoD-V	0.57
GPT4O	SG	0.45
GPT4O	VD	0.42
GPT4O	CoD-V	0.56
GeminiFlash	SG	0.47
GeminiFlash	VD	0.48
GeminiFlash	CoD-V	0.55
Proposed	SG	0.59
Proposed	VD	0.58
Proposed	CoD-V	0.62

The paper interprets these results as evidence that complaint generation is a distinct task because complaint-specific nature is preserved better under CoD-V prompting. Traditional summary and description settings often yield more neutral descriptions or action summaries. A typical failure mode is a visually correct but complaint-agnostic output such as “the person is trying to take a screenshot,” which may describe the video while missing the complaint intent entirely (Das et al., 24 Sep 2025).

The qualitative examples are consistent with this interpretation. For the mouse example, the ground truth states that the user received a defective and damaged mouse and that “The scrolling button is not working properly.” The proposed VideoLLaMA2-7b+MR with emotion preserves defective/damaged and the scrolling-button failure, which would plausibly raise aspect retention and complaint tone retention. By contrast, weaker outputs reduce the complaint to generic quality language or distort the issue by introducing “old product,” which weakens fidelity to the complaint aspects (Das et al., 24 Sep 2025).

The error-analysis example shows the same structure. A keyboard output preserves that the keyboard is not functioning properly and that wrong typing occurs, but hallucinates “refund” and “unable to contact customer care” and misses the “USB jack” issue. This would plausibly reduce aspect retention even if the text remains negative and fluent. The paper therefore uses CR not merely as a sentiment score, but as a compound fidelity measure for complaint-bearing content (Das et al., 24 Sep 2025).

5. Operational extensions in complaint analytics and CRM

Although the formal CR metric belongs to complaint generation, adjacent literatures provide operational machinery that can feed complaint-retention systems in the broader sense. In finance, the MulComp dataset frames complaint understanding at the aspect level rather than as a single binary label. It contains 433 publicly accessible multimodal video instances, annotated at utterance level across five aspect categories—Customer Service, Service Types, Transaction, Claimed Benefit, and Miscellaneous—with a 3-state label scheme in which 2 means aspect present with complaint, 1 means aspect present with non-complaint, and 0 means aspect absent. Solution 3.0 performs aspect classification and complaint identification jointly, but it does not model retention outcomes such as churn, closure, or service recovery effectiveness (Das et al., 26 Feb 2025).

In customer-service speech, complaint detection functions as the upstream sensing layer of broader retention workflows. The cascaded cross-modal transformer (CCMT) combines French transcripts, English translation, and Wav2Vec2.0 features, reaching complaint UAR of 65.41% and request UAR of 85.87% on the private test set of the ACM Multimedia 2023 ComParE Requests Sub-Challenge (Ristea et al., 2023). On the HealthCall corpus, transcript-based linguistic features reach 70.3 UAR on complaint detection, outperforming acoustic and turn-taking baselines, while a reduced turn-taking feature set still reaches 57.0 UAR with only six features (Lackovic et al., 2022). These systems do not solve complaint retention directly, but they provide real-time or post-call complaint flags that can drive escalation, agent assist, QA review, or downstream churn-risk models.

In structured complaint databases, operational CR is usually proxied rather than measured. In CFPB analytics, the most retention-relevant outputs are “timely response” and company_response with eight categories: Closed with explanation, Closed with non-monetary relief, In progress, Closed with monetary relief, Closed without relief, Closed, Untimely response, and Closed with relief. Here “Timely response = No,” “In progress,” and “Untimely response” serve as the clearest persistence proxies, while recurring topics from LDA identify complaint cohorts such as credit reporting, banking and loans, mortgages, and debt collection/compliance (Vaishnav et al., 2024).

Municipal complaint systems make the operational notion even more explicit. In the RL framework for NYC complaint intake, actions such as escalate, batch, defer, and inspect now determine whether complaints are immediately serviced or remain in queue. State variables include backlog, open count, average close time, complaint age, time since last resolution, recurrence flags, and unresolved burden. The reward contains a retention term $[-1,1]$3, although its exact formula is not visible in the provided text. Here complaint retention is effectively the sequential survival of complaints in constrained triage systems under finite classification capacity (Aldridge et al., 7 May 2026).

A related CRM perspective appears in downward-trend prediction and “silent sufferer” detection. That framework models customers whose purchase behavior deteriorates relative to their own 12-month baseline and then uses semi-supervised learning to identify likely issue-driven decline even when no explicit complaint is filed. Its central contribution to CR is the idea that many dissatisfied customers never speak out, so explicit complaints are only partially observed labels (Hu et al., 2018).

6. Limitations, ambiguities, and research directions

The formal CR metric has several underspecified components. The final aggregation formula is malformed in print; per-sample versus corpus-level aggregation is not fully formalized; emotion is modeled as the mean of Happy, Angry, Surprise, and Fear without direct comparison to the dataset’s gold emotion label; and the aspect-prompt wording is inconsistent with the surrounding prose. The metric also depends on GPT-4 for aspect presence, which raises reproducibility and stability questions, and the paper does not report component ablations, significance tests, or direct correlation between CR and human judgments (Das et al., 24 Sep 2025).

These limitations matter because CR is intended to function as a complaint-aware alternative to lexical metrics, yet its own internals are heterogeneous and partly externalized. The metric mixes lexicon-based sentiment, library-based emotion extraction, and LLM-mediated aspect verification. This suggests that future complaint-generation evaluation may need cleaner aggregation protocols, explicit human-correlation studies, and tighter definitions of what counts as complaint fidelity rather than affect alone.

The broader complaint-retention literature is even less standardized. Many systems rely on proxies such as timely response, response category, recurrence, backlog, average close time, or repeat usage, rather than on a direct retention variable (Vaishnav et al., 2024). Outside complaint mining proper, repeat-usage models for web information goods separate one-time-only users from repeat-users through the LSD/OTB framework, with $[-1,1]$4 as the one-time-user proportion and $[-1,1]$5 as the proportion of usage attributable to repeat-users; this suggests one possible count-based formalism for complaint-channel revisits, but that transfer remains inferential rather than demonstrated for complaint systems (0803.1104). Likewise, the “silent sufferer” framework suggests that complaint retention should not be limited to observed complaint traffic, because latent dissatisfaction may be behaviorally visible before it is textually or procedurally registered (Hu et al., 2018).

A plausible implication is that the field will need a unified view in which content retention, case persistence, and service-recovery outcome are jointly modeled. On that view, a complaint-generation system would be evaluated not only on whether it preserves complaint nature in text, but also on whether its outputs support better routing, lower unresolved burden, reduced recurrence, and improved post-complaint customer outcomes. The present literature provides the components for such an overview, but not yet the synthesis itself.