---
title: 'CSConDa: Vietnamese Customer Support QA Benchmark'
url: https://www.emergentmind.com/topics/customer-support-conversations-dataset-csconda
type: topic
---

# CSConDa: Vietnamese Customer Support QA Benchmark

Searching arXiv for the cited CSConDa-related papers to ground the article in current sources.
CSConDa is a domain-specific benchmark designed to evaluate Vietnamese Large Language Models in customer support question answering, with an emphasis on intrinsic model capabilities such as accuracy, fluency, consistency, and hallucination tendency without external knowledge augmentation [2507.22542]. It is a curated benchmark of over 9,000 QA pairs drawn from real interactions with human advisors at DooPage, a large Vietnamese software company serving approximately 30,000 customers and 45,000 advisors, and it covers pricing, product and service availability, onboarding and configuration, technical troubleshooting, account management, multi-location support, and platform limitations [2507.22542]. Although “Conversations” appears in the name, the released resource is structured as independent single-turn QA pairs extracted from multi-turn customer-advisor chat logs rather than as multi-turn dialogues [2507.22542].

## 1. Definition and task scope

CSConDa was introduced to address the absence of benchmark datasets that reflect real customer interactions in Vietnamese customer support, particularly for enterprises deploying lightweight open-source LLMs in the 7–9B parameter range [2507.22542]. Its stated purpose is evaluation rather than training-time augmentation: the benchmark targets model behavior in standalone question answering under realistic customer-service language conditions, rather than retrieval-augmented or externally grounded setups [2507.22542].

A defining property of the dataset is its linguistic realism. The benchmark includes teencode, abbreviations, code-switching, acronyms, and typos, including English terms such as “API,” shorthand such as “ko” for “không” and “lsao” for “làm sao,” and acronyms such as OA and YTB [2507.22542]. This distinguishes it from Wikipedia-style Vietnamese QA datasets and from more structured or extractive resources. The corpus is also explicitly organized by interaction complexity through three labels: General, Simple, and Complex. General denotes short, routine exchanges with minimal reasoning; Simple covers concrete requests such as pricing inquiries requiring brief structured replies; Complex denotes multi-faceted questions involving domain terminology, constraints, and follow-ups, even though the released items remain single-turn pairs [2507.22542].

The release size is reported with a minor discrepancy in the source paper. The paper reports “over 9,000 QA pairs,” and the dataset analysis gives All = 9,849 items, with Train = 8,349 and Test = 1,500, while a comparison table elsewhere lists 9,862 items; the paper notes that this likely reflects a version count change and treats approximately 9.8k items as the working size [2507.22542].

## 2. Data provenance, curation, and privacy

The source material consists of raw logs from DooPage’s multi-channel chat platform. Construction followed a five-phase process: worker recruitment, conversation collection, dataset creation, validation and categorization, and dataset splitting [2507.22542]. Ten annotators were trained under a domain expert identified as a CEO, who established selection, anonymization, and categorization guidelines; the annotators signed confidentiality agreements [2507.22542].

Conversation collection applied filtering for coherence, topic diversity, and exclusion of sensitive or inappropriate content. Dataset creation then used an automated pipeline to extract QA pairs while preserving contextual consistency. Personally identifiable information was removed, sensitive spans were replaced with placeholders such as “<số điện thoại>,” system-generated artifacts such as syntax markers and reaction icons were removed, and formatting was standardized [2507.22542]. Validation included post-checks for security compliance, formatting quality, and annotation accuracy, followed by categorization into General, Simple, and Complex according to complexity, reasoning demands, and domain knowledge [2507.22542].

The privacy model is notably conservative. No raw multi-turn transcripts are released; only curated single-turn QA pairs are distributed, explicitly to minimize re-identification risk [2507.22542]. The paper does not state formal IRB approval or provide a detailed license in the text, and it advises checking the dataset card for deployment-relevant terms [2507.22542]. This suggests a release philosophy centered on utility for benchmarking while constraining exposure of sensitive operational chat data.

## 3. Corpus composition and linguistic characteristics

The dataset spans a range of customer-support topics, including pricing inquiries, product and service availability, onboarding and configuration, technical troubleshooting such as channels, Zalo OA integration, and downloads, account management, multi-location support, and platform limitations such as YouTube support [2507.22542]. By type, the All split contains 3,523 General items, 5,211 Simple items, and 1,114 Complex items [2507.22542]. The train split contains 3,023 General, 4,711 Simple, and 614 Complex items, while the standardized test split contains 1,500 questions, evenly divided into 500 per type [2507.22542].

Length statistics reinforce the complexity stratification. On the All split, average question length is 16.82 and average answer length is 39.60 [2507.22542]. By type, question length is 9.18 for General, 18.23 for Simple, and 34.39 for Complex, while answer length is 39.16 for General, 38.27 for Simple, and 47.26 for Complex [2507.22542]. The paper notes that length is measured by token or word count at the sentence level, although the exact unit is not explicitly stated [2507.22542].

The linguistic profile is strongly informal. The All split has a vocabulary size of 4,683, an abbreviation count of 12,343 with frequency approximately 0.10, an acronym count of 6,901 with frequency approximately 0.07, and a typo count of 1,807 with frequency approximately 0.02 [2507.22542]. These statistics matter because they characterize a regime in which lexical normalization is nontrivial and where conventional benchmark assumptions about orthographic cleanliness do not hold.

The schema exposed in the paper is partially specified rather than fixed. The release is described as containing the question and answer texts, the type labels, and anonymization of sensitive spans, but the exact Hugging Face schema field names are not given; the paper explicitly notes that fields such as `question`, `answer`, `type`, or `category` should be verified on the dataset card [2507.22542]. Timestamp fields and anonymization flags are likewise not specified as distributed metadata [2507.22542].

## 4. Evaluation framework and benchmark design

CSConDa is paired with a comprehensive evaluation framework for Vietnamese customer-support QA models [2507.22542]. Automatic metrics include BLEU-2, ROUGE-L, METEOR, Cosine Similarity, BERTScore, and Hallucination Score, with Hallucination Score defined via Kolena’s GPT-4-based prompt metric, where lower is better [2507.22542]. The framework also introduces a penalty-adjusted aggregation to penalize failures such as non-answers or nonsensical loops:
$$
Score_M = \left( \frac{\sum_{i=1}^{N} x_i}{N} \right)\times \rho,\qquad \rho = \left(\frac{A}{N}\right)^{M_c}
$$
where $x_i$ is the per-item score, $A$ is the number of successfully generated answers, $N$ is the total number of test items, and $M_c = 1$ if higher is better for the metric and $M_c = -1$ otherwise [2507.22542].

Syntactic analysis complements these automatic scores. The framework uses Stanza for POS tagging and underthesea for phrase tagging, NER, and dependency parsing [2507.22542]. Reported syntactic measures include Word Count, POS Ratio, Phrase Ratio, Named Entity Difference, and Dependency Length. POS Ratio is defined as the ratio of content words to function words, while Phrase Ratio measures the ratio of content phrases to function phrases; Dependency Length is the average distance between heads and dependents across all dependencies [2507.22542]. Model rankings across item types are aggregated through a rank-based scheme:
$$
Score_{X,T} = \frac{1}{\sum_i r_{i,T}}, \qquad Score_{X,\mathrm{overall}} = \frac{1}{\sum_T R_T(X)}
$$
with higher scores indicating better overall standing [2507.22542].

The benchmark evaluates 11 lightweight open-source LLMs, including Vietnamese-finetuned models such as URA-LLaMa-2.1 8B, GemSUra 7B, Vistral 7B, and VinaLLaMA 7B, as well as multilingual models with Vietnamese support such as SeaLLMs 7B, Sailor 7B, Qwen2 7B, Ghost 8B, SEA-LION 7B, BLOOMZ 7B, and Aya-Expanse 8B [2507.22542]. Experiments used zero-shot prompting, a single NVIDIA A100 40GB GPU, tuned sampling parameters intended to balance diversity and stability, and response-length constraints to approximate human answers, although exact decoding hyperparameters were not reported [2507.22542].

## 5. Empirical findings

The overall ranking reported in the paper places Vistral 7B, SeaLLMs 7B, Sailor 7B, SEA-LION 7B, and GemSUra 7B as the top five models, while also emphasizing that absolute scores remain low, indicating domain difficulty [2507.22542]. This is consistent with the dataset’s combination of informal language, enterprise-specific terminology, and open-ended answer generation.

Performance varies by complexity type. On the General subset, SEA-LION 7B attains ROUGE-L of approximately 0.188 and BERTScore of approximately 0.678, while Sailor 7B reaches BLEU-2 of approximately 0.012 [2507.22542]. On the Simple subset, Vistral 7B achieves ROUGE-L of approximately 0.227, SEA-LION 7B achieves BERTScore of approximately 0.674, and SeaLLMs 7B attains a hallucination score of approximately 0.568, which is among the best in that subset because lower is better [2507.22542]. On the Complex subset, Vistral 7B and SeaLLMs 7B both reach ROUGE-L of approximately 0.247, SEA-LION 7B reaches BERTScore of approximately 0.670, and SeaLLMs 7B attains a hallucination score of approximately 0.642 [2507.22542]. Aya-Expanse 8B shows comparatively high BLEU-2 on Simple and Complex items, approximately 0.017 and 0.027 respectively, but also the worst hallucination behavior in those subsets, with scores of approximately 0.984 and 0.982 [2507.22542].

A central empirical pattern is the divergence between lexical and semantic similarity. BERTScore is relatively high, roughly in the 0.60–0.68 range, whereas BLEU-2 and ROUGE-L remain modest, indicating that models often capture semantic similarity but diverge lexically from human references, thereby producing responses that are less human-like in phrasing and format [2507.22542]. Syntactic analysis further shows that Vietnamese LLMs are verbose: model answers have higher word counts than human references across all types, and verbosity correlates with higher hallucination scores on Simple and Complex items [2507.22542]. Dependency Length is also consistently higher for model outputs, suggesting more rigid sentence construction relative to human answers, which the paper characterizes as concise and structurally flexible [2507.22542].

These findings support a narrow but important interpretation of the benchmark. CSConDa is not merely measuring factual matching; it is also exposing the interaction between register control, brevity, structural naturalness, and hallucination under real customer-support language conditions. A plausible implication is that improvements on this benchmark require more than generic instruction following, particularly for models expected to operate under Vietnamese informal-chat conventions.

## 6. Positioning, limitations, and nomenclature

Within Vietnamese QA research, CSConDa is positioned as the first large-scale Vietnamese customer support QA dataset sourced from human interactions, in contrast to datasets such as UIT-ViQuAD, UIT-ViWikiQA, VIMQA, VnYQA, and ViMedAQA, which are described as primarily extractive and structured around sources such as Wikipedia or curated articles [2507.22542]. Its distinctive contribution is the combination of customer-support content, open-ended answers, and informal linguistic phenomena specific to real user interactions in Vietnam [2507.22542].

The benchmark’s limitations are also explicit. First, it is single-turn QA only, so multi-turn phenomena such as follow-up, repair, and incremental disambiguation cannot be directly evaluated [2507.22542]. Second, topic balance is skewed toward Simple items, while Complex questions are substantially fewer at approximately 1.1k items [2507.22542]. Third, the data originate from one enterprise domain, so product-specific jargon may limit generalizability without adaptation [2507.22542]. The paper therefore recommends structurally aware fine-tuning to optimize for brevity and syntactic flexibility, domain adapters and normalization for teencode and abbreviations, retrieval-augmented generation for complex technical queries, and future extensions incorporating multi-turn context and knowledge snippets while preserving privacy and realism [2507.22542].

The name “Customer Support Conversations Dataset” also requires disambiguation in the broader literature. In “Evaluating, Synthesizing, and Enhancing for Customer Support Conversation,” the term “Customer Support Conversations Dataset (CSConDa)” does not appear; that paper instead introduces CSConv, a strategy-annotated evaluation set of 1,855 rewritten real-world conversations, and RoleCS, a synthetic training set of 11,232 strategy-rich conversations aligned to a COPC-based customer-support framework [2508.04423]. Likewise, “Real-World En Call Center Transcripts Dataset with PII Redaction” presents CallCenterEN, a large English transcript corpus of 91,706 conversations with word-level timestamps and PII redaction, but it is a different resource with different modality assumptions and release conditions [2507.02958]. Earlier work on conversational document prediction uses a public Twitter customer-care dataset rather than a resource officially named CSConDa [2010.02305], and the bilingual MAIA corpus with dialogue-quality and emotion annotations is likewise a distinct benchmark whose official name is MAIA rather than CSConDa [2311.13910].

In that sense, CSConDa names a specific Vietnamese benchmark rather than a generic category of customer-support corpora. Its importance lies less in scale alone than in its role as a controlled evaluation substrate for Vietnamese customer-support QA under real-world informal language conditions, where lexical mismatch, verbosity, hallucination, and enterprise-domain conventions all materially affect model behavior [2507.22542].

Source: https://www.emergentmind.com/topics/customer-support-conversations-dataset-csconda