---
title: 'SRAG-MAV: Structured Extraction for Chinese Hate Speech'
url: https://www.emergentmind.com/topics/srag-mav-framework
type: topic
---

# SRAG-MAV: Structured Extraction for Chinese Hate Speech

SRAG-MAV is a structured large-language-model framework introduced for **CCL25-Eval Task 10** on the **STATE ToxiCN** benchmark for **Fine-Grained Chinese Hate Speech Recognition (FGCHSR)**. The framework combines **Task Reformulation (TR)**, **Self-Retrieval-Augmented Generation (SRAG)**, and **Multi-Round Accumulative Voting (MAV)** in a single inference and training pipeline built on **Qwen2.5-7B**. Its central design is to simplify the target structure, use the annotated training set itself as retrieval memory, and stabilize structured predictions through repeated inference with frequency-thresholded aggregation [2507.18580].

## 1. Task setting and formal output structure

The framework is designed for a benchmark in which a system must extract a structured **quadruplet** from Chinese social media text. The required fields are **Target**, **Argument**, **Targeted Group**, and **Hateful**. The paper emphasizes that this problem is harder than ordinary hate-speech classification because the model must recover structured, context-dependent information rather than output a single label [2507.18580].

The reported dataset scale is **4,000 training samples** and **1,602 test samples**, with training conducted on **4× NVIDIA L40S 40GB GPUs**. Evaluation is based on three scores. **Hard Score** is exact F1 on exact quadruplet matches. **Soft Score** is a partial-match F1 requiring the same **Targeted Group**, the same **Hateful** label, and **Target** and **Argument** similarity above **50%**. **Average Score** is the mean of Hard and Soft. These definitions matter because SRAG-MAV is optimized for exact structured extraction rather than only coarse classification, and the framework’s design directly targets error modes that degrade exact-match performance.

The paper’s problem framing implies a strong dependence between structural consistency and evaluation sensitivity. A plausible implication is that even small output inconsistencies can sharply reduce Hard Score, which helps explain why the framework allocates separate mechanisms to structure simplification, contextual grounding, and inference stabilization.

## 2. End-to-end pipeline and backbone configuration

The SRAG-MAV pipeline is specified as a nine-stage process. It first **reformulates the task from quadruplet extraction to triplet extraction**. It then **embeds all training texts** with a Chinese retrieval encoder, **retrieves similar training examples** for each input, **constructs prompts** by concatenating retrieved examples with the input, and **fine-tunes Qwen2.5-7B** on these retrieval-enriched training prompts. At inference time, it **retrieves top-\(k\) similar training samples** for each test input, runs **multi-round inference** over multiple prompts, **votes accumulatively** until one triplet candidate reaches threshold \(\tau\), and finally converts the selected triplet back to a **quadruplet** by inferring the hatefulness label [2507.18580].

The implementation stack is explicit. The backbone is **Qwen2.5-7B**. Fine-tuning uses **LLaMA-Factory**. Inference uses **vLLM**. The retrieval model is **bge-large-zh-v1.5**. Reported hyperparameters are: fine-tuning temperature **0.7**, MAV inference temperature **0.1**, retrieval top-\(k\) **10**, and voting threshold \(\tau\) **200**. The paper notes that larger thresholds generally improve results, but increase computational cost.

Architecturally, the framework is not a generic external-corpus RAG system. Its retrieval source is the benchmark’s own annotated training set, and its output is a structured extraction object rather than free-form answer text. This places SRAG-MAV at the intersection of prompt-based structured generation, example retrieval, and test-time ensembling.

## 3. Task Reformulation (TR)

The **Task Reformulation** component is motivated by a dataset regularity: when the **Targeted Group** is `"no-hate"`, the **Hateful** label is also `"no-hate"`; otherwise, the hatefulness is `"hate"` [2507.18580]. On that basis, the framework replaces direct quadruplet prediction with triplet prediction.

The original target is

\[
(Target, Argument, Targeted\ Group, Hateful)
\]

and the reformulated target is

\[
(Target, Argument, Targeted\ Group).
\]

The final quadruplet is reconstructed deterministically:

- if `Targeted Group = no-hate`, then `Hateful = no-hate`
- else `Hateful = hate`

The paper characterizes this as a structure simplification step. It reduces the number of structured elements that must be generated directly, lowers the chance of inconsistent outputs, and simplifies sequence generation. In the reported ablation, adding **TR** to the base model moves performance from **23.70 / 47.03 / 35.365** to **24.33 / 47.35 / 35.840** in **Hard / Soft / Average** terms, indicating a modest but measurable gain.

Conceptually, TR converts a partially redundant output schema into a more compact target space. This suggests that part of SRAG-MAV’s effectiveness comes not from stronger parametric knowledge alone, but from aligning the prediction interface with a deterministic property of the dataset.

## 4. Self-Retrieval-Augmented Generation (SRAG)

The **SRAG** component uses the **training set itself** as the retrieval corpus rather than relying on an external knowledge base. The paper argues that this is especially suitable when external high-quality domain corpora may be unavailable and when the training data already contains annotated structured examples that can function as few-shot demonstrations [2507.18580].

All training texts are encoded with **bge-large-zh-v1.5** and stored in a retrieval database. Similarity is computed by **cosine similarity**:

\[
\text{sim}(x, x_i) = \cos(\mathbf{e}_x, \mathbf{e}_{x_i})
\]

where \(\mathbf{e}_x\) and \(\mathbf{e}_{x_i}\) are embeddings from the retrieval model.

During training, the system retrieves the **most similar sample**, excludes the input itself, concatenates the retrieved sample with the original input, and trains the model to output the corresponding triplet. During inference, it retrieves the **top-\(k\)** most similar training samples for each test input; each retrieved sample is paired with the test input, yielding **\(k\) distinct prompts**.

The paper interprets SRAG as a form of dynamic in-context learning. Retrieved examples are semantically close to the test input, provide task format guidance, bias the model toward valid structured outputs, and reduce ambiguity in difficult cases. In the ablation, adding **SRAG** on top of **TR** improves performance from **24.33 / 47.35 / 35.840** to **25.30 / 47.85 / 36.575**, which is larger than the gain from TR alone.

A notable aspect of SRAG is that retrieval is not auxiliary metadata retrieval in the usual RAG sense. It is retrieval over labeled instances whose value lies in prompt conditioning and output regularization. That design makes the retrieval memory both domain-specific and annotation-aware.

## 5. Multi-Round Accumulative Voting (MAV) and empirical performance

The **MAV** component is introduced to address output variability across prompts and decoding rounds. The paper states that MAV draws inspiration from the idea behind **Parallel Scaling Law (PARSCALE)**: use multiple diverse input variants and aggregate outputs instead of relying on a single pass [2507.18580].

Its procedure is explicit. The system retrieves top-\(k\) similar training examples, builds \(k\) prompts, runs inference on each prompt, collects the predicted triplets, counts how often each triplet appears, continues rounds of inference until the most frequent triplet’s count exceeds threshold \(\tau\), selects that triplet as the final answer, and converts the triplet into a quadruplet by restoring the hatefulness label. The aggregation rule is described as:

\[
\hat{y}^* = \arg\max_y \text{count}(y)
\]

subject to

\[
\text{count}(\hat{y}^*) \ge \tau.
\]

The paper describes this as a **frequency-thresholded accumulation scheme** rather than a single majority vote over a fixed number of samples. Its intended effects are reduced dependence on any one prompt, smoothing of decoding randomness, favoring repeated consistent outputs, and trading extra inference cost for better stability.

Empirically, the full SRAG-MAV system achieves **Hard Score: 26.66**, **Soft Score: 48.35**, and **Average Score: 37.505** on the **STATE ToxiCN** test set. The reported baselines include **Qwen2.5-7B: 23.70 / 47.03 / 35.365**, **GPT-4o: Average Score 15.63**, **mT5-base: 27.605 average**, **Mistral-7B: 34.670 average**, **LLaMA3-8B: 35.175 average**, **ShieldLM-14B-Qwen: 34.585 average**, and **ShieldGemma-9B: 35.315 average**. Relative to the vanilla Qwen2.5-7B baseline, the full system improves by **+2.96 Hard**, **+1.32 Soft**, and **+2.14 Average**.

The ablation sequence is cumulative and consistent with the architectural rationale:

- **Base Model**: **23.70 / 47.03 / 35.365**
- **+ TR**: **24.33 / 47.35 / 35.840**
- **+ TR + SRAG**: **25.30 / 47.85 / 36.575**
- **+ TR + SRAG + MAV**: **26.66 / 48.35 / 37.505**

The sensitivity analysis for MAV tests thresholds

\[
[1, 2, 3, 5, 8, 10, 15, 20, 30, 40, 50, 80, 100, 200].
\]

With \(k=10\), the paper reports increases from **25.30** to **26.66** in Hard Score, from **47.85** to **48.35** in Soft Score, and from **36.575** to **37.505** in Average Score, with the most notable improvement occurring at higher thresholds.

## 6. Nomenclature, scope, and related arXiv usages

In current arXiv usage, **SRAG-MAV** most directly denotes the framework for **Fine-Grained Chinese Hate Speech Recognition** introduced in "System Report for CCL25-Eval Task 10: SRAG-MAV for Fine-Grained Chinese Hate Speech Recognition" [2507.18580]. The name should be distinguished from several unrelated systems that may invite confusion because they share either the **MAV** acronym or adjacent terminology.

One source of confusion is **SCARF**, the framework introduced in "A System for Comprehensive Assessment of RAG Frameworks" [2504.07803]. That paper explicitly states that **SCARF** is the framework name and that it is **not** explicitly called the “SRAG-MAV Framework”; the wording is described there as a naming variant or a misremembered label. SCARF is a modular, black-box evaluation framework for deployed **Retrieval-Augmented Generation (RAG)** systems rather than a hate-speech extraction system.

A second source of ambiguity comes from MAV robotics literature. "Global-Local MAV Detection under Challenging Conditions based on Appearance and Motion" introduces **GLAD**, a global-local detector for **micro aerial vehicles**, with components such as a global detector, a local detector, an adaptive local search region, and a detector switcher [2312.11008]. Likewise, "Efficient Submap-based Autonomous MAV Exploration using Visual-Inertial SLAM Configurable for LiDARs or Depth Cameras" presents an autonomous exploration framework for MAVs based on visual-inertial SLAM, submaps, frontier computation, and next-best-view planning [2409.16972]. These systems are unrelated in task, model class, and evaluation setting.

Within its own scope, SRAG-MAV is best understood as a retrieval-augmented structured extraction architecture for a narrow benchmarked task. The paper’s stated practical takeaway is that it is especially useful when high-quality external knowledge is unavailable. Its computational tradeoff is also explicit: larger voting thresholds improve results but increase cost. A plausible implication is that the framework is most attractive when exact structured output quality is more critical than single-pass inference latency.

Source: https://www.emergentmind.com/topics/srag-mav-framework