Text Detoxification: Rewriting Toxic Text

• Text detoxification is a text style transfer task that converts toxic language into neutral or polite paraphrases while maintaining original content and fluency.
• It employs techniques like supervised rewriting, edit-based replacement, prompt-based paraphrasing, and diffusion methods to balance toxicity reduction with semantic preservation.
• Evaluation integrates style accuracy, semantic similarity, and fluency metrics (e.g., joint J score) while addressing challenges such as cultural specificity and limited parallel data.

Searching arXiv for recent and foundational papers on text detoxification. Searching for "text detoxification style transfer arXiv". Text detoxification is a text style transfer task in which toxic text is rewritten into a non-toxic or neutral paraphrase while preserving content and fluency; in this literature, detoxification does not ordinarily mean deleting content or refusing to answer, but specifically means rewriting the toxic text into a non-toxic paraphrase (Xie et al., 2024, Dementieva et al., 2024). The task is also framed as transforming text from toxic or offensive style to polite, neutral, or respectful style, with the central tension that stronger detoxification often competes with semantic preservation and contextual fidelity (Moskovskiy et al., 2022, Yu et al., 23 Jun 2025). Research on the topic spans parallel and non-parallel style transfer, edit-based rewriting, prompt-based paraphrasing, diffusion, preference and reinforcement learning, representation editing, and multilingual corpus construction (Floto et al., 2023, Dementieva et al., 2024).

1. Task definition and evaluation

Text detoxification is commonly defined through three simultaneous objectives: reduce toxicity, preserve content, and maintain fluency. In the style-transfer formulation, the source style is toxic and the target style is neutral or polite, and the desired output is a meaning-preserving paraphrase rather than a refusal or a generic safe reply (Moskovskiy et al., 2022). Some work emphasizes that the speaker’s stance and communicative intent should remain intact “as much as possible,” which sharpens the distinction between detoxification and blunt moderation by deletion (Yu et al., 23 Jun 2025).

This objective is usually operationalized through style transfer accuracy, semantic similarity, and fluency. A common aggregate score is $J=\text{STA}\times\text{SIM}\times\text{FL}$, used to force simultaneous success on all three dimensions (Xie et al., 2024). Other benchmarks define a dataset-level joint score as

$$J=\frac{1}{n}\sum_{i=1}^{n} STA(x_i)\cdot SIM(x_i)\cdot FL(x_i),$$

with indicator-valued component judgments (Dementieva et al., 2023). The metric suite is typically reference-free or weakly reference-based: style accuracy is computed by a toxicity classifier, semantic preservation by sentence-level similarity, and fluency by an acceptability or corruption detector (Pesaranghader et al., 2024).

The precise instantiation of these metrics varies across languages and benchmarks. One English setup uses a RoBERTa-based toxicity classifier for style accuracy, ParaNMT-trained sentence embeddings for semantic similarity, and a RoBERTa acceptability classifier trained on CoLA for fluency (Xie et al., 2024). Cross-lingual work later replaced older Russian metrics with a stronger toxicity classifier, a RuBERT conversational paraphrase classifier for semantic similarity, and a RuCoLA-based acceptability classifier, improving correlations with manual judgment for Russian system rankings from $0.231$ to $0.802$ on the joint score (Dementieva et al., 2023). This indicates that evaluation in detoxification is not merely a bookkeeping issue; it materially affects conclusions about which systems are best.

Scope also varies across corpora. Some datasets focus on vulgar or profane language, where detoxification is often a localized edit problem, while others explicitly target subtle toxicity, microaggressions, or hate-related toxicity (Dementieva et al., 2024, Hallinan et al., 2022). A plausible implication is that “text detoxification” names a family of related rewriting tasks rather than a single uniformly defined benchmark.

2. Data regimes and corpus construction

A defining property of the field is the scarcity of parallel toxic-to-neutral rewrites. Many datasets contain toxic and non-toxic texts but not aligned sentence-level pairs, which makes standard supervised seq2seq training difficult (Xie et al., 2024). By contrast, parallel corpora such as English ParaDetox and the Russian detoxification corpus enabled strong supervised baselines and became reference points for later work (Dementieva et al., 2023).

Parallel data have since been extended to new languages through dedicated collection pipelines. MultiParaDetox generalized the ParaDetox crowdsourcing procedure to Russian, Ukrainian, and Spanish via toxic-corpus preparation, language adaptation of the annotation tasks, and verification of meaning preservation and non-toxicity; manual inspection of 100 pairs per language found inappropriate pairs in less than 10% of cases, with Krippendorff’s $\alpha$ of $0.85$ for Russian, $0.90$ for Ukrainian, and $0.67$ for Spanish (Dementieva et al., 2024). Later multilingual work expanded parallel detoxification corpora to German, Chinese, Arabic, Hindi, and Amharic, creating a 9-language setup and selecting 1,000 samples per language, split into 400 training and 600 test examples (Dementieva et al., 2024).

Low-resource settings have produced additional corpus-building strategies. BanglaNirTox introduced 68,041 Bengali samples containing toxic text, a rationale, a class label among Explicit, Implicit, and Not Toxic, a detoxified version, and a paraphrase indication (Mohsin et al., 3 Nov 2025). Tatoxa added a manually annotated Tatar dataset of 701 toxic–non-toxic pairs with toxicity-level and edit-type labels (Alimova et al., 24 Jun 2026). For isiXhosa and Yorùbá, a curated parallel corpus of 178 toxic-to-neutral pairs per language was released alongside a detection-and-rewriting system (Agbeyangi, 9 Jan 2026).

These data regimes also motivate non-parallel learning. One line of work argues that when only toxic inputs and a binary toxicity screener are available, the true obstacle is not simply lack of references but lack of complete preferences between candidate outputs (Xie et al., 2024). Another line treats a small filtered parallel subset as a cold start, then shifts to reinforcement learning on unlabeled toxic inputs to reduce dependence on annotated data (Yu et al., 23 Jun 2025). Taken together, these results suggest that corpus design is not ancillary to detoxification; it largely determines which learning paradigm is even well-posed.

3. Rewriting, editing, and prompting paradigms

The most direct detoxification systems are rewriting models trained on toxic-to-neutral pairs. Supervised seq2seq models such as BART, ruT5, and mBART became strong baselines once parallel corpora were available, and multilingual fine-tuning on new parallel corpora was shown to outperform unsupervised baselines and prompted LLM baselines in several settings (Dementieva et al., 2024). In low-resource English–Hindi experiments, even small curated parallel subsets were sufficient for knowledge-transfer and multi-task variants to outperform deletion-style baselines on the balance between detoxification, content preservation, and fluency (Mukherjee et al., 2024).

A major subfamily exploits the fact that detoxification often requires only local edits. LEWIP for Russian casts the task as a two-step tagging-and-filling pipeline with token-level edit actions insert, keep, replace, and delete; it achieved the best style transfer accuracy in the RUSSE Detox shared task, and on 238 of 875 test examples the second-stage generator was not run at all because no replace or insert tags were predicted (Gusev, 2022). In low-resource settings, similar controlled editing appears in sentence-level lookup plus token-level substitution systems, which preserve non-toxic inputs unchanged and restrict intervention to detected toxic content (Agbeyangi, 9 Jan 2026).

Detoxification of subtle toxicity requires more than lexical deletion. MaRCo addresses microaggressions and veiled toxicity by first masking words whose local context is modeled differently by a non-toxic expert and a toxic anti-expert, then infilling with a Product of Experts over a base autoencoder LM, a non-toxic expert, and a toxic anti-expert (Hallinan et al., 2022). On MAgr, SBF, and DynaHate, it achieved the lowest toxicity and best fluency among compared methods, while human raters preferred its rewrites over CondBERT by $2.2\times$ and over ParaGeDi by $1.9\times$ (Hallinan et al., 2022). This is important because subtle toxicity often has no explicit slur to delete.

Prompt-based paraphrasing with instruction-tuned LLMs is another major paradigm. GPT-DETOX uses GPT-3.5 Turbo for zero-shot and few-shot detoxification, with word-matching example selection, context-matching example selection, and ensemble in-context learning (Pesaranghader et al., 2024). Zero-shot prompting achieved strong style accuracy and fluency, few-shot prompting improved semantic preservation, and ensemble selection reached $$J=\frac{1}{n}\sum_{i=1}^{n} STA(x_i)\cdot SIM(x_i)\cdot FL(x_i),$$0 on ParaDetox and $$J=\frac{1}{n}\sum_{i=1}^{n} STA(x_i)\cdot SIM(x_i)\cdot FL(x_i),$$1 on APPDIA (Pesaranghader et al., 2024). Multilingual prompting was later extended through cluster-aware Chain-of-Thought-style prompts that condition GPT-4 on descriptive toxicity features and cluster-specific editing strategies; in a 9-language setup, this achieved the best average joint score among evaluated systems (Dementieva et al., 2024).

4. Preference, reward, context, and representation-based methods

When aligned rewrites are unavailable, one approach is to learn from responses rather than pairwise preferences. Stackelberg Response Optimization formulates detoxification fine-tuning as a game between an LLM and a binary toxicity screener: if a generated paraphrase passes screening, the method applies a DPO-style update against the toxic source; if it fails, it applies a one-sided penalty that simply lowers the probability of that failed paraphrase (Xie et al., 2024). On ParaDetox-style evaluation, SRO reached STA $$J=\frac{1}{n}\sum_{i=1}^{n} STA(x_i)\cdot SIM(x_i)\cdot FL(x_i),$$2, SIM $$J=\frac{1}{n}\sum_{i=1}^{n} STA(x_i)\cdot SIM(x_i)\cdot FL(x_i),$$3, FL $$J=\frac{1}{n}\sum_{i=1}^{n} STA(x_i)\cdot SIM(x_i)\cdot FL(x_i),$$4, and $$J=\frac{1}{n}\sum_{i=1}^{n} STA(x_i)\cdot SIM(x_i)\cdot FL(x_i),$$5, matching the human reference on the aggregate score and outperforming several automatic baselines (Xie et al., 2024).

A related but explicitly RL-based line combines small-scale supervised initialization with reward-guided optimization on unlabeled toxic inputs. One recent framework first performs supervised fine-tuning on 20% of ParaDetox, filtered by a semantic-similarity threshold of $$J=\frac{1}{n}\sum_{i=1}^{n} STA(x_i)\cdot SIM(x_i)\cdot FL(x_i),$$6, then uses GRPO with the reward

$$J=\frac{1}{n}\sum_{i=1}^{n} STA(x_i)\cdot SIM(x_i)\cdot FL(x_i),$$7

where the best reported setting uses $$J=\frac{1}{n}\sum_{i=1}^{n} STA(x_i)\cdot SIM(x_i)\cdot FL(x_i),$$8 (Yu et al., 23 Jun 2025). On ParaDetox, Llama-3.1-8B plus SFT and GRPO achieved STA $$J=\frac{1}{n}\sum_{i=1}^{n} STA(x_i)\cdot SIM(x_i)\cdot FL(x_i),$$9, SIM $0.231$0, FL $0.231$1, and $0.231$2, while also improving OOD performance on DetoxLLM and a HuggingFace detoxification dataset (Yu et al., 23 Jun 2025).

Context-aware methods argue that toxic continuations are often driven by toxic prompts. CMD first detoxifies the context and then trains the model to generate along the detoxified context, using a toxic contrastive loss

$0.231$3

with $0.231$4 in experiments (Tang et al., 2023). Across GPT2-XL, Flan-T5-XL, Alpaca-7B, and LLaMA2 models, CMD reduced Expected Maximum Toxicity and Toxicity Probability while also lowering perplexity relative to the base models (Tang et al., 2023).

Diffusion-based detoxification treats the task as one-to-many conditional generation. DiffuDetox mixes a conditional diffusion model trained on toxic-to-neutral pairs with an unconditional diffusion model trained to reconstruct fluent text; conditional training is selected with probability $0.231$5, the guidance weight is $0.231$6, and inference samples 20 candidates before selecting the best using $0.231$7 (Floto et al., 2023). On ParaDetox evaluation, DiffuDetox achieved STA $0.231$8, SIM $0.231$9, FL $0.802$0, and $0.802$1, improving on its conditional-only ablation and exceeding the human reference on the reported $0.802$2 score (Floto et al., 2023).

Two further paradigms operate on representations rather than full rewrites. One paper hypothesizes that toxic information in pretrained LM embeddings lies in a low-dimensional linear subspace and detoxifies sentence representations by projecting that subspace away: $0.802$3 showing that toxic predictions collapse to near-random after projection while cosine similarity remains relatively high (Wang et al., 2021). UniDetox instead distills “detoxifying text” by contrastive decoding between a base model and a toxic variant, then fine-tunes arbitrary target LLMs on that synthetic text; GPT-2-distilled data transferred to OPT, Falcon, and LLaMA-2 without model-specific hyperparameter retuning (Lu et al., 29 Apr 2025). These methods broaden the notion of detoxification beyond direct rewriting.

5. Multilinguality, cross-lingual transfer, and low-resource systems

Multilingual detoxification has developed along two axes: expanding parallel corpora and studying transfer across languages. Early experiments with mT5 and mBART showed that multilingual fine-tuning on English and Russian can support multilingual detoxification, but zero-shot cross-lingual detoxification does not work well; direct fine-tuning on the target language was found to be necessary (Moskovskiy et al., 2022). Later work on English–Russian transfer provided a more nuanced picture: backtranslation gave the best automatic scores, but for English-to-Russian transfer an adapter-based mBART variant was judged best by humans because it preserved content much better than backtranslation (Dementieva et al., 2023).

The presence of target-language parallel data repeatedly changes what is feasible. MultiParaDetox showed that once such data exist, fine-tuned models surpass unsupervised baselines and prompted LLMs for Russian, Ukrainian, and Spanish (Dementieva et al., 2024). The 9-language explainable setup extended this observation to German, Chinese, Arabic, Hindi, and Amharic, and also found that edit patterns differ across languages: Spanish favored deletion, Chinese and Russian heavily favored rephrasing, and insertion was rare (Dementieva et al., 2024). A plausible implication is that multilingual detoxification is not a single monolithic transfer problem; the preferred rewrite operation itself is language-dependent.

Low-resource studies reinforce this conclusion. BanglaNirTox used Pareto-optimized LLM generation with Chain-of-Thought prompting to create a Bengali parallel corpus with rationales and toxicity classes, then fine-tuned Bengali-capable models for detoxification (Mohsin et al., 3 Nov 2025). Tatoxa for Tatar used adapted Russian–Tatar NLLB translation, synthetic translated parallel pairs, LoRA fine-tuning of mT0-XL, and 180-candidate reranking by neutrality and semantic similarity; it achieved $0.802$4 on CLEF-Tatar and outperformed commercial LLM baselines on the joint score (Alimova et al., 24 Jun 2026). A 15-language PAN system combined multilingual toxic lexicons, <toxic> ... </toxic> tags, a fine-tuned s-nlp/mt0-xl-detox-orpo model, and a classifier gate, reaching STA $0.802$5 and official $0.802$6 (Lai-Lopez et al., 24 Jul 2025).

The strongest low-resource result may be conceptual rather than numerical: native data matter. In Tatar cross-lingual experiments, training on Tatar data yielded the highest $0.802$7, while transfer from Russian performed unexpectedly poorly despite cultural proximity and larger Russian corpora (Alimova et al., 24 Jun 2026). Similarly, the isiXhosa–Yorùbá study favored an interpretable TF-IDF and Logistic Regression detector plus curated lookup and token-level rewriting over heavy generative models, arguing that controllability and cultural adaptation can outweigh raw generative capacity in under-resourced settings (Agbeyangi, 9 Jan 2026).

6. Limitations, controversies, and open directions

The central difficulty in text detoxification remains the trade-off among detoxification strength, semantic preservation, and fluency. Systems that maximize style transfer accuracy often rewrite too aggressively, while systems that preserve wording too conservatively leave residual toxicity. This is visible in benchmark comparisons where translated or prompted models attain very high STA at the cost of semantic collapse, and in edit-based systems that sometimes preserve local syntax but retain hostile stance (Mukherjee et al., 2024, Pesaranghader et al., 2024).

A second recurring issue is dependence on classifiers, reward models, and screeners. SRO is highly sensitive to screener quality: on its benchmark, even 10% relaxation of the screener reduced $0.802$8 from $0.802$9 to $\alpha$0, and 10% overkill reduced it to $\alpha$1 (Xie et al., 2024). RL-based approaches likewise inherit the blind spots of the toxicity classifier used in the reward; one recent study explicitly warns about reward hacking and classifier bias, noting that subtle toxicity may survive if the reward model misses it (Yu et al., 23 Jun 2025). Evaluation metrics have similar problems. Several papers caution that automatic metrics can be brittle, especially in multilingual or adversarial settings, and that style accuracy alone is an inadequate objective (Gusev, 2022).

Implicit toxicity and culturally specific offense remain especially hard. MaRCo was motivated precisely by the failure of lexicon-based approaches on microaggressions and veiled hate (Hallinan et al., 2022). Multilingual work further shows that toxicity expressions are culture- and language-specific: homosexual references were salient in Ukrainian, Russian, and Chinese; animal-based insults in Hindi and Amharic; and German toxic wordplay reflected local political discourse (Dementieva et al., 2024). This suggests that simple transfer of English-centric resources will remain limited unless coupled with native data or culturally grounded supervision.

There is also a normative controversy around what detoxification should do. Some papers explicitly position detoxification as preferable to outright deletion because it preserves communicative intent, while others warn that detoxification can be misused to launder abusive intent into superficially acceptable text (Yu et al., 23 Jun 2025, Mohsin et al., 3 Nov 2025). Several low-resource and multilingual papers therefore emphasize suggestion-style deployment, human oversight, and careful attention to identity terms, dialectal variation, and annotator bias (Alimova et al., 24 Jun 2026, Agbeyangi, 9 Jan 2026).

Open directions follow directly from these tensions. One is richer supervision: binary toxic/non-toxic signals are coarse, and several papers suggest that finer-grained rationales, class labels, or response signals can improve learning (Xie et al., 2024, Mohsin et al., 3 Nov 2025). Another is better multilingual generalization without sacrificing target-language specificity, since current evidence indicates that multilinguality alone does not induce cross-lingual detoxification (Moskovskiy et al., 2022). A third is evaluation that better tracks human judgments on subtle toxicity and semantic preservation, since improvements in benchmark $\alpha$2 do not automatically imply faithful or socially appropriate rewriting (Dementieva et al., 2023). The broader trajectory of the field suggests not a single dominant solution, but an increasingly heterogeneous toolbox matched to data availability, language, toxicity type, and deployment constraints.