Antislop Sampler: Repetitive Pattern Suppression

• Antislop Sampler is a decoding mechanism designed to suppress repetitive, formulaic phrases (slop) in large language models through a backtracking and resampling procedure.
• It employs a tunable ban-strength parameter and mathematical probability adjustments to efficiently handle thousands of banned sequences while maintaining vocabulary coherence.
• Combined with Final Token Preference Optimization (FTPO), the framework preserves output quality and lexical diversity, making it ideal for quality-critical applications.

The Antislop Sampler is a decoding and inference-time suppressor, developed as part of the Antislop framework, for mitigating repetitive, overused phraseology—termed “slop”—in LLMs. “Slop” describes recurring strings, motifs, and formulaic expressions frequently appearing in machine-generated text but rarely present in human writing. The Antislop Sampler operates by locally suppressing banned sequences in real-time generation through a backtracking procedure, as opposed to destructive approaches like token removal. This enables granular control over phrase suppression, preservation of overall vocabulary coherence, and retention of fluidity, even with extensive banlists.

1. Mechanism and Algorithmic Structure

The Antislop Sampler accumulates the complete inference trace during generation, subsequently scanning for banned strings. Bans may be individual words, multi-word phrases, or regex patterns (e.g., “It’s not X, it’s Y”). When a banned pattern is detected, the sampler backtracks to the token where the sequence begins, modifies the generation probabilities, and resamples. Suppression is mathematically formalized:

$p_{\text{new}} = p_{\text{old}} \cdot 10^{-10s}$

where $s \in [0, 1]$ is a configurable ban-strength parameter (0 = no suppression, 1 = hard ban). Rather than outright exclusion, soft-banning ensures user-requested banned phrases are still possible, preserving output flexibility.

After probability adjustment, min‑p filtering is applied, discarding candidates with insufficient likelihood, ensuring coherent alternative continuations. To avoid infinite looping, re-sampling of the same token within identical context causes the violation to be ignored on subsequent checks.

Pseudocode (from the source):

While generating tokens generate token t If banned_pattern detected then backtrack to pattern start, reduce probability using p_new = p_old · 10^(–10·s) resample with min‑p filtering EndIf EndWhile

This local, context-sensitive, reversible modification distinguishes the Antislop Sampler from other suppression algorithms, avoiding disruption to the global vocabulary and overall model semantics.

2. Comparison to Alternative Suppression Methods

Direct token banning removes tokens initiating banned sequences, but interferes with decoding, penalizes valid usage, and quickly becomes untenable with large banlists; the approach becomes “unusable” past 2,000 banned items due to extensive collateral damage. In contrast, the Antislop Sampler supports suppression of 8,000+ patterns without notable degradation in output quality.

Direct Preference Optimization (DPO) functions as a training-time alternative, optimizing model preference according to label pairs. While DPO achieves 80–82% slop suppression, it results in a substantial reduction in writing quality: 6–15 points (on a 100-point expert rubric) and depleted lexical diversity (74–92% baseline retention). The Antislop Sampler, in combination with Final Token Preference Optimization (FTPO), maintains or even slightly improves writing quality under high suppression rates (83–92%) and supports banlists far exceeding those manageable through token-level exclusion.

3. Final Token Preference Optimization (FTPO)

FTPO is a fine-tuning protocol for permanent suppression of slop. Rather than tuning on complete sequences, FTPO targets the final token before an unwanted pattern commences, forming a training triple:

• Prompt + response up to the illicit pattern start
• First token of the banned sequence (rejected token)
• Set of contextually suitable alternative tokens

The FTPO loss function:

• Preference loss: Enforces margin $m$ between candidate and rejected token logits:

$$L_{\text{pref}} = \frac{\sum_{c \in C} w_c \cdot \text{softplus}\left(\frac{m - (y[c] - y[r])}{\tau}\right)}{\sum_{c \in C} w_c}$$

where $w_c$ is a tapering function constraining the learning signal as margin is achieved.

• Target regularization: Mean squared error on chosen (“target”) tokens.
• Non-target regularization: Anchors remaining tokens to their reference logits.

Combined loss:

$$L_{\text{FTPO}} = L_{\text{pref}} + \lambda_{\text{target}} \cdot L_{\text{target}} + \lambda_{\text{nontarget}} \cdot L_{\text{nontarget}}$$

Gradient switch-offs eliminate learning signal once margin is met, reducing risk of overtraining. FTPO supports simultaneous updates across multiple alternative tokens, yielding 90% reduction in slop with unchanged or improved performance on GSM8K (math), MMLU (multidomain knowledge), and creative writing evaluations. This is a notable improvement over DPO, which both suppresses less and damages output diversity and quality.

4. Quantitative Performance and Diversity Metrics

The Antislop Sampler and FTPO were evaluated on functional and creative metrics:

• GSM8K, MMLU: FTPO maintained 97–99% of baseline performance; output quality remained within 1–3% of baseline.
• Expert writing analysis: Quality assessed via spelling/grammar, formatting, coherence, tense consistency, avoidance of repetition, and overall fluency. FTPO and the sampler preserved or modestly improved ratings compared to baseline; token banning and DPO degraded them.
• Lexical diversity: Aggregate metrics, including MATTR-500, Root-TTR, HD-D, and Distinct-n, demonstrate that FTPO retained 95–102% of model baseline diversity, whereas DPO dropped to 74–92%.

Table: Summary of Suppression and Quality Metrics (as reported)

Method	Max Banlist Size	Slop Suppression (%)	Writing Quality Impact
Token Banning	~2,000	65–70	Severe Degradation
DPO	--	80–82	−6 to −15 points
Antislop Sampler	8,000+	100	Baseline or Improved
FTPO	--	90	Baseline or Improved

This demonstrates the scalability and quality preservation of the Antislop approach in suppressing widespread slop phenomena in LLM outputs.

5. Implementation and Practical Considerations

The Antislop Sampler introduces a tunable ban-strength parameter, enabling flexible enforcement of pattern suppression. While soft bans avoid total collapse on adversarial or user-driven prompts, the backtracking and resampling procedure can reduce throughput: the most severe cases showed 69%–96% output slowdown.

A practical implication is its suitability for quality-critical applications where generation speed is secondary, such as editorial content production or creative writing assistance. As output from the sampler can be utilized to generate training data for FTPO, model developers can encode suppression directly into the weights, eliminating need for inference-time interventions and restoring throughput for deployed models.

The fine-tuning approach provided by FTPO preserves core linguistic capabilities while ensuring reproducible, context-respecting avoidance of slop. A plausible implication is that this combination is optimal in scenarios requiring both high output quality and robust suppression of formulaic expressions.

6. Codebase and Reproducibility Resources

All components of the Antislop framework—including the Antislop Sampler, slop profiling pipeline, and FTPO fine-tuning tools—are released under an MIT license. The repository (https://github.com/sam-paech/auto-antislop) provides:

• HuggingFace-based single-threaded implementations (with support for streaming).
• Multi-threaded implementations compatible with OpenAI API and vLLM for production use.
• Performance benchmarks, configuration examples (e.g., for gemma-3-12b), and automated slop fingerprinting.
• Scripts for iterative training data generation and reproducibility.

These resources facilitate direct adoption, experimental replication, and domain adaptation for both research and production settings.

7. Context and Significance

The Antislop Sampler, as detailed in "Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in LLMs" (Paech et al., 16 Oct 2025), provides a targeted solution to the growing challenge of formulaic, immediately recognizable outputs from current LLMs. Its approach—backtracking to selectively suppress banned sequences without penalizing overall vocabulary, combined with the FTPO protocol for targeted fine-tuning—demonstrates superior slop suppression, high resilience to scaling (banlist length), and retention of lexical diversity and quality, outperforming traditional methods including token banning and DPO.

The framework establishes a paradigm for stylistic refinement in LLM generations, balancing the demands for humanlike text variability with robustness in utility, making it especially relevant for applications in academic, literary, and editorial contexts. The public codebase further encourages adoption, cross-domain adaptation, and future research into repetitive pattern minimization and output naturalization in LLMs.