RSA-Control: Controllable Text Generation

• RSA-Control is a training-free framework that employs dual agents—a literal speaker and pragmatic listener—to balance content fidelity and attribute control.
• It uses recursive reasoning with an adaptive rationality parameter to modulate next-token probabilities based on both the language model's output and target attributes.
• Empirical results demonstrate RSA-Control reduces toxicity, mitigates bias, and enhances readability while preserving fluency in generated text.

RSA-Control is a pragmatics-grounded, training-free framework for controllable text generation, designed to modulate the output of pretrained autoregressive LMs according to target attributes, such as toxicity, bias, or readability, while maintaining content fidelity and fluency. RSA-Control leverages recursive speaker-listener reasoning drawn from the Rational Speech Acts (RSA) paradigm in pragmatics, combining two levels of imaginary agents—a literal speaker and a pragmatic listener—culminating in a pragmatic speaker that mixes content and attribute utilities. The method is notable for operating without any model fine-tuning or external discriminators, relying solely on prompting and likelihood computations from a single pretrained LM (Wang et al., 2024).

1. Formal Model Structure: Components and Definitions

RSA-Control is built around two core promptable modules derived from an underlying pretrained autoregressive LM: the literal speaker ($S_0$) and the pragmatic listener ($L_1$), operating in tandem with a context variable $c$ and target attribute $a \in A$, with $A$ denoting the set of candidate attributes.

• Base LLM (LM): The content generator is any autoregressive LM yielding token-wise conditional probabilities:

$P_{LM}(w_n \mid w_{<n},\,c)$

where $w_n$ is the next token, $w_{<n}$ the prefix, and $c$ the content context.

• Literal Speaker ($S_0$): To induce attribute-specific behaviors, $L_1$0 repurposes the LM by prepending attribute-specific control prompts:

$L_1$1

Example: prompt$L_1$2 might be “The following text is polite, respectful…” for “non-toxic” generation.

• Pragmatic Listener ($L_1$3): $L_1$4 inverts the generative process to infer the likelihood of each attribute $L_1$5 from partially decoded output, treated as Bayesian evidence:

$L_1$6

with uniform prior $L_1$7 at the start of generation.

2. Recursive Pragmatic Reasoning and Utility Construction

RSA-Control truncates the classic RSA speaker/listener recursion at the first pragmatic speaker, $L_1$8, which aggregates utilities using the literal speaker and pragmatic listener.

• Content Utility:

$L_1$9

• Attribute Utility:

$c$0

• Total Utility: The pragmatic speaker $c$1 computes a weighted sum:

$c$2

where $c$3 is the rationality parameter.

• Pragmatic Speaker ($c$4): The next-token distribution is:

$c$5

Normalization is performed across the vocabulary. The blending of content and attribute signals here is central for balancing generation fidelity against attribute control (Wang et al., 2024).

3. Adaptive Rationality Parameter for Contextual Control

Distinct from classical RSA, RSA-Control introduces a self-adjusting, token-level rationality parameter $c$6, adapting control strength to local decoding circumstances.

• Content Ratio:

$c$7

• Attribute Ratio:

$c$8

By construction, $c$9 and $a \in A$0, ensuring $a \in A$1.

• Adaptive Rationality:

$a \in A$2

with $a \in A$3 lying in $a \in A$4.

• Resulting Token Distribution:

$a \in A$5

This mechanism ensures the pragmatic speaker dynamically modulates control, increasing $a \in A$6 where attribute adherence is weak but capping it to mitigate degradation in fluency or content.

4. Decoding Procedure

The RSA-Control procedure iterates at every decoding step and is agnostic to the outer sampling or beam search strategy. The workflow is as follows:

$a \in A$7

No additional training or model parameters are introduced—the approach is “training-free” and makes exclusive use of the original pretrained LM for both content and attribute modeling. Control prompts for multiple attributes/distractors are managed by prepending the string descriptions to the LM input (Wang et al., 2024).

5. Empirical Evaluation and Main Results

Extensive experiments evaluate RSA-Control on toxicity reduction, stereotypical bias mitigation, and readability-controlled summarization. Results are summarized in the table below:

Task/Metric	Vanilla LM	Self-Debias	RSA-Control
Toxicity (GPT-2-large, RealToxicityPrompts)	22.2% avg toxicity, PPL 27.48	11.1%, PPL 39.27	8.8%, PPL 42.53
Manual Toxicity (Human Eval 50 prompts)	—	0.39	0.28, fluency ≈4.69/5
Stereotypical Bias (GPT-2-large, CrowS-Pairs, Race)	62.2%	54.8%	45.9%
Readability Control (Llama-2-7b-chat, CNN/DM, readable)	FRE 76.07	—	FRE 79.58, DCR 7.52
Summarization Quality Drop	BERTScore 86.28	—	BERTScore 84.94

• Toxicity: RSA-Control achieves the lowest toxicity rates among prompt-based methods tested, both using automated (Perspective API) and manual evaluation, with minor degradation in perplexity and fluency.
• Bias Mitigation: RSA-Control surpasses Self-Debias on 8/9 bias types, yielding substantial reductions in the stereotypical bias probability.
• Readability-Controlled Summarization: RSA-Control boosts readability metrics (FRE: +3.51, DCR, GFI, CLI all improved), with some loss in similarity metrics (BERTScore, ROUGE-L), but achieves strong attribute adherence and faithfulness scores in human evaluation.

The method thus demonstrates that recursive, pragmatics-grounded decoding with an adaptive rationality schedule yields consistent improvements in attribute control for multiple LMs and generation tasks, balancing attribute adherence with preservation of content fidelity (Wang et al., 2024).

6. Interpretation, Context, and Impact

RSA-Control exemplifies a new direction in controllable generation that eschews task-specific fine-tuning or adversarial discriminators in favor of a lightweight, prompt-based, and training-free framework, operating exclusively at the decoding stage. By leveraging recursive reasoning between prompted agents within an existing LM, it achieves high controllability across a spectrum of attributes and demonstrates the functional value of pragmatic theories in LM decoding. This suggests a broad potential applicability to a range of generative control tasks where attribute adherence, faithfulness, and fluency must be balanced without invasive model modifications or computationally expensive training cycles.