---
title: 'RSA-Control: Controllable Text Generation'
url: https://www.emergentmind.com/topics/rsa-control
type: topic
---

# RSA-Control: Controllable Text Generation

RSA-Control is a pragmatics-grounded, training-free framework for controllable text generation, designed to modulate the output of pretrained autoregressive language models (LMs) according to target attributes, such as toxicity, bias, or readability, while maintaining content fidelity and fluency. RSA-Control leverages recursive speaker-listener reasoning drawn from the Rational Speech Acts (RSA) paradigm in pragmatics, combining two levels of imaginary agents—a literal speaker and a pragmatic listener—culminating in a pragmatic speaker that mixes content and attribute utilities. The method is notable for operating without any model fine-tuning or external discriminators, relying solely on prompting and likelihood computations from a single pretrained LM [2410.19109].

## 1. Formal Model Structure: Components and Definitions

RSA-Control is built around two core promptable modules derived from an underlying pretrained autoregressive LM: the literal speaker ($S_0$) and the pragmatic listener ($L_1$), operating in tandem with a context variable $c$ and target attribute $a \in A$, with $A$ denoting the set of candidate attributes.

- **Base Language Model (LM):** The content generator is any autoregressive LM yielding token-wise conditional probabilities:
  $$
  P_{LM}(w_n \mid w_{<n},\,c)
  $$
  where $w_n$ is the next token, $w_{<n}$ the prefix, and $c$ the content context.

- **Literal Speaker ($S_0$):** To induce attribute-specific behaviors, $S_0$ repurposes the LM by prepending attribute-specific control prompts:
  $$
  P_{S_0}(w_n \mid w_{<n},\,a) = P_{LM}(w_n \mid w_{<n},\,\text{prompt}(a))
  $$
  Example: prompt$(a)$ might be “The following text is polite, respectful…” for “non-toxic” generation.

- **Pragmatic Listener ($L_1$):** $L_1$ inverts the generative process to infer the likelihood of each attribute $a$ from partially decoded output, treated as Bayesian evidence:
  $$
  P_{L_1}(a \mid w_{\le n}) = \frac{P_{S_0}(w_n \mid w_{<n}, a) \, P_{L_1}(a \mid w_{<n})}{\sum_{a' \in A} P_{S_0}(w_n \mid w_{<n}, a') P_{L_1}(a' \mid w_{<n})}
  $$
  with uniform prior $P_{L_1}(a) = 1 / |A|$ at the start of generation.

## 2. Recursive Pragmatic Reasoning and Utility Construction

RSA-Control truncates the classic RSA speaker/listener recursion at the first pragmatic speaker, $S_1$, which aggregates utilities using the literal speaker and pragmatic listener.

- **Content Utility:**
  $$
  U_c(w_n \mid w_{<n}, c) = \log P_{LM}(w_n \mid w_{<n}, c)
  $$
- **Attribute Utility:**
  $$
  U_a(w_n \mid w_{<n}, a) = \log P_{L_1}(a \mid w_{\le n})
  $$
- **Total Utility:** The pragmatic speaker $S_1$ computes a weighted sum:
  $$
  U(w_n \mid w_{<n}, c, a) = U_c(w_n \mid w_{<n}, c) + \alpha U_a(w_n \mid w_{<n}, a)
  $$
  where $\alpha \ge 0$ is the rationality parameter.

- **Pragmatic Speaker ($S_1$):** The next-token distribution is:
  $$
  P_{S_1}(w_n \mid w_{<n}, c, a) \propto P_{LM}(w_n \mid w_{<n}, c) \cdot [P_{L_1}(a \mid w_{\le n})]^\alpha
  $$
  Normalization is performed across the vocabulary. The blending of content and attribute signals here is central for balancing generation fidelity against attribute control [2410.19109].

## 3. Adaptive Rationality Parameter for Contextual Control

Distinct from classical RSA, RSA-Control introduces a self-adjusting, token-level rationality parameter $\tilde{\alpha}_n$, adapting control strength to local decoding circumstances.

- **Content Ratio:**
  $$
  r^c_n = \frac{P_{LM}(t \mid w_{<n}, c; \alpha=\alpha_0)}{P_{LM}(t \mid w_{<n}, c; \alpha=0)}
  $$
- **Attribute Ratio:**
  $$
  r^a_n = \frac{P_{L_1}(a \mid w_{<n} \oplus t; \alpha=\alpha_0)}{P_{L_1}(a \mid w_{<n} \oplus t; \alpha=0)}
  $$
  By construction, $r^c_n \leq 1$ and $r^a_n \geq 1$, ensuring $\frac{r^c_n}{r^a_n} \in [0,1]$.

- **Adaptive Rationality:**
  $$
  \tilde{\alpha}_n = \alpha_0 + \frac{r^c_n}{r^a_n} \cdot \alpha_1
  $$
  with $\tilde{\alpha}_n$ lying in $[\alpha_0, \alpha_0+\alpha_1]$.

- **Resulting Token Distribution:**
  $$
  P_{S_1}(w_n \mid w_{<n}, c, a) \propto P_{LM}(w_n \mid w_{<n}, c) \cdot [P_{L_1}(a \mid w_{\le n})]^{\tilde{\alpha}_n}
  $$
  This mechanism ensures the pragmatic speaker dynamically modulates control, increasing $\alpha$ where attribute adherence is weak but capping it to mitigate degradation in fluency or content.

## 4. Decoding Procedure

The RSA-Control procedure iterates at every decoding step and is agnostic to the outer sampling or beam search strategy. The workflow is as follows:

```
Inputs:
  - base LM, context c, attribute prompt a
  - control prompts for all a' ∈ A (for S_0)
  - fixed hyperparameters α0, α1
  - current prefix w_<n

Step n of decoding:
1. Compute the top-k next tokens {t₁,…,t_k} and their LM log-probs ℓ_c(t_i)
   ℓ_c(t_i) ← log P_LM(t_i | w_<n , c)
2. For each candidate t_i:
   - compute listener log-prob under S₀ prompts:
       ℓ_a_orig(i)   ← log P_L1(a | w_<n ⊕ t_i)    // with basic α0
       ℓ_a_zerocontrol(i) ← log P_L1(a | w_<n ⊕ t_i)    // with α=0
   - compute content ratio r^c_i and attribute ratio r^a_i
       r^c_i ← exp(ℓ_c(t_i;α0) - ℓ_c(t_i;α=0))
       r^a_i ← exp(ℓ_a_orig(i)   - ℓ_a_zerocontrol(i))
   - compute adaptive rationality
       \tilde α_i ← α0 + (r^c_i / r^a_i) * α1
   - compute final score
       score(t_i) ← ℓ_c(t_i)  +  \tilde α_i * ℓ_a_orig(i)
3. Renormalize the score vector {score(t_i)} into a distribution over t_i.
4. Select or sample the next token w_n from this distribution.
5. Update listener state L₁ with the chosen w_n via Bayes update.

Repeat until end-of-sequence.
```

No additional training or model parameters are introduced—the approach is “training-free” and makes exclusive use of the original pretrained LM for both content and attribute modeling. Control prompts for multiple attributes/distractors are managed by prepending the string descriptions to the LM input [2410.19109].

## 5. Empirical Evaluation and Main Results

Extensive experiments evaluate RSA-Control on toxicity reduction, stereotypical bias mitigation, and readability-controlled summarization. Results are summarized in the table below:

| Task/Metric                         | Vanilla LM        | Self-Debias    | RSA-Control          |
|-------------------------------------|-------------------|----------------|---------------------|
| **Toxicity (GPT-2-large, RealToxicityPrompts)** | 22.2% avg toxicity, PPL 27.48 | 11.1%, PPL 39.27 | 8.8%, PPL 42.53      |
| **Manual Toxicity (Human Eval 50 prompts)**   | —                 | 0.39           | 0.28, fluency ≈4.69/5| 
| **Stereotypical Bias (GPT-2-large, CrowS-Pairs, Race)** | 62.2%             | 54.8%          | 45.9%                |
| **Readability Control (Llama-2-7b-chat, CNN/DM, readable)** | FRE 76.07        | —              | FRE 79.58, DCR 7.52  | 
| **Summarization Quality Drop**      | BERTScore 86.28   | —              | BERTScore 84.94     |

- **Toxicity:** RSA-Control achieves the lowest toxicity rates among prompt-based methods tested, both using automated (Perspective API) and manual evaluation, with minor degradation in perplexity and fluency.
- **Bias Mitigation:** RSA-Control surpasses Self-Debias on 8/9 bias types, yielding substantial reductions in the stereotypical bias probability.
- **Readability-Controlled Summarization:** RSA-Control boosts readability metrics (FRE: +3.51, DCR, GFI, CLI all improved), with some loss in similarity metrics (BERTScore, ROUGE-L), but achieves strong attribute adherence and faithfulness scores in human evaluation.

The method thus demonstrates that recursive, pragmatics-grounded decoding with an adaptive rationality schedule yields consistent improvements in attribute control for multiple LMs and generation tasks, balancing attribute adherence with preservation of content fidelity [2410.19109].

## 6. Interpretation, Context, and Impact

RSA-Control exemplifies a new direction in controllable generation that eschews task-specific fine-tuning or adversarial discriminators in favor of a lightweight, prompt-based, and training-free framework, operating exclusively at the decoding stage. By leveraging recursive reasoning between prompted agents within an existing LM, it achieves high controllability across a spectrum of attributes and demonstrates the functional value of pragmatic theories in LM decoding. This suggests a broad potential applicability to a range of generative control tasks where attribute adherence, faithfulness, and fluency must be balanced without invasive model modifications or computationally expensive training cycles.

Source: https://www.emergentmind.com/topics/rsa-control