---
title: Anchor-Conditioned Token Generation (ACTG)
url: https://www.emergentmind.com/topics/anchor-conditioned-token-generation-actg
type: topic
---

# Anchor-Conditioned Token Generation (ACTG)

Anchor-Conditioned Token Generation (ACTG) specifies a family of conditional text generation frameworks in which generation is guided by a set of “anchor” tokens or attributes that act as explicit conditions or control points. These systems enforce the inclusion and order of given anchors in the generated text, enabling fine-grained instruction following, targeted editing, and privacy-preserving synthesis by decoupling the specification of control features from the underlying generative mechanism. Contemporary ACTG frameworks employ hierarchical decompositions, multi-agent reinforcement learning, adversarial signal optimization, and differentially private mechanisms to realize anchored control at scale [2005.02794][2510.18232].

## 1. Formal Problem Definition

ACTG formulates generation as learning a mapping from a set of anchors (A or f) to a target sequence S (or text x), enforcing that the output sequence respects anchor constraints. Specifically, the basic structure is as follows:

- **Anchor tokens**: $A = \{a_1, ..., a_m\}$ are provided as immutable sequence constraints (“seed” tokens or structured attributes).
- **Target sequence**: $S = \{s_1, ..., s_n\}$ must include all anchors in the prescribed order, with intervening or manipulated content generated to fill in gaps or extend context.
- **Blank positions**: $B \subseteq \{1,...,n\}$ denotes positions selected for replacement or insertion by a dedicated agent.
- **Objective**: Maximize the conditional likelihood $P(S|A) = \prod_t P(s_t | s_{<t}, A)$, optionally under adversarial and privacy constraints.

In privacy-constrained ACTG [2510.18232], anchors are extracted as categorical schema fields $f=\phi_S(x)$ and both anchor synthesis $G_f$ and conditional sequence generation $G_{x|f}$ are required to be $(\epsilon, \delta)$-differentially private.

## 2. Hierarchical Task Decomposition and Multi-Agent RL

Token Manipulation GAN (Token-MANGAN) [2005.02794] operationalizes ACTG via a hierarchical multi-agent reinforcement learning architecture:

- **Make-a-Blank Agent ($G_{man}$)**: Given partial output $y_{<t}$, full anchor set $A$, and an anchor pointer idx, this agent decides to (a) insert a blank for generation, (b) consume the next anchor, (c) replace an anchor to be refilled, or (d) pass (rarely used).
- **Fill-in-the-Blank Agent ($G_{tok}$)**: Activated upon “add” or “replace” actions, it fills the specified position with a sampled token from the vocab $V$.
- **Policy updates**: Both agents are updated via policy gradients, maximizing expected cumulative reward $R(\tau)=\sum_{t=1}^T \gamma^{T-t} r_t$ with advantage estimation $A_t = R(\tau) - V_\phi(s_t)$, where $V_\phi$ is a learned critic.

This architecture allows dynamic manipulation of anchor placement and content in the output sequence, supporting both insertion and selective replacement in an end-to-end setup.

## 3. Conditional Adversarial and Privacy-Preserving Learning

ACTG frameworks integrate adversarial learning and differential privacy as follows:

- **Adversarial loss (Token-MANGAN)**: The generator optimizes against a discriminator $D(S,A)$ trained to distinguish human-written $(S,A)$ pairs from machine-generated $(\hat{S},A)$; generator and discriminator losses are formulated analogously to GAN literature.
- **Differential privacy (ACTG-ARL)**: Both anchor synthesis $G_f$ (via AIM, an adaptive tabular synthesizer) and conditional text generator $G_{x|f}$ (via DP-Adam) enforce $(\epsilon_1, \delta)$- and $(\epsilon_2, \delta)$-DP respectively, with total budget $\epsilon_1+\epsilon_2\leq\epsilon$. Feature extraction stages incur no privacy cost when implemented via trusted LLM oracles.
- **Reward design**: In privacy-preserving settings, the Reyenforcement Learning reward signals are defined by field-wise anchor matching $r(f, x)=\frac{1}{K} \sum_{k=1}^K \mathbf{1}[f_k = \phi_S(x)_k]$.

Table 1: Summary of ACTG Optimization Objectives

| Framework      | Conditioning      | Adversarial Signal    | Privacy Guarantee     |
|----------------|------------------|----------------------|----------------------|
| Token-MANGAN   | Hard anchor tokens| Discriminator loss   | None                 |
| ACTG-ARL       | Rich tabular schema| RL + best-of-N anchor| $(\epsilon,\delta)$-DP|

## 4. Model Architectures and Training Schedules

Architectural designs in ACTG systems reflect the decomposition of generation and control tasks:

- **Token-MANGAN [2005.02794]**:
    - Both agents $G_{man}$, $G_{tok}$: 2-layer LSTMs (embedding dim $d=300$, hidden $h=512$), action heads for manipulation and vocab.
    - Critic: single-layer MLP on hidden state.
    - Discriminator: 2-layer uni-LSTM ($h=512$) with sigmoid output.
    - Hyperparameters: Generator LR $1e$-$4$, Discriminator LR $5e$-$5$, RL discount $\gamma=0.95$, batch size 64, vocab $\sim$5,000.

- **ACTG-ARL [2510.18232]**:
    - Feature extractor via LLM oracle.
    - DP tabular synthesizer (AIM) for anchors.
    - Conditional LM (gemma-3-1b-pt, fine-tuned via DP-Adam and LoRA).
    - Post-training RL (PPO surrogate) using anchor-matching reward.
    - Hybrid loss combining RL objective and best-of-$N$ supervised fine-tuning.

Training schedules typically involve an initial MLE phase (teacher-forcing on reference data), followed by adversarial RL or RL-boosted control with privacy constraints.

## 5. Pseudocode and Workflow

Token-MANGAN’s adversarial multi-agent RL training can be summarized:
```python
for epoch in 1…E_adversarial:
  for batch in data:
    A ← sample_anchors(batch)
    Ŝ ← rollout(G_man, G_tok | A)
    rewards ← [ log D(Ŝₜ | Ŝ₍<ₜ₎,A) for t in 1…T ]
    R ← discounted_sum(rewards, γ)
    adv ← R − Vϕ(Ŝ states)
    θ_man ← θ_man + α * Σₜ advₜ ∇ log G_man(aₜ|sₜ)
    θ_tok ← θ_tok + α * Σₜ advₜ ∇ log G_tok(yₜ|sₜ)
    ϕ    ← ϕ − α_c * ∇(Vϕ(sₜ) − R)²
    D    ← D − α_d * ∇_D L_D(real=(S,A), fake=(Ŝ,A))
```
In ACTG-ARL [2510.18232], post-processing comprises:
1. Feature extraction: anchors from private corpus.
2. DP tabular anchor synthesis with AIM.
3. DP fine-tuned conditional LM.
4. Best-of-$N$ anchor dataset construction.
5. RL rounds with PPO updates for anchor matching.
6. Hybrid SFT + RL loss for final model selection.

## 6. Evaluation Criteria and Comparative Results

ACTG systems are evaluated on both traditional and specialized metrics:

- **Content quality**: BLEU($n$), perplexity, semantic alignment (MAUVE).
- **Diversity**: self-BLEU.
- **Control accuracy**: instruction-following accuracy (IFAcc), per-field anchor match.
- **Distributional alignment**: mean Jensen–Shannon distance $d_{JS}^f$ between private and synthetic anchor distributions.
- **Privacy metrics**: $(\epsilon, \delta)$-DP compliance, error decomposition traced by RDP/PLD accountants.

Representative results include:

- On COCO captions [2005.02794], Token-MANGAN improves quality-diversity tradeoff over MaskGAN and SeqGAN at high mask rates (BLEU-5 at mask=0.5: MaskGAN GAN −0.23, Token-MANGAN −0.19; lower is better).
- On bioRxiv [2510.18232], ACTG (“schema + AIM + DP-FT”) achieves MAUVE = 0.775 vs CTCL = 0.647 (+20%), $d_{JS}^f = 0.087$ vs CTCL = 0.175 (−50%), IFAcc under DP = 0.534, and ACTG-ARL boosts IFAcc to ≈0.65 without collapse in MAUVE (≈0.76).

## 7. Limitations and Prospective Directions

Noted limitations of current ACTG instantiations:

- Hierarchical RL with discrete action spaces exhibits sample inefficiency [2005.02794].
- Scalability to long output sequences ($n \gg 20$) is constrained by policy gradient variance and architectural bottlenecks.
- Fixed manipulator action sets; limited span-level editing and context-aware operation.
- Reward hacking can occur in RL-boosted setups, necessitating hybrid objectives (best-of-$N$ SFT anchors) [2510.18232].
- Model architectures still rely on LSTMs; Transformer-based replacements are proposed for improved context handling.
- Rich tabular schema selection substantially affects anchor distributional fidelity; the greatest improvement opportunity lies in conditional text generation error.

Future enhancements include replacing LSTMs with Transformers, augmenting reward signals (coherence, topic coverage), and extending manipulators to span-level control. The hybrid ACTG-ARL approach is shown to restore instruction fidelity under privacy constraints and sets new benchmarks in differentially private conditional text generation [2510.18232].

Source: https://www.emergentmind.com/topics/anchor-conditioned-token-generation-actg