TimpaTeks: In-Place Diffusion Text Steering

• TimpaTeks is an automatic in-place text modification framework that applies activation steering in diffusion language models to transform text concepts while maintaining syntactic integrity.
• It employs bidirectional masked denoising and selective hidden state manipulation to perform targeted, efficient token replacements without retraining.
• Experimental evaluations on sentiment and synthetic datasets demonstrate robust conceptual shifts, fluency retention, and significant computational savings compared to prompt-based approaches.

TimpaTeks is an automatic, in-place text sequence modification framework leveraging activation steering in Diffusion LLMs (DLMs). It enables the targeted transformation of a text’s “concept”—such as sentiment or topic—while maintaining the original sequence length and syntactic structure. Through bidirectional masked denoising and direct manipulation of hidden representations, TimpaTeks achieves efficient and controlled conceptual changes in text without retraining or instruction tuning, with reduced computational overhead compared to prompt-based DLM steering (Diandaru et al., 7 Jun 2026).

1. Problem Definition and Motivation

TimpaTeks addresses the problem of in-place text modification: given a discrete sequence of tokens $x = (x_1, ..., x_N)$ exhibiting a source concept $c_1$, the objective is to modify $x$ to manifest a target concept $c_2$, preserving sentence length ($|x| = |x'| = N$) and narrative structure. In contrast to unconditional generation or left-to-right autoregressive editing, in-place modification restricts interventions to a (possibly adaptive) subset of token positions $R \subset \{1, ..., N\}$, which are masked and replaced. This “surgical” approach is enabled by the bidirectional and token-agnostic denoising process of DLMs, distinguishing TimpaTeks from prompt-based or sequential generative techniques (Diandaru et al., 7 Jun 2026).

2. Mathematical Formulation

TimpaTeks operates on the hidden state space of DLMs, such as LLaDA, using activation steering:

• Steering vector construction: For each Transformer layer $l$, compute class-conditional mean hidden representations $$V_c^l = \frac{1}{M}\sum_{x\in \mathcal S_c} \frac{1}{N_x}\sum_{i=1}^{N_x} h_i^l(x)$$, with $c \in \{c_1, c_2\}$. These are $\ell_2$-normalized, and the steering vector is $c_1$0, optionally scaled per layer by $c_1$1.
• Steering injection: For hidden states $c_1$2, the modified (steered) hidden state is $c_1$3 for all $c_1$4, $c_1$5.
• Token replacement selection: For each token $c_1$6, average the cosine similarity between $c_1$7 and $c_1$8 over a specified subset of layers $c_1$9: $x$0. A masking probability is obtained as $x$1, where $x$2 is a temperature parameter.

This methodology targets tokens least aligned with the steering direction for replacement, anchoring conceptual edits while maintaining local and global syntactic integrity (Diandaru et al., 7 Jun 2026).

3. Inference Procedure

TimpaTeks comprises an iterative steering-refilling routine:

1. Initialization: Set $x$3.
2. Steering iterations (for $x$4):
   • Obtain hidden states $x$5 via forward pass.
   • Sample replacement set $x$6 according to $x$7.
   • Generate masked sequence $x$8 by masking tokens in $x$9.
   • Refilling is performed over $c_2$0 sub-steps: in each, select $c_2$1 masks with highest model confidence; refill these positions; terminate if no masks remain.
   • Update $c_2$2.
3. Termination: Output the final modified sequence $c_2$3.

Key properties: Sequence length is always preserved; only the minimal set of tokens necessary for conceptual steering are replaced; all operations occur in-place without concatenating external prompts (Diandaru et al., 7 Jun 2026).

4. Comparison to Prompt-Based Steering

TimpaTeks fundamentally differs from prompt-based DLM steering in both mechanism and efficiency:

Feature	TimpaTeks	Prompt-Based DLM Steering
Conditioning	Direct hidden state injection	Concatenated instruction+source
Sequence length	$c_2$4	$c_2$5
Denoising passes (FLOPs)	$c_2$6	$c_2$7
Masking	In-place, adaptive	New [MASK] block

TimpaTeks achieves computational savings because it skips prompt concatenation and always operates on the sequence of fixed length $c_2$8, with the FLOPs ratio $c_2$9 favoring TimpaTeks for typical prompt lengths. Prompt-based steering requires models to interpret instructions, while TimpaTeks is zero-shot with respect to concept control (Diandaru et al., 7 Jun 2026).

5. Experimental Results and Evaluation

Experiments evaluate TimpaTeks on IMDB movie reviews (sentiment) and a synthetic CatDog dataset (animal concept steering):

• Datasets: IMDB uses 1,000 train and 1,000 test samples per class; CatDog comprises 2,626 balanced sentences, with 1,000 test samples.
• Metrics:

1. Steering success: Probability of target class (using Qwen2.5-0.5B-Instruct classifier).
2. Coherence & fluency: Sequence perplexity (change and robust-normalized score).
3. Structural retention: Human annotator preference for structural similarity.

Key observations:

• Steering success correlates with the number of steering steps $|x| = |x'| = N$0; most IMDB samples flip polarity within 3–5 iterations.
• Perplexity change remains near zero or negative, indicating fluency retention or improvement.
• Human annotators overwhelmingly preferred TimpaTeks on IMDB (46–49/50), while on CatDog, the prompt baseline was slightly preferred (41–49/50), attributable to the nature of direct noun swaps in synthetic data.
• Steering vectors constructed with small sample sizes (e.g., $|x| = |x'| = N$1) achieve cross-direction harmonic mean validation scores of approximately 0.47 (IMDB) and 0.30 (CatDog) (Diandaru et al., 7 Jun 2026).

6. Analysis, Strengths, and Limitations

TimpaTeks is characterized by the following strengths:

• Selects a minimal subset of tokens for replacement, preserving narrative content.
• Achieves targeted conceptual shifts with maintained or improved fluency (perplexity).
• Demonstrates computational efficiency versus prompt-based approaches, due to in-place operations and hidden state manipulation.
• Robustness to hyperparameter settings (layer choice, steering and sampling temperatures, refilling steps, sentence length) is observed empirically.

Limitations:

• Hyperparameters, including layer selection and $|x| = |x'| = N$2, $|x| = |x'| = N$3, $|x| = |x'| = N$4, $|x| = |x'| = N$5 were not exhaustively optimized.
• Evaluation is limited to a single DLM architecture (LLaDA-8B-Base); generalizability remains untested.
• CatDog’s synthetic nature may not reflect natural language variability.
• No explicit protections against factual inconsistencies (hallucination, entity loss) (Diandaru et al., 7 Jun 2026).

7. Applications and Future Directions

TimpaTeks’ in-place modification paradigm is suitable for tasks requiring preservation of document-level structures while altering deep semantic properties. Suggested extensions include:

• Joint and adaptive optimization of layer-wise steering strengths ($|x| = |x'| = N$6) and refilling schedules via gradient-free methods.
• Application to broader editing tasks such as stylistic transformation or toxicity mitigation.
• Exploration of transfer to stronger or multi-modal DLMs.

A plausible implication is that TimpaTeks may provide a template for efficient, low-disruption controlled editing in generative and discriminative language tasks, contingent on further validation across architectures and use cases (Diandaru et al., 7 Jun 2026).