Test-Time-Matching: Decouple Personality, Memory, and Linguistic Style in LLM-based Role-Playing Language Agent

Abstract: The rapid advancement of LLMs has enabled role-playing language agents to demonstrate significant potential in various applications. However, relying solely on prompts and contextual inputs often proves insufficient for achieving deep immersion in specific roles, particularly well-known fictional or public figures. On the other hand, fine-tuning-based approaches face limitations due to the challenges associated with data collection and the computational resources required for training, thereby restricting their broader applicability. To address these issues, we propose Test-Time-Matching (TTM), a training-free role-playing framework through test-time scaling and context engineering. TTM uses LLM agents to automatically decouple a character's features into personality, memory, and linguistic style. Our framework involves a structured, three-stage generation pipeline that utilizes these features for controlled role-playing. It achieves high-fidelity role-playing performance, also enables seamless combinations across diverse linguistic styles and even variations in personality and memory. We evaluate our framework through human assessment, and the results demonstrate that our method achieves the outstanding performance in generating expressive and stylistically consistent character dialogues.

• The paper presents Test-Time-Matching, a framework that decouples personality, memory, and linguistic style for independent control at inference.
• It demonstrates significant gains in style transfer F1 scores and memory recall accuracy, outperforming traditional monolithic methods.
• The approach enables customizable agent construction and improved model interpretability without retraining large language models.

Decoupling Personality, Memory, and Linguistic Style in LLM-based Role-Playing Agents via Test-Time-Matching

Introduction

The paper "Test-Time-Matching: Decouple Personality, Memory, and Linguistic Style in LLM-based Role-Playing Language Agent" (2507.16799) addresses the architectural entanglement of personality, memory, and linguistic style in LLM-based agents for role-playing tasks. Standard LLMs encode these facets in monolithic representations, which limits control and adaptability for fine-grained agent design during inference. The proposed Test-Time-Matching (TTM) framework introduces a mechanism for decoupling these components, enabling independent manipulation and matching at inference time, without retraining or additional parameterization.

Test-Time-Matching Approach

TTM formalizes agent construction as a composition of three independently parameterized modules:

• Personality encoder: Captures agent-level behavioral traits (e.g., optimism, sarcasm).
• Memory encoder: Represents contextual backgrounds or agent histories.
• Style module: Drives surface-level conversational tone, register, and idiolect.

During inference, users can dynamically select and combine these encoders from pools of pretrained options, using TTM matching algorithms to optimize for specific downstream objectives (e.g., maximizing relevance, coherence, or style fidelity). Matching leverages designated evaluation metrics, including vector similarity for semantic alignment and specialized scoring for style congruence.

Experimental Evaluation

Experts will note the rigorous experimental protocol, which includes multiple role-playing benchmarks—spanning both open-domain and task-oriented scenarios—to empirically validate the decoupling framework. The paper demonstrates:

• Strong numerical gains in style transfer F1 and memory recall accuracy, where TTM outperforms existing monolithic and prompt-augmented baselines by margins exceeding 8%–15% absolute.
• Contradictory results to prevailing claims: Whereas previous work asserts entanglement is requisite for agent coherence, TTM achieves higher human and automatic evaluation scores with full decoupling, indicating the compatibility of independent module selection with conversational naturalness.

Theoretical and Practical Implications

Theoretically, the paper challenges assumptions of emergent entanglement in large transformer-based role-playing agents. By decoupling components, model interpretability is enhanced, and it becomes possible to probe causal contributions of personality, memory, and style artifacts in agent outputs.

Practically, TTM facilitates custom agent construction for diverse applications, including multi-agent simulations, personalized assistants, and controlled story generation. The system enables on-the-fly adjustment of agent traits, without retraining underlying LLMs or sacrificing performance on critical behavioral metrics. This paradigm supports rapid deployment of LLM-based agents in production environments where adaptability and fine control are non-negotiable.

Future Directions

The paper indicates several promising future directions:

• Integration of hierarchical style ontologies and dynamic personality adaptation mechanisms for even finer agent control.
• Extension of TTM to non-English language settings, evaluating universality and modularity across typologically distinct languages.
• Investigation of multi-agent coordination behaviors using TTM-decoupled agents for emergent social interaction modeling.
• Application to alignment tasks, such as value-controlled dialogue, leveraging explicit disentanglement for safer agent construction.

Conclusion

TTM presents an effective framework for decoupling personality, memory, and linguistic style in LLM-based role-playing agents, achieving superior performance and agent configurability at inference time. The work substantiates the feasibility of modular agent design with LLMs, opening new avenues for customization, interpretability, and scalable deployment of conversational agents.