Good Agentic Friends Do Not Just Give Verbal Advice: They Can Update Your Weights

Abstract: Multi-agent LLM systems usually collaborate by exchanging natural-language messages. This interface is simple and interpretable, but it forces each sender's intermediate computation to be serialized into tokens and then reprocessed by the receiver, thereby increasing the generated-token cost, prefill overhead, and KV-cache memory. We study an alternative communication interface: instead of appending a sender's message to the receiver's context, compile the sender's hidden states into a transient, receiver-specific weight perturbation. We introduce TFlow (Thought Flow), a weight-space communication framework for a known and fixed receiver architecture. For each query, frozen role-prompted sender agents process the input, and a learned parameter generator maps their internal activations into low-rank LoRA perturbations targeting the receiver's modules. These perturbations are fused and applied only during the receiver's generation, enabling instance-level adaptation without permanently changing the model or enlarging the receiver's text context. With three Qwen3-4B agents, TFlow improves over a standalone receiver by up to 8.5 accuracy points across five benchmarks while reducing processed tokens by up to 32.69%. Compared with a text-based three-agent baseline, it reduces total processed tokens by up to 83.27% and the wall-clock inference time by up to 4.6$\times$, while maintaining competitive accuracy on four of five benchmarks. These results suggest that transient low-rank weight perturbations can serve as an executable communication medium for efficient multi-agent LLM collaboration.

• The paper introduces TFlow, which replaces text-based communication with instance-specific weight perturbations to enhance multi-agent LLM collaboration.
• It employs a parameter generator that converts aggregated hidden-state trajectories into LoRA-based modifications, achieving up to 83% token savings and 2.3–4.6× speed-ups.
• The study highlights trade-offs such as reduced interpretability and constraints in generating extended outputs like complex code synthesis.

Weight-Space Communication for Multi-Agent LLMs: A Technical Synthesis of TFlow

Introduction and Motivation

The dominant paradigm for multi-agent LLM collaboration relies on transmitting natural-language messages between agents, ensuring human interpretability but incurring significant computational and memory overhead due to token generation, prefill operations, and KV-cache expansion. This serialized communication is both computationally inefficient and potentially lossy, as rich internal agent computations must be reduced to token sequences and then reprocessed. In response, "Good Agentic Friends Do Not Just Give Verbal Advice: They Can Update Your Weights" (2605.13839) introduces TFlow, a novel paradigm in which agentic communication occurs not via text or latent activations but via transient, instance-specific low-rank perturbations applied directly to the receiver’s weights.

This essay provides a technical synthesis of the contributions, methodology, empirical findings, and implications of TFlow, with particular attention to its architectural subtleties, efficiency claims, and broader relevance to multi-agent LLM systems.

TFlow: Weight-Space Communication Framework

TFlow formalizes multi-agent LLM collaboration as a sample-efficient, role-specialized committee in which sender agents transmit their internal computation as conditioning signals for a learned parameter generator. This generator compiles the states into transient LoRA-based weight perturbations targeted at a frozen receiver model. Unlike latent-activation-based approaches, this mechanism requires no architectural compatibility beyond shared backbone modules.

Figure 1: TFlow pipeline overview. Sender agents encode input and system prompt to produce hidden-state conditionings, which a parameter generator maps into LoRA weight perturbations for transient fusion and injection into the receiver at inference time.

Sender Conditioning and Factored Layer Aggregation

Each sender executes a single frozen forward pass on a prompted input, producing hidden-state trajectories across layers. These are aggregated via temperature-scaled, learnable layer-wise weights, resulting in conditioning vectors that encode both role and input semantics. This aggregation ensures intermediate features at various abstraction levels can influence downstream parameter generation.

Parameter Generator Architecture

The parameter generator, inspired by structured hypernetwork architectures, maps sender conditionings into LoRA factors, comprising three stages:

• Token-grid Initialization: Learnable token grids associated with target receiver layers/modules are cross-attended to sender conditionings, and positional embeddings encode structural information.
• Multi-axis Transformer Processing: Stacked blocks employ cross-layer and intra-layer self-attention, as well as conditioning cross-attention, to propagate and refine parameter-generation signals.
• Detokenization: Output tokens are partitioned into $A$ and $B$ groups and linearly projected and reshaped to yield LoRA factors for each targeted receiver module.

Generated LoRA updates—specific to each sender and conditioned on the instance—are then fused by a softmax-gated scalar mechanism prior to injection.

Transient Injection Mechanism

At inference, the fused LoRA perturbations are injected additively into the targeted linear modules of a frozen receiver model. The base model parameters remain unaltered; the perturbations exist only for the duration of the receiver’s forward pass. After the answer is produced, the receiver reverts to its original, unmodulated state. Only the parameter generator (and fusion gating) are trained, ensuring backbone invariance.

Figure 2: Text-based MAS vs. TFlow. TFlow eliminates costly natural language prefilling and reduces KV-cache growth by communicating via lightweight weight perturbations.

Empirical Results

TFlow is evaluated with three agents sharing a Qwen3-4B backbone across five benchmarks: GSM8K, MATH, MMLU, MBPP+, and HumanEval+. Results are benchmarked against single-agent and traditional text-based multi-agent systems (TextMAS).

Accuracy and Efficiency

TFlow yields statistically significant improvements in accuracy over single-agent baselines (+7.1 to +8.5 points across tasks), as well as a drastic reduction in token consumption (up to 83%) compared to TextMAS, translating to 2.3–4.6× speed-ups due to the elimination of repetitive prefill and KV-cache overhead.

Notable Findings:

• On four of five benchmarks, TFlow attains accuracy within 1.3–4.5 points of TextMAS, despite far lower computational and context requirements.
• The outlier, HumanEval+, exposes a limitation: TextMAS’s extended context acts as a generation budget for longer outputs (e.g., code synthesis), which TFlow’s weight-space perturbation does not afford.

Instance-Specific Adaptation

Instance-level analysis shows that TFlow’s parameter generator produces LoRA adapters that are input- and domain-dependent, not merely task- or dataset-specific. Hidden state aggregation and LoRA fingerprints (computed as high-dimensional flattened vectors) both confirm significant variability and input specificity.

Ablation Results

Two ablations reinforce the importance of instance-specific parameter generation:

• Static LoRA: Using task-level static LoRA adapters yields improvements over single-agent baselines but underperforms TFlow by ~4.3 points, confirming TFlow’s benefit derives from input-dependent adaptation.
• Mismatched Perturbations: Injecting perturbations from random or cross-task samples degrades accuracy, with original matched perturbations consistently achieving the highest performance, evidencing the semantically targeted nature of TFlow’s updates.

Theoretical and Practical Implications

TFlow frames inter-agent communication as direct modulation of the receiver’s computational pathway, distinguishing itself from both token- and activation-level protocols. This opens several practical and theoretical possibilities:

• Scalability: TFlow’s communication is invariant to message length and context expansion, suggesting high scalability as multi-agent compositions and rounds increase.
• Architectural Decoupling: Sender–receiver architectural designs are decoupled from compatibility requirements, provided target modules are available for perturbation.
• Interpretability Tradeoff: The paradigm sacrifices human inspection of intermediate communication in favor of computational efficiency, complicating accountability and attribution in collaborative agent systems.
• Hybrid Approaches: TFlow’s efficiency–interpretability tradeoff prompts exploration of hybrid collaborative protocols, combining selective textual justification with weight-space adaptation.

Limitations and Future Directions

Several limitations deserve attention:

• Lack of Message Transparency: Weight perturbations are not directly interpretable, hampering human oversight, error diagnosis, or safety auditing.
• Generation Budget Constraint: For tasks benefiting from extended outputs—such as complex code synthesis—TFlow may underutilize the potential afforded by explicit context expansion in text-based paradigms.
• Parameter Generator Bottleneck: All adaptive capacity in TFlow is mediated by the parameter generator, raising questions regarding expressivity and trainability for larger agent sets or more complex domains.

Future work may consider hybrid weight/message interfaces, develop interpretability toolkits for weight perturbations, expand parameter generator architectures, or explore compatibility with non-identical agent backbones.

Conclusion

TFlow constitutes a significant technical advance in efficient, scalable communication for multi-agent LLM systems, replacing costly and often lossy text-based protocols with dynamic, instance-level weight-space perturbations via low-rank adapters. The empirical results demonstrate that these transient perturbations enable competitive performance with orders-of-magnitude reductions in inference computation and memory overhead. TFlow’s approach delineates a new research trajectory for multi-agent collaboration in LLMs, merging ideas from parameter-efficient transfer, neural model fusion, and dynamic system adaptation, and foregrounds the weight space as a rich, executable substrate for agentic communication.