Graph-based Target Back-Propagation for Context Adaptation in Multi-LLM Agentic Systems

Abstract: Context adaptation automates prompt engineering in LLM-based systems by iteratively revising tunable prompts from task feedback, without modifying model weights. Extending this paradigm to multi-LLM agentic systems is crucial: existing methods suffer from inaccurate credit assignment and lack convergence guarantees. We propose \textbf{G}raph-based \textbf{T}arget \textbf{B}ack-\textbf{P}ropagation (GTBP), a context adaptation framework for agentic workflows modeled as directed acyclic graphs. GTBP propagates local target outputs backward through the workflow graph and uses target--output discrepancies to guide a stage-wise prompt update mechanism. Theoretically, we show that GTBP's stage-wise prompt updates become stable over iterations, and that a sufficiently capable LLM optimizer can decrease the overall objective. Empirically, GTBP consistently outperforms strong baselines across three benchmarks while maintaining comparable computational cost.

• The paper presents GTBP, a method that generalizes difference target propagation to achieve structured, module-wise credit assignment and prompt optimization in LLM workflows.
• It provides theoretical guarantees on optimization dynamics, including stability and convergence, backed by empirical improvements on benchmarks like HotpotQA and LiveBench-Math.
• Experimental results demonstrate performance gains (up to 4% accuracy improvements) and efficiencies in reducing LLM call budgets while maintaining modular prompt interpretability.

Graph-based Target Back-Propagation for Context Adaptation in Multi-LLM Agentic Systems

Motivation and Context

The proliferation of multi-agent LLM systems executing complex, multi-step workflows has led to acute challenges in credit assignment and prompt engineering. Context adaptation—iteratively refining module prompts based on task-level feedback while keeping model weights static—offers a scalable alternative to manual engineering or full-model finetuning. However, current context adaptation methods suffer from two core deficiencies: imprecise module-level credit assignment (attribution ambiguity) and an absence of theoretical convergence guarantees, particularly in agentic workflows with multiple interacting modules. The examined work introduces Graph-based Target Back-Propagation (GTBP), which brings an explicit, structured, and theoretically-anchored approach to both credit assignment and prompt optimization in directed graph workflows over LLM modules.

Methodology: Graph-based Target Back-Propagation (GTBP)

GTBP generalizes the principle of Difference Target Propagation (DTP) from neural networks to agentic LLM workflows. The workflow is modeled as a directed acyclic graph (DAG), where each node corresponds to a prompt-tunable LLM module and edges denote inter-module information flow. GTBP operates in two principal phases—LLM-guided target propagation and stage-wise prompt updating:

1. LLM-Guided Target Propagation: The system runs a standard forward pass to compute outputs at each node. Then, given a reference output at the final node, local target outputs are inferred for all upstream modules by recursively solving sub-problems that approximate each module's optimal output conditioned on the desired downstream target, using LLM-based backward operators in lieu of gradients.

   Figure 1: GTBP decomposes end-to-end workflow error into module-local targets by propagating backward through the computation DAG, producing explicit module-wise credit signals in text space.

2. Stage-wise Prompt Updating: Each module's current output is compared with its propagated target, and the discrepancy forms the optimization signal. Module prompts are updated by LLMs using edit budgets and claim-addition schedules, enforcing structure and stability in prompt length over iterations. The process is designed to mirror—on textual data—the local descent property of first-order optimization in parameter space.

Theoretical Guarantees

A central contribution is the formal analysis of GTBP's optimization dynamics under a single-token, one-hidden-layer workflow abstraction. By leveraging implicit-weight-equivalence theorems and regularity conditions for LLM-based module outputs, the authors derive the following key results:

• Stability: The magnitude of implicit parameter shifts induced by prompt edits decays as $\mathcal{O}(t^{-1})$ over training iterations (Theorem 1), under reasonable assumptions of prompt-length growth and contextual representation regularity.
• Convergence: The objective change per iteration is tightly upper-bounded by the shrinking prompt-induced update magnitude, ensuring stability and eventual stagnation of the objective gap.
• Descent Property: Sufficiently capable LLM optimizers guarantee that prompt updates, driven by accurate local targets, induce actual improvement in the system-level objective (Theorem 2). The decrease in the global loss converges to zero at rate $\mathcal{O}(t^{-1})$.

These results sharply contrast with prior approaches (e.g., GEPA, ACE, TextGrad), which lack structured descent guarantees or rely on reflection heuristics without explicit optimization linkage.

Experimental Evaluation

GTBP is validated on three complex multi-module benchmarks: SubPOP (opinion distribution prediction), HotpotQA (multi-hop QA), and LiveBench-Math (symbolic mathematical reasoning). The main findings are:

• Performance: GTBP achieves best-in-class performance relative to prompting, few-shot, and strong context-adaptation baselines. For HotpotQA and LiveBench-Math, absolute improvements over GEPA are non-trivial (up to 4% in F1/accuracy), and in SubPOP, Wasserstein Distance is reduced by nearly 8%.
• Efficiency: Training efficiency (LLM call budget) and per-sample inference token usage are competitive with baselines. Notably, on tasks with heavier per-call overhead, GTBP's module-local propagation substantially reduces total LLM calls needed for optimization.
• Readability and Modularity: Resultant module prompts are human-interpretable and structured, supporting auditing and modular debugging.

  Figure 3: GTBP training trajectory on SubPOP exhibits rapid objective decline and prompt-length growth followed by convergence and a monotonic reduction in prompt update volume, matching theoretical predictions.

Implications and Future Directions

This work substantiates that explicit, target-based, module-wise credit assignment—via graph-based target back-propagation—yields both practical gains and a path for theoretical understanding of prompt adaptation in black-box, multi-LLM systems. Key implications and prospects include:

• Agentic System Scalability: GTBP is compatible with black-box/proprietary LLMs and natively supports expansion to deeper and more dynamic workflow graphs, not just two-layer constructs.
• Rigorous Credit Routing: The explicit credit assignment framework prevents credit diffusion and misattribution, which are common in reflective or trajectory-level update methods.
• Limits and Extensions: The theoretical analysis relies on strong assumptions about the LLM optimizer and prompt-update regularity. Open questions remain regarding scaling to heterogeneous modules (e.g., memory nodes, tool-use components), deep graphs with branching/merging, and dynamic agentic workflows with state or memory.
• Efficiency/Usability Tradeoffs: While GTBP accumulates prompt claims, potentially incurring token bloat, its structured, modular nature invites extensions for adaptive prompt pruning, claim retrieval, or compositionality constraints to reduce inference overhead.

In practice, GTBP strengthens the toolkit for automating prompt engineering in agentic architectures where direct gradientization is unfeasible. It offers a foundation for future research into optimization-theoretic underpinnings of prompt-based adaptation and more complex forms of agentic reasoning—beyond the frozen-weight setting—potentially including memory-augmented, tool-augmented, or hierarchical multi-agent systems.

Conclusion

GTBP concretely advances prompt optimization within multi-LLM agentic systems by importing structured, explicit target propagation into the domain of context adaptation. The theoretical convergence and empirical superiority relative to reflection-based and playbook-style baselines underscore the value of localizing credit assignment in the prompt space. The framework establishes a foundation for scalable, theoretically-sound adaptation pipelines in next-generation, modular LLM workflows, with broad practical applicability and clear future extension pathways.