Evolutionary Graph Optimization (EGO-Prompt)

• EGO-Prompt is a methodology that combines semantic causal graphs with evolutionary optimization to systematically refine prompts for large language models.
• It employs a two-stage process using a causal-guided textual gradient mechanism to iteratively update both the prompt and the underlying graph structure.
• Empirical results demonstrate that EGO-Prompt enhances performance and interpretability, enabling small models to match larger LLMs while reducing computational cost.

Evolutionary Graph Optimization for Prompting (EGO-Prompt) is a methodology that integrates graph-based domain knowledge modeling with evolutionary optimization to systematically refine prompts and reasoning processes for LLMs in complex, domain-specific tasks. EGO-Prompt formalizes domain knowledge as a Semantic Causal Graph (SCG), leverages a causal-guided textual gradient mechanism to iteratively improve both structured knowledge and prompt components, and optimizes for empirical performance and interpretability. The approach achieves superior performance over existing prompting strategies while enabling small LLMs to match or exceed the performance of much larger models on real-world tasks, at a reduced computational cost (Zhao et al., 24 Oct 2025).

1. Semantic Causal Graph Representation

EGO-Prompt encodes expert domain knowledge as a Semantic Causal Graph (SCG), denoted

$\mathcal{G} = (\mathcal{N},\,\mathcal{R}),$

where:

• $\mathcal{N} = \{n_1, ..., n_{|\mathcal{N}|}\}$ comprises semantic nodes derived from the input prompt (e.g., "Vaccination Coverage," "Population Immunity").
• $$\mathcal{R} \subseteq \mathcal{N} \times \mathcal{N}$$ contains directed edges $r_{ij} = (n_i \to n_j)$, each annotated with a natural language causal statement.

The SCG is constrained as a Directed Acyclic Graph (DAG), ensuring no cycles. Initial SCGs are provided by human experts and may be partial or inaccurate; EGO-Prompt incorporates mechanisms to correct and refine both the graph structure and the attached semantics during optimization. Each edge provides explicit interpretable priors for LLM reasoning and can be augmented, pruned, or re-annotated via subsequent evolutionary steps (Zhao et al., 24 Oct 2025).

2. Causal-Guided Textual Gradient Mechanism

EGO-Prompt decomposes each LLM inference into two stages:

• Instance-Specific Guidance: Extract deterministic causal hints $$z^*(x, \mathcal{G}) = \mathcal{M}_F(x; \mathcal{P}_{\mathrm{cau}}, \mathcal{G})$$, where $x$ is the instance, $\mathcal{M}_F$ the forward model, and $\mathcal{P}_{\mathrm{cau}}$ the causal prompt.
• Final Prediction: The LLM computes $$\hat y = \mathcal{M}_{F'}(x, z^*; \mathcal{P}_{\mathrm{sys}})$$, using both the input and instance-specific guidance.

Optimization employs a textual loss $\mathcal{L}(\hat y, y)$ (e.g., "match"/"no match") and calculates textual gradients using a backward LLM $\mathcal{N} = \{n_1, ..., n_{|\mathcal{N}|}\}$0:

• Gradients are approximated with natural language feedback for both the system prompt and the SCG.
• Each gradient triggers atomic edits: add, delete, or modify nodes/edges (in $\mathcal{N} = \{n_1, ..., n_{|\mathcal{N}|}\}$1) or text in prompts ($\mathcal{N} = \{n_1, ..., n_{|\mathcal{N}|}\}$2, $\mathcal{N} = \{n_1, ..., n_{|\mathcal{N}|}\}$3).

This causal-guided mechanism allows simultaneous refinement of structured knowledge and prompt formulation. Feedback includes explicit suggestions ("Add causal link $\mathcal{N} = \{n_1, ..., n_{|\mathcal{N}|}\}$4 because...") which is directly actioned on the SCG or prompts (Zhao et al., 24 Oct 2025).

3. Iterative Evolutionary Optimization Loop

The core evolutionary loop of EGO-Prompt iteratively updates the system prompt, causal prompt, and SCG:

1. Initialization: Start from expert-supplied $\mathcal{N} = \{n_1, ..., n_{|\mathcal{N}|}\}$5.
2. Sampling: For each iteration $\mathcal{N} = \{n_1, ..., n_{|\mathcal{N}|}\}$6, sample instance $\mathcal{N} = \{n_1, ..., n_{|\mathcal{N}|}\}$7.
3. Guidance Extraction: Derive $\mathcal{N} = \{n_1, ..., n_{|\mathcal{N}|}\}$8 and $\mathcal{N} = \{n_1, ..., n_{|\mathcal{N}|}\}$9.
4. Loss and Gradients: Compute text-based loss and textual gradients $$\mathcal{R} \subseteq \mathcal{N} \times \mathcal{N}$$0.
5. Stage 1—Prompt Refinement: Apply edits to $$\mathcal{R} \subseteq \mathcal{N} \times \mathcal{N}$$1, accept if F1 improves.
6. Stage 2—SCG and Causal Prompt Refinement: Apply edits to $$\mathcal{R} \subseteq \mathcal{N} \times \mathcal{N}$$2 and $$\mathcal{R} \subseteq \mathcal{N} \times \mathcal{N}$$3, accept if F1 improves.
7. Termination: Output the best $$\mathcal{R} \subseteq \mathcal{N} \times \mathcal{N}$$4 found.

Key features include two-stage updates (prompt before graph), acceptance based on empirical F1 improvement, and the use of mutation operators (add/delete/edit node or edge in $$\mathcal{R} \subseteq \mathcal{N} \times \mathcal{N}$$5) (Zhao et al., 24 Oct 2025).

4. Theoretical Properties

Theoretical justification for EGO-Prompt centers on two assumptions:

• (A1) Given $$\mathcal{R} \subseteq \mathcal{N} \times \mathcal{N}$$6, the rest of $$\mathcal{R} \subseteq \mathcal{N} \times \mathcal{N}$$7 does not affect $$\mathcal{R} \subseteq \mathcal{N} \times \mathcal{N}$$8: $$\mathcal{R} \subseteq \mathcal{N} \times \mathcal{N}$$9.
• (A2) The forward extraction model is deterministic: $r_{ij} = (n_i \to n_j)$0.

Consequently,

$r_{ij} = (n_i \to n_j)$1

so optimizing the prompt to effectively communicate $r_{ij} = (n_i \to n_j)$2 is sufficient to encode the available causal structure, even as the SCG evolves or is corrected. This result justifies EGO-Prompt’s focus on instance-wise reasoning guidance and joint textual-graph optimization (Zhao et al., 24 Oct 2025).

5. Empirical Results

Evaluation encompasses tasks in pandemic forecasting, traffic crash prediction, and travel mode choice with multiple LLMs. EGO-Prompt demonstrates:

• Average F1 increases of 7.3–12.6 pp over the strongest baseline (Zero-Shot‐CoT, Auto-CoT, ProTeGi, PHP, TextGrad).
• Relative improvement vs. organized prompt up to +24.9% (GPT-4o-mini) and +24.6% (Gemini 2.5 Flash).
• Small-model matching: GPT-4o-mini with EGO-Prompt (F1 ≈ 0.410) matches o4-mini (F1 ≈ 0.404) at approximately 1/6 the inference cost.
• Cost per 100 samples: o4-mini $r_{ij} = (n_i \to n_j)$3 EGO-Prompt (GPT-4o-mini) $r_{ij} = (n_i \to n_j)$4; o1 $r_{ij} = (n_i \to n_j)$5 EGO-Prompt (GPT-4o-mini) $r_{ij} = (n_i \to n_j)$6.

Ablation studies confirm the critical role of both instance-specific guidance and two-stage optimization, with joint graph+prompt tuning outperforming prompt-only or graph-only variants (Zhao et al., 24 Oct 2025).

6. Interpretability and Cost-Effectiveness

EGO-Prompt not only delivers accuracy gains but also outputs an empirically-refined SCG. The resulting SCG is more compact and aligned with data, providing interpretable, causal explanations for LLM outputs. SCG refinement involves adding salient causal links and deleting weak or spurious connections, improving transparency for domain experts. The framework enables low-cost deployment of compact LLMs, with smaller models achieving parity with or surpassing the capabilities of much larger systems—an essential advantage in domains where inference latency and budget are critical constraints (Zhao et al., 24 Oct 2025).