---
title: 'SayCanPay: LLM & Heuristic Planning'
url: https://www.emergentmind.com/topics/saycanpay
type: topic
---

# SayCanPay: LLM & Heuristic Planning

SayCanPay is a planning framework that integrates the world knowledge of large language models (LLMs) with learnable domain knowledge and heuristic search to generate plans that are both feasible (affordance-grounded) and cost-effective in terms of plan length. Developed to address limitations of pure-LLM and classical heuristic planning, SayCanPay combines three complementary modules—Say (LLM plan proposals), Can (learned affordance evaluator), and Pay (learned reward/payoff estimator)—within a heuristic search procedure over action sequences. This architecture generalizes across domains, from language-based embodied tasks to end-to-end agentic payment workflows, and has demonstrated superior planning success, cost-effectiveness, and generalization relative to baseline LLM or heuristic-only methods [2308.12682, 2602.24068].

## 1. Formal Problem Formulation

SayCanPay frames planning as a finite-horizon partially observable Markov decision process (POMDP) defined as $\mathcal{P} = \langle \mathcal{S}, \mathcal{S}_G, b_0, \mathcal{A}, \mathcal{O}, R, \mathbb{T} \rangle$, where:
- $\mathcal{S}$ is the (hidden) state space; $\mathcal{S}_G \subseteq \mathcal{S}$ a goal set.
- $b_0$ is the initial belief (one-hot at $s_0$).
- $\mathcal{A}$ is the discrete action set; $\mathcal{O}$ is the observation set, with $o_t = O(s_t)$.
- $R\!: \mathcal{O} \rightarrow \mathbb{R}$ is a sparse reward (success/failure).
- $\mathbb{T}$ is the transition kernel.

The planning objective is, given a natural-language goal $g$ and initial history $h_0 = (o_0)$, to find an action sequence $a_{1:T}$ maximizing $P(a_{1:T} \mid h_0, g)$. The value function is factorized as:
\[
f(h_t) = f_{\mathrm{acc}}(h_{t-1}) + \log p(a_t \mid h_{t-1}, g) + \log f_{\mathrm{heur}}(h_t, g)
\]
where $f_{\mathrm{acc}}$ is the log-probability of prior actions, $p(a_t \mid h_{t-1}, g)$ is the model likelihood for the next action (“Say”), and $f_{\mathrm{heur}}$ is a learned estimate of expected future reward (“Pay”). Specializations correspond to greedy best-first or A*-like strategies, depending on the combination of terms and search procedure [2308.12682].

## 2. Say, Can, and Pay Modules

The SayCanPay framework decomposes heuristic planning into three distinct modules:

- **Say module:** A pretrained LLM (e.g., Vicuna-13B, Flan-T5-11B) prompts to generate candidate actions at each step. Candidates are selected by beam search or greedy decoding over language outputs, mapped via edit distance to admissible domain actions. The log-likelihood $p(a_t \mid h_{t-1}, g)$ forms the “Say” component.

- **Can module:** A BERT-based binary classifier $M_{\mathrm{can}}$ trained on expert demonstrations estimates the feasibility of candidate actions, outputting $p_\mathrm{can} = p(\text{pre}(a_t) \mid h_{t-1}, g)$, reflecting whether action preconditions are met. Negative samples are drawn by swapping time steps and/or sampling unrelated actions in trajectory space. The loss uses InfoNCE over batch.

- **Pay module:** A BERT-based regression model $M_{\mathrm{pay}}$ estimates the expected, discounted, arc-to-go reward $f_{\mathrm{heur}}(h_t, g)$ by back-propagating terminal rewards along expert trajectories and discounting by $\delta$. Negative samples (off-expert trajectories) receive zero reward. Mean squared error loss is used for training. Both $M_{\mathrm{can}}$ and $M_{\mathrm{pay}}$ share architecture and optimizer: uncased BERT, AdamW ($\text{lr}=10^{-4}$, weight decay $10^{-5}$, batch=50), trained for 20 epochs [2308.12682].

The combined score for an action sequence is given by:
\[
f(h_t) = f_{\mathrm{acc}}(h_{t-1}) + \log \left[ p(a_t \mid h_{t-1}, g) \cdot p_\mathrm{can} \cdot f_{\mathrm{heur}}(h_t, g) \right]
\]
i.e., SayCanPay = Say $\times$ Can $\times$ Pay.

## 3. Heuristic Search Framework

SayCanPay uses the output of the Say, Can, and Pay modules in classical heuristic-search algorithms:

- **Greedy-Action:** At step $t$, expand the current state by proposing actions via Say, scoring each with SayCanPay, and selecting the highest-scoring candidate.

- **Beam-Action:** Maintain $k$ beams, expanding each by $m$ candidates, scoring each descendant:
\[
\text{score} = \frac{f_{\mathrm{acc}}(h_{t-1}^i) + \log p(a_t^j) + \log p_\mathrm{can} + \log f_\mathrm{heur}}{|h_t^{i,j}|}
\]
Prune to the top-$k$ beams by length-normalized score and continue expansion until termination.

Each beam is a complete sequence of actions to the goal or until maximum allowed length. This results in grounded, reward-oriented plan sequences, outperforming LLM-only or heuristic-only strategies in experimental benchmarks [2308.12682].

## 4. Experimental Evaluation

SayCanPay has been evaluated on multiple domains:
- **Ravens tasks:** Tower of Hanoi (mean plan length 3.3, $|\mathcal{A}|\approx 7.5$), Put Blocks in Bowls (6.1, $|\mathcal{A}|\approx25$).
- **BabyAI:** Pickup tasks, mean length 6.7.
- **VirtualHome:** Household activities, length 5.9, $|\mathcal{A}|\approx150$.

Training data consisted of $\approx$800 expert trajectories per Ravens/VirtualHome task, $\approx$400 for BabyAI; 100 test episodes per split. SayCanPay leverages off-the-shelf LLMs for Say, without LM fine-tuning.

Key results (\textit{beam size k=3, per-task Flan-T5/Vicuna-13B, success rates}):
- Ravens (Hanoi): SayCanPay Beam 70\%/50\% vs. Say 54\%/38\%.
- Ravens (Blocks): 56\%/98\% vs. 52\%/98\%.
- BabyAI: 94\%/36\% vs. 72\%/1\%.
- VirtualHome: 53\%/50\% vs. 48\%/30\%.
- Cost-effectiveness (optimal length plans) highest for SayCanPay, e.g., Ravens (Hanoi): 34\%/26\% vs. 27\%/26\% for Say.
- Generalization to OOD tasks: SayCanPay achieves up to +89\% over greedy.

Planning time per episode (Vicuna-13B): Greedy-Token 9 s, Greedy-Action 17 s, Beam-Action 35 s; Can/Pay add negligible overhead. Say (LM proposal) remains the primary computational bottleneck.

Ablation studies show that increasing beam size monotonically improves plan success and optimality; “Perfect Say” oracle (always proposes the correct action) increases success by 16–84 percentage points, localizing bottlenecks to the action-generation stage. Scalability is favorable since action space size does not increase LLM inference time, as candidates are generated, not enumerated [2308.12682].

## 5. Extension to Agentic Payment Workflows

SayCanPay has been generalized to payment workflows via a hierarchical multi-agent system (HMASP), structured in four levels:
- **Level I (CPA):** Conversational Payment Agent serves as natural-language entry, intent parsing, routing, and final consolidation.
- **Level II (Supervisors):** CardsSupervisor and PaymentsSupervisor manage orthogonal state: card onboarding/retrieval and payment initiation/history, respectively.
- **Level III (Routing Agents):** Specialized for specific sub-tasks (e.g., CardRegistrationRouter, PaymentProcessingRouter), trigger deterministic function modules.
- **Level IV (Process Summary Agents):** Aggregate results, produce structured summaries, and propagate outputs upstream.

State management combines persisted role-based state ($\mathrm{PayAgentState}_t$, $\mathrm{CardsState}_t$, $\mathrm{PaymentState}_t$) with decoupled message buffers $\mathcal{M}^A_t$ per agent. The handoff protocol between agents is governed by a probability model over successors, computed via LLM prompt and softmax. Transition selection is modeled as:
\[
P(A_{i+1} \mid A_i, \mathrm{State}_t, \mathcal{M}^{A_i}_t)
\]
with agent transitions choosing the highest-probability next agent or REJECT.

Experimental evaluation on a 1,000-point payment workflow dataset shows mean task success rate 97.7\% (Qwen2.5:32b), handoff F1 99.2\%, and near-parity between Qwen2.5 and GPT-4.1 [2602.24068].

## 6. Practical Considerations and Limitations

- **Deterministic vs. LLM boundaries:** All sensitive data handling, validation (Luhn, CVV, expiry), cryptographic operations, and API gateway invocations are fully deterministic. LLMs serve exclusively for natural language intent, agent handoffs, and summary/error generation; no raw PAN or CVV flows through prompts.
- **Risk and compliance:** Anti-fraud checks (velocity, geo-velocity, device, etc.) are performed by deterministic modules, with an LLM-augmented agent for interpretability and escalation. PCI DSS constraints are adhered to via strict data isolation.
- **Robustness:** Faults or timeouts in Routing agents result in structured error propagation to Summary agents for graceful failure communication.

Failure modes are primarily attributed to the Say step (LM generation failures/omissions), and to Can/Pay models failing to extrapolate beyond trained domains. *A plausible implication is that online adaptation or symbolic heuristics could further enhance OOD robustness* [2308.12682].

## 7. Impact, Generalization, and Future Directions

SayCanPay demonstrates that the fusion of LLM world knowledge, learned feasibility, and long-horizon reward modeling under classical heuristic search yields grounded, cost-efficient plans and workflows, bridging gaps between data-driven and symbolic planning methods. Its architecture has enabled the first end-to-end multi-agent conversational payment system capable of operating across hierarchical functional boundaries, modularizing state, and maintaining agentic explainability [2602.24068].

Potential future research directions include adaptive domain model learning, symbolic-LLM hybrid heuristics, and extension to broader real-world transactional domains, with particular attention to safety, compliance, and further reducing reliance on LLM action proposal bottlenecks.

## References

- SayCanPay: Heuristic Planning with Large Language Models using Learnable Domain Knowledge [2308.12682]
- A Novel Hierarchical Multi-Agent System for Payments Using LLMs [2602.24068]

Source: https://www.emergentmind.com/topics/saycanpay