POMO+: Modular Enhancements in Policy Optimization

• POMO+ is a family of extensions to the original POMO method that incorporates diverse modifications in learning rules, reward shaping, and search procedures.
• It includes variants like baseline-free optimization, leader-focused credit assignment, and distributionally robust training aimed at improving performance and stability.
• POMO+ also leverages decomposition and metaheuristic augmentation to adapt the multi-start paradigm for complex routing scenarios and large-scale VRPs.

POMO+ denotes a family of extensions to Policy Optimization with Multiple Optima (POMO) in neural combinatorial optimization rather than a single standardized algorithm. In the recent literature, the label is used for methods that preserve POMO’s core attention-model, multi-start, and symmetry-exploiting construction paradigm while altering the learning rule, reward shaping, robustness objective, representation module, or surrounding search procedure. The reference point is the original POMO formulation, which trains an autoregressive attention policy with multiple rollouts per instance and a shared baseline over those rollouts, and reports a $0.14\%$ optimality gap on TSP100 while reducing inference time by more than an order of magnitude (Kwon et al., 2020). Subsequent work has used “POMO+” to denote baseline-free optimization (Sepúlveda et al., 9 Jun 2026), distributionally robust training with CNN-enhanced representations (Jiang et al., 2022), leader-focused credit assignment (Wang et al., 2024), PPO-based adaptation to time-constrained finite-fleet VRP (Deineko et al., 2024), evolutionary augmentation (Gu et al., 11 Jun 2025), large-scale ruin-recreate decomposition (Falkner et al., 2023), and search-graph controllers that treat POMO as a constructive subroutine or baseline (Yakovenko et al., 20 May 2026).

1. Foundational POMO mechanics

POMO originates as a reinforcement-learning scheme for neural combinatorial optimization in which a single instance is decoded from multiple symmetry-related starting points, yielding multiple trajectories that correspond to multiple optimal or near-optimal sequence representations of the same underlying solution structure (Kwon et al., 2020). In TSP, the canonical case, each starting node induces a cyclic shift of a tour; in CVRP and 0–1 knapsack, the symmetry is less exact but still sufficient to motivate multi-start training and inference.

The core gradient estimator uses the mean return over the $K$ rollouts on the same instance as a shared baseline: $$\nabla_\theta J(\theta)=\frac{1}{K}\sum_{k=1}^{K}\nabla_\theta \log \pi_\theta(\tau_k)\,(R_k-\bar R).$$ This shared baseline is central to POMO’s variance reduction. Relative to single-start REINFORCE with a greedy rollout baseline, it turns each instance into an intra-group comparison problem, so that trajectories are reinforced or penalized relative to other starts on the same graph rather than against an external baseline policy (Kwon et al., 2020).

Architecturally, POMO is typically instantiated with the attention model used in neural routing: a Transformer-like encoder and autoregressive pointer-style decoder. In the standard AM instantiation used in later comparisons, this corresponds to a 6-layer Transformer encoder with 8 heads and hidden dimension 128, with multi-start decoding and shared-baseline REINFORCE updates (Sepúlveda et al., 9 Jun 2026). Inference combines multi-start greedy decoding with $\times 8$ data augmentation based on symmetries of the coordinate space, which is one reason the original method remains a strong baseline in TSP and CVRP (Kwon et al., 2020).

2. Meanings of the “plus” suffix

Across the literature, the “plus” in POMO+ identifies different kinds of augmentation rather than a single canonical recipe. The common pattern is retention of POMO’s multi-start constructive policy while replacing or surrounding one of its training or search components.

Variant family	Main modification	Representative paper
Baseline-free POMO	Replace rollout/shared-baseline REINFORCE with GRPO or P3O	(Sepúlveda et al., 9 Jun 2026)
Distributionally robust POMO	Group DRO objective plus CNN latent pattern module	(Jiang et al., 2022)
Leader-focused POMO	Reweight advantages so the instance leader dominates credit assignment	(Wang et al., 2024)
Realistic constrained POMO	PPO training, ASAP urgency factor, finite fleet, multi-objective reward	(Deineko et al., 2024)
Evolution-augmented POMO	GA refinement of sampled tours with reinjection into REINFORCE	(Gu et al., 11 Jun 2025)
Large-scale POMO	Neural ruin–recreate with POMO only on subproblems near training size	(Falkner et al., 2023)
Search-controlled POMO	POMO treated as a constructive baseline inside PSG-based learn-to-search	(Yakovenko et al., 20 May 2026)

This diversity suggests that POMO+ is best understood as a design pattern: keep the POMO inductive bias toward multiple optima, but modify the optimization layer, robustness objective, representation, or search control to address specific failure modes.

3. Learning-rule and credit-assignment redesign

One line of work reinterprets POMO+ as “POMO with a different learning rule underneath.” The clearest example is baseline-free policy optimization for neural combinatorial optimization, which retains the AM/POMO architecture and multi-start idea but replaces baseline-dependent REINFORCE with Group Relative Policy Optimization (GRPO) or, secondarily, Pairwise Preference Policy Optimization (P3O) (Sepúlveda et al., 9 Jun 2026). GRPO computes normalized within-group advantages,

$$\hat A_{bg}=\frac{R(\tau_{bg})-\mu_G}{\max(\sigma_G,\epsilon)},$$

and optimizes a PPO-style clipped objective over sampled trajectories. In the reported TSP-100 comparison, REINFORCE with rollout baseline exhibits a structural failure: performance improves during warmup to cost $9.8$ and then collapses to $52.1$ when the rollout baseline activates, without recovery even under extended training. GRPO avoids this collapse and, at matched gradient updates, remains within about $1.5\%$–$1.6\%$ of POMO on TSP-100 and CVRP-100 while requiring no external baseline (Sepúlveda et al., 9 Jun 2026). The same study also finds that P3O is competitive on TSP but shows higher variability on CVRP.

A second reinterpretation changes not the baseline mechanism but the objective implicit in training. “Leader Reward for POMO-Based Neural Combinatorial Optimization” argues that standard POMO optimizes average rollout quality even though inference keeps the best of many rollouts, and therefore introduces leader-focused credit assignment (Wang et al., 2024). For each instance, the best trajectory among the $N$ sampled starts receives relative weight $K$0, while non-leaders are down-weighted: $K$1 The paper uses this in a main training phase and then an aggressive final phase with $K$2, where only the leader receives credit. On TSP100, the reported greedy POMO gap falls from $K$3 to $K$4, and with SGBS+EAS the gap reaches $K$5, described as more than a 100-fold reduction relative to original POMO with almost no additional computational overhead (Wang et al., 2024). The same paper also reports an important trade-off: the final aggressive phase improves best-of-many behavior while worsening average sampled solution quality and increasing variance.

A third modification keeps the POMO protocol but replaces REINFORCE with PPO in a more realistic constrained VRP environment. In the time-constrained, finite-fleet setting, the policy is trained with PPO, a critic, and GAE, while preserving parallel POMO trajectories per instance (Deineko et al., 2024). The decoder adds an ASAP urgency factor,

$K$6

directly into attention logits to encode time urgency, and the reward combines distance with a penalty on unused vehicle load when returning to depot. This is a distinct sense of POMO+: the plus lies in replacing the original deterministic CVRP setting and REINFORCE update with a richer MDP and PPO-based policy optimization.

4. Robustness to distribution shift and richer data regimes

Another interpretation of POMO+ is robustness-oriented rather than optimizer-oriented. “Learning to Solve Routing Problems via Distributionally Robust Optimization” turns POMO into DROP, a distributionally robust POMO that trains on multiple instance distributions and optimizes worst-group performance via group DRO (Jiang et al., 2022). If $K$7 are group distributions and $K$8 are group weights, the training problem is

$K$9

The paper applies this to two-group and six-group mixtures built from uniform, explosion, implosion, expansion, cluster, and grid distributions, and augments POMO with a CNN-based local latent pattern module using $$\nabla_\theta J(\theta)=\frac{1}{K}\sum_{k=1}^{K}\nabla_\theta \log \pi_\theta(\tau_k)\,(R_k-\bar R).$$0 nearest neighbors, kernel size 11, and 128 kernels. In TSP100 atypical-group results, DROP reduces the explosion gap from $$\nabla_\theta J(\theta)=\frac{1}{K}\sum_{k=1}^{K}\nabla_\theta \log \pi_\theta(\tau_k)\,(R_k-\bar R).$$1 for POMO to $$\nabla_\theta J(\theta)=\frac{1}{K}\sum_{k=1}^{K}\nabla_\theta \log \pi_\theta(\tau_k)\,(R_k-\bar R).$$2, the implosion gap from $$\nabla_\theta J(\theta)=\frac{1}{K}\sum_{k=1}^{K}\nabla_\theta \log \pi_\theta(\tau_k)\,(R_k-\bar R).$$3 to $$\nabla_\theta J(\theta)=\frac{1}{K}\sum_{k=1}^{K}\nabla_\theta \log \pi_\theta(\tau_k)\,(R_k-\bar R).$$4, and the grid gap from $$\nabla_\theta J(\theta)=\frac{1}{K}\sum_{k=1}^{K}\nabla_\theta \log \pi_\theta(\tau_k)\,(R_k-\bar R).$$5 to $$\nabla_\theta J(\theta)=\frac{1}{K}\sum_{k=1}^{K}\nabla_\theta \log \pi_\theta(\tau_k)\,(R_k-\bar R).$$6. On TSPLib and CVRPLib, the reported average gap over all instances is $$\nabla_\theta J(\theta)=\frac{1}{K}\sum_{k=1}^{K}\nabla_\theta \log \pi_\theta(\tau_k)\,(R_k-\bar R).$$7 for DROP versus $$\nabla_\theta J(\theta)=\frac{1}{K}\sum_{k=1}^{K}\nabla_\theta \log \pi_\theta(\tau_k)\,(R_k-\bar R).$$8 for POMO (Jiang et al., 2022). Here, POMO+ means worst-group robustness plus a representation module tailored to local spatial patterns.

The need for such robustness is reinforced by SVRPBench, a benchmark for stochastic vehicle routing with time-dependent congestion, log-normal delays, probabilistic accidents, and heterogeneous residential versus commercial time windows (Heakl et al., 28 May 2025). Its abstract reports that state-of-the-art RL solvers like POMO and AM degrade by over $$\nabla_\theta J(\theta)=\frac{1}{K}\sum_{k=1}^{K}\nabla_\theta \log \pi_\theta(\tau_k)\,(R_k-\bar R).$$9 under distributional shift. In the aggregate benchmark table, POMO attains total cost $\times 8$0, constraint violation rate $\times 8$1, feasibility rate $\times 8$2, runtime $\times 8$3 s, and robustness $\times 8$4, compared with OR-Tools at $\times 8$5, $\times 8$6, $\times 8$7, $\times 8$8 s, and $\times 8$9 respectively (Heakl et al., 28 May 2025). The benchmark’s analysis suggests that POMO’s multi-start baseline does not by itself provide robustness to stronger stochasticity, larger scales, or structural shifts.

The finite-fleet, time-constrained PPO-based variant can be read as a complementary response to realism rather than distributional robustness per se. It embeds hard end times, finite fleet size, and a scalarized multi-objective reward into the POMO protocol, and reports shorter total distance than PyVRP on eight reported test instances ranging from $$\hat A_{bg}=\frac{R(\tau_{bg})-\mu_G}{\max(\sigma_G,\epsilon)},$$0 to $$\hat A_{bg}=\frac{R(\tau_{bg})-\mu_G}{\max(\sigma_G,\epsilon)},$$1 customers under the paper’s evaluation setting (Deineko et al., 2024). In this usage, POMO+ denotes adaptation to operational constraints that are absent from the original POMO benchmark suite.

5. Search augmentation, decomposition, and outer-loop control

A different branch of the literature leaves the core POMO policy largely intact and instead augments the search process around it. “Synergizing Reinforcement Learning and Genetic Algorithms for Neural Combinatorial Optimization” introduces the Evolutionary Augmentation Mechanism (EAM), which samples tours from POMO, refines them with elitist selection, order crossover, and 2OPT-style mutation, and reinjects the evolved tours into the same REINFORCE training loop (Gu et al., 11 Jun 2025). The method is explicitly model-agnostic and plug-and-play. On TSP100 with $$\hat A_{bg}=\frac{R(\tau_{bg})-\mu_G}{\max(\sigma_G,\epsilon)},$$2 augmentation, the reported cost improves from $$\hat A_{bg}=\frac{R(\tau_{bg})-\mu_G}{\max(\sigma_G,\epsilon)},$$3 for POMO to $$\hat A_{bg}=\frac{R(\tau_{bg})-\mu_G}{\max(\sigma_G,\epsilon)},$$4 for EAM-POMO; on CVRP100 with $$\hat A_{bg}=\frac{R(\tau_{bg})-\mu_G}{\max(\sigma_G,\epsilon)},$$5 augmentation, the reported cost improves from $$\hat A_{bg}=\frac{R(\tau_{bg})-\mu_G}{\max(\sigma_G,\epsilon)},$$6 to $$\hat A_{bg}=\frac{R(\tau_{bg})-\mu_G}{\max(\sigma_G,\epsilon)},$$7 (Gu et al., 11 Jun 2025). The plus here is a training-only evolutionary layer that improves exploration without changing inference-time behavior.

“Too Big, so Fail?” addresses a different failure mode: length generalization (Falkner et al., 2023). The paper shows that POMO trained on CVRP100 is overtaken by simple heuristics when applied directly to much larger instances. Its proposed Neural Ruin–Recreate (NRR) method never applies POMO to the full $$\hat A_{bg}=\frac{R(\tau_{bg})-\mu_G}{\max(\sigma_G,\epsilon)},$$8–$$\hat A_{bg}=\frac{R(\tau_{bg})-\mu_G}{\max(\sigma_G,\epsilon)},$$9 problem; instead, it constructs an initial solution with Clarke–Wright savings, decomposes the global solution into subgraphs of size near the POMO training size, scores those subgraphs with a GNN, ruins them, and recreates them with pre-trained POMO under a simulated-annealing acceptance rule. On the synthetic mixed dataset, the reported cost at $9.8$0 is $9.8$1 for greedy POMO, $9.8$2 for SGBS, and $9.8$3 for NRR (Falkner et al., 2023). In this formulation, POMO+ is not a new policy gradient at all; it is a metaheuristic that constrains POMO to the regime where it is known to work.

COAgents goes further by recasting search as traversal of a partial search graph (PSG) whose nodes are complete solutions and whose edges are local moves or learned jumps (Yakovenko et al., 20 May 2026). POMO appears there as a strong learn-to-construct baseline rather than the object being modified, but the paper is directly relevant to the POMO+ idea because it demonstrates a systematic outer-loop controller for constructive neural solvers. COAgents trains a Node Selection Agent, a Move Selection Agent, and a Jump Agent over PSG embeddings. On VRPTW, it reduces the gap to best-known solutions by $9.8$4 at $9.8$5 and $9.8$6 at $9.8$7 relative to POMO (Yakovenko et al., 20 May 2026). This suggests a broader interpretation of POMO+: retain POMO as an initializer or subroutine, but let a separate learn-to-search layer control refinement and diversification.

6. Conceptual synthesis, trade-offs, and open directions

Taken together, the literature suggests that POMO+ is best understood as a modular research program organized around several recurring axes. One axis concerns variance reduction and stability: baseline-free GRPO removes frozen rollout baselines and avoids the TSP-100 collapse observed with REINFORCE, but at matched updates remains slightly behind POMO on large TSP/CVRP instances (Sepúlveda et al., 9 Jun 2026). Another concerns the target objective: leader reward improves best-of-many inference performance dramatically, but its aggressive final phase intentionally sacrifices average sampled quality and increases variance (Wang et al., 2024). A third concerns distributional robustness: group DRO and CNN-enhanced local embeddings improve cross-distribution generalization on atypical routing patterns, while SVRPBench shows that plain POMO remains vulnerable to stochastic distribution shift (Jiang et al., 2022, Heakl et al., 28 May 2025). A fourth concerns scale: direct POMO deteriorates badly on large CVRP, whereas ruin–recreate decomposition restores competitiveness by keeping reconstruction near the training scale (Falkner et al., 2023). A fifth concerns search control: evolutionary augmentation and PSG-based controllers show that constructive neural policies can benefit from outer-loop mechanisms that act on complete solutions rather than only on autoregressive action sequences (Gu et al., 11 Jun 2025, Yakovenko et al., 20 May 2026).

A common misconception is that the original multi-start mechanism alone solves robustness, scalability, and realism. The collected evidence does not support that interpretation. Baseline-dependent REINFORCE can become structurally fragile on harder instances (Sepúlveda et al., 9 Jun 2026); cross-distribution generalization degrades when training remains tied to uniform node distributions (Jiang et al., 2022); realistic stochastic travel times and time windows expose nontrivial feasibility and robustness gaps (Heakl et al., 28 May 2025); and length generalization to $9.8$8–$9.8$9 can fail badly without decomposition (Falkner et al., 2023). The empirical strength of POMO on standard synthetic benchmarks therefore does not imply that a single, unmodified formulation suffices across routing regimes.

The open directions identified by these papers are correspondingly heterogeneous. The baseline-free optimization study points to larger sizes such as TSP-200+, curriculum learning, better exploration schedules, and more robust pairwise-preference methods (Sepúlveda et al., 9 Jun 2026). Leader Reward suggests dynamic $52.1$0 scheduling and broader application to other RL frameworks (Wang et al., 2024). Distributionally robust POMO leaves room for richer grouping strategies and broader routing distributions beyond the synthetic group mixtures studied (Jiang et al., 2022). The finite-fleet, time-constrained formulation points toward heterogeneous fleets, multi-depot settings, and dynamic or online VRPs (Deineko et al., 2024). EAM emphasizes task-specific operator design and annealing of GA intensity (Gu et al., 11 Jun 2025). NRR and COAgents suggest that the next stage of POMO+ may lie less in decoder redesign than in learned decomposition, learned jumps, and history-aware search control (Falkner et al., 2023, Yakovenko et al., 20 May 2026).

In this sense, POMO+ names a transition from a single end-to-end constructive policy into a broader modular ecosystem: the original POMO multi-start attention model remains the anchor, but the surrounding system may now include baseline-free policy optimization, leader-biased credit assignment, group-robust objectives, PPO-based realistic environments, evolutionary refinement, ruin–recreate decomposition, or graph-based search orchestration.