Rethinking Positional Encoding for Neural Vehicle Routing

Abstract: Transformer-based models have become the dominant paradigm for neural combinatorial optimization (NCO) of vehicle routing problems (VRPs), yet the role of positional encoding (PE) in these architectures remains largely unexplored. Unlike natural language, where tokens are uniformly spaced on a line, routing solutions exhibit several properties that render standard NLP positional encodings inadequate. In this work, we formalize three such structural properties that a routing-aware PE should respect, namely anisometric node distances, cyclic and direction-aware topology, and hierarchical depot-anchored global multi-route structure, combining them with a unifying design principle of geometric grounding. Guided by these criteria, we analyze and compare PE methods spanning NLP, graph-transformer, and routing-specific families, and propose a hierarchical anisometric PE that combines a distance-indexed, circularly consistent in-route encoding with a depot-anchored angular cross-route encoding. Extensive experiments across diverse VRP variants demonstrate that geometry-grounded PE consistently outperforms index-based alternatives, with gains that transfer across problem variants, model architectures, and distribution shifts.

• The paper introduces a hierarchical anisometric positional encoding that aligns embedding distances with true physical path lengths.
• It develops in-route and cross-route encodings to capture cyclicity, distance-awareness, and hierarchical structure in VRP solutions.
• Empirical evaluations demonstrate reduced optimality gaps across benchmarks, outperforming traditional NLP and graph-based positional encoding methods.

Rethinking Positional Encoding for Neural Vehicle Routing

Motivation and Structural Analysis of Routing-Specific Positional Encoding

The utilization of Transformer-based models for Neural Combinatorial Optimization (NCO) in Vehicle Routing Problems (VRPs) is now established, yet the adaptation of positional encodings (PE) – a critical architectural feature – remains underanalyzed for the unique structural properties inherent to routing tasks. Unlike natural language sequences, VRP solutions consist of unordered node sets embedded in Euclidean space, organized into cyclic, anisometric, and hierarchically-structured multi-route tours. This work formalizes three desiderata for routing-aware PEs: Distance-awareness (D1), Circularity (D2), and Hierarchical structure (D3). Analysis shows that standard NLP and graph-based PEs do not fully satisfy these requirements, limiting the representational fidelity of Transformer models in routing contexts.

Figure 1: Properties of positional encoding for routes.

The paper identifies that routing tasks differ from language modeling along three axes:

1. Anisometry: Inter-node physical separations vary, so discrete index-based PEs ignore route metric information.
2. Circularity: Tours are closed; hence, PE must identify start and end nodes, account for reversal symmetry where relevant, and encode cyclicity.
3. Hierarchy: VRP solutions consist of multiple, indistinguishable routes distinguished only by depot-centric geometry, necessitating route-level and intra-route-level disambiguation.

These properties imply that the mapping of positions to embeddings should be grounded in geometry rather than abstract indices.

Taxonomy and Limitations of Existing Positional Encoding Methods

The survey of existing PEs spans three families: NLP-originated (e.g., Absolute, Sinusoidal, RoPE, Relative, ALiBi), graph transformer-based (e.g., Laplacian, Random Walk, Shortest-Path Distance), and prior routing-specific designs (e.g., DACT's cyclic encoding, CycleFormer PE). None satisfy all three routing desiderata. NLP encodings inherently assume uniform, linear order, and lack geometric grounding. Graph-based encodings capture global structure but are not tailored for route-specific hierarchy or cyclicity. Routing-specific encodings achieve cyclicity but fail to encode physical distance or distinguish between routes.

A critical result is the demonstration via dataset probing that no prior encoding achieves high fidelity to all three structural desiderata, indicating a clear need for an encoding tailored to the geometric realities of VRPs.

Hierarchical Anisometric Positional Encoding: Construction and Theoretical Properties

To address these shortcomings, the authors propose a Hierarchical Anisometric Positional Encoding made of two components:

• In-Route PE (IPE): Encodes cumulative, normalized physical path distance using multi-frequency (sinusoidal) encoding. A direction-invariant cosine-only variant is employed where route reversal symmetry holds (e.g., CVRP, TSP), and a direction-aware sine–cosine version for problems with directionality (e.g., VRPTW, PDTSP).
• Cross-Route PE (XPE): Encodes each node’s polar angle with respect to the depot, capturing its global, depot-anchored geometric location. This component uses per-node angular features rather than aggregated per-route summaries, retaining maximal disambiguation between similar in-route positions across different routes.

Both PEs are concatenated per node, fused with node coordinates via a feed-forward layer, and supplied to the Transformer encoder. No architectural or training schedule changes are needed; only the PE module is swapped.

Theoretical analysis confirms that:

• IPE aligns embedding-space distances with true path-lengths (D1) and encodes required cyclicity and directionality (D2).
• XPE ensures nodes at identical positions in different routes are distinguishable (D3), especially in solutions where routes are segregated into angular sectors.

Empirical Evaluation: Encoding Probing, Ablation, and Large-Scale VRP Benchmarks

The evaluation is organized in three layers:

Probing and Component Ablation:

Probing on CVRP-100 shows that IPE+XPE is the only encoding to achieve high Spearman correlation between embedding distances and the three structural targets (distance gap, cyclic arc, and inter-route/intra-route discrimination). Ablation demonstrates that both IPE and XPE are necessary to achieve minimal solution optimality gaps.

Controlled Benchmarking:

Across multiple VRP types and models (including DACT, NDS, and N2S backbones), hierarchical anisometric PE consistently yields lower optimality gaps versus state-of-the-art neural and operations research solvers. For large-scale CVRP, VRPTW, and PCVRP, the new PE drives the neural model to outperform traditional heuristics and previous best neural methods on most problem sizes.

Numerical highlights:

• On CVRP-100, IPE+XPE achieves a $1.16\%$ average optimality gap, beating index-based baselines by more than $3$ percentage points.
• For CVRP-2000, NDS with IPE+XPE attains a $-2.45\%$ gap compared to HGS, indicating solutions better than the strong OR baseline.
• Out-of-distribution and cross-architecture transferability is documented: plugging IPE+XPE into N2S for PDTSP significantly narrows the gap to LKH as $N$ increases.

Robustness Across Variants and Distribution Shifts:

The methods generalize to different VRP constraints, scales, and data distributions (e.g., low-capacity, clustered locations, and CVRPLib instances), with the new PE demonstrating resilience and improved out-of-distribution performance.

Implications and Future Directions

The findings force a revision of the conventional wisdom regarding PE as a secondary or task-agnostic Transformer component. Geometry-grounded, hierarchically-structured, and cyclic-aware PEs are demonstrated to be critical for high-performance neural solvers in VRPs. The methodology applies broadly to any sequential decision task where output tokens are not evenly spaced and possess geometric or topological structure.

From a theoretical perspective, this work shifts PE design towards a problem-centric approach, suggesting future research in:

• Tailoring PE to other spatial or hierarchical combinatorial domains (e.g., multi-agent pathfinding, scheduling).
• Extending to richer VRP generalizations (multi-depot, time-dependent travel).
• Jointly optimizing PE functional form alongside neural architectures for dynamic adaptation to problem structure.

Conclusion

This work provides the first systematic study of positional encoding for neural vehicle routing, mapping out structural PE desiderata neglected by prior methods, and demonstrating both theoretically and empirically that respecting the anisometric, cyclic, and hierarchical structures of VRP solutions leads to measurable and transferable improvements in performance. The practical implication is unequivocal: for neural VRP solvers, the choice and structure of positional encoding is a first-order concern, not a minor architectural detail.