Learning with Local Search MCMC Layers (2505.14240v1)

Abstract: Integrating combinatorial optimization layers into neural networks has recently attracted significant research interest. However, many existing approaches lack theoretical guarantees or fail to perform adequately when relying on inexact solvers. This is a critical limitation, as many operations research problems are NP-hard, often necessitating the use of neighborhood-based local search heuristics. These heuristics iteratively generate and evaluate candidate solutions based on an acceptance rule. In this paper, we introduce a theoretically-principled approach for learning with such inexact combinatorial solvers. Inspired by the connection between simulated annealing and Metropolis-Hastings, we propose to transform problem-specific neighborhood systems used in local search heuristics into proposal distributions, implementing MCMC on the combinatorial space of feasible solutions. This allows us to construct differentiable combinatorial layers and associated loss functions. Replacing an exact solver by a local search strongly reduces the computational burden of learning on many applications. We demonstrate our approach on a large-scale dynamic vehicle routing problem with time windows.

• The paper introduces a principled framework that transforms local search heuristics into MCMC proposal distributions for differentiable combinatorial layers.
• It reformulates neighborhood exploration as a Markov chain process, aligning local search with the Metropolis-Hastings acceptance criterion.
• The approach achieves computational efficiency and scalability in NP-hard problems, as demonstrated on dynamic vehicle routing tasks.

Integrating Local Search Heuristics via MCMC for Differentiable Combinatorial Layers

Recent investigations have explored the integration of combinatorial optimization procedures as layers within neural network architectures. This paradigm seeks to imbue neural networks with the capacity to reason about structured combinatorial problems, enabling end-to-end learning for tasks involving discrete decision variables. A significant challenge in this area pertains to the differentiability of combinatorial operations, which are inherently non-smooth and non-differentiable. Existing methodologies often rely on either exact solvers or differentiable relaxations. However, the computational intractability of many NP-hard problems necessitates the use of inexact solvers, such as heuristic or metaheuristic approaches, in practical scenarios. Integrating these inexact solvers into differentiable pipelines while maintaining theoretical principles remains an active area of research.

A critical limitation arises when extant approaches attempt to utilize inexact solvers, as they frequently lack theoretical guarantees on the quality of gradients or fail to achieve adequate performance. This is particularly pertinent for neighborhood-based local search heuristics, which are widely employed to find approximate solutions to complex optimization problems. These heuristics operate by iteratively exploring a defined neighborhood around the current solution, evaluating candidate solutions, and transitioning to a new solution based on a specified acceptance criterion. Developing a principled framework for learning with such widely used, yet inexact, combinatorial solvers is essential for scaling combinatorial optimization layers to larger and more complex problem instances.

Methodology: Transforming Local Search Neighborhoods into MCMC Proposal Distributions

The paper "Learning with Local Search MCMC Layers" addresses this challenge by proposing a theoretically-principled approach that leverages the connection between simulated annealing and Metropolis-Hastings algorithms. The core technical contribution lies in transforming problem-specific neighborhood systems, which are fundamental components of local search heuristics, into proposal distributions suitable for Markov Chain Monte Carlo (MCMC) sampling on the combinatorial space of feasible solutions.

In standard local search, a neighborhood function $\mathcal{N}(s)$ maps a solution $s$ to a set of neighboring solutions. The heuristic then evaluates a candidate solution $s' \in \mathcal{N}(s)$. The proposed methodology reframes this neighborhood exploration within the MCMC framework. Specifically, the neighborhood structure is utilized to define the proposal distribution $q(s' | s)$, representing the probability of proposing a move from the current state $s$ to a candidate state $s'$. By constructing $q(s' | s)$ based on the neighborhood system, the iterative process of local search can be interpreted as a sequence of transitions within a Markov chain over the combinatorial state space.

Drawing inspiration from simulated annealing, which can be viewed as an MCMC procedure targeting a distribution that becomes increasingly concentrated on optimal solutions, the paper establishes a connection between the local search acceptance rule and the Metropolis-Hastings criterion. This allows for the formulation of the local search steps as transitions governed by a Metropolis-Hastings acceptance probability, ensuring that the sampling process converges to a stationary distribution. This theoretically grounded connection provides a basis for analyzing the properties of the resulting combinatorial layer.

Differentiable Layer Construction via MCMC Sampling

The MCMC formulation on the combinatorial state space facilitates the construction of differentiable combinatorial layers. By framing the output of the local search process as a sample from the stationary distribution of the defined Markov chain, it becomes possible to utilize techniques for differentiating through stochastic sampling processes. This typically involves methods like the score function estimator (REINFORCE) or pathwise derivatives, provided suitable conditions on the stochastic process and the dependency on parameters are met.

Within this framework, the parameters of the neural network can influence the MCMC process in several ways. For instance, the network's output might parameterize the objective function being optimized, the acceptance criterion (analogous to the temperature parameter in simulated annealing), or even the proposal distribution itself by influencing the selection probabilities of neighbors within the defined neighborhood structure. By establishing these dependencies, the entire optimization procedure, now cast as an MCMC sampling process guided by the network's parameters, becomes amenable to gradient-based learning.

Replacing an exact combinatorial solver with this local search-based MCMC layer offers significant computational advantages, particularly for large-scale problems. Exact solvers for NP-hard problems often have exponential complexity in the worst case, making them impractical for integration into training pipelines that require numerous forward and backward passes. Local search heuristics, while inexact, typically have polynomial time complexity per iteration, and the number of iterations required to find a good solution is often manageable in practice. Integrating such a computationally cheaper, inexact solver as a differentiable layer strongly reduces the computational burden associated with learning on applications involving combinatorial optimization.

The resulting differentiable layer takes input from the neural network (e.g., problem instance parameters, context information) and outputs a sampled solution obtained by running the MCMC-enhanced local search process for a certain number of steps. The loss function can then be defined based on the quality of the sampled solution (e.g., its objective value) or its relationship to target outputs, and gradients can be propagated back through the MCMC sampling process to update the neural network parameters.

Application to Dynamic Vehicle Routing Problems

The paper demonstrates the proposed approach on a large-scale dynamic vehicle routing problem with time windows (DVRPTW). This problem class is a classic example of a complex combinatorial optimization problem that is notoriously difficult to solve exactly, especially in dynamic settings where requests arrive over time. DVRPTW instances typically require sophisticated heuristics or metaheuristics for real-time decision-making.

In this context, the neural network could potentially learn to predict problem parameters, guide the local search process by suggesting promising neighborhoods or move evaluations, or directly output parameters for the MCMC sampling (e.g., acceptance probabilities or transition rates). By applying the MCMC local search layer to this problem, the paper aims to show that the framework can effectively learn to optimize complex, large-scale combinatorial problems within an end-to-end learning pipeline while leveraging the computational efficiency of local search. The application serves as empirical validation that transforming neighborhood-based local search heuristics into MCMC proposal distributions enables the construction of effective differentiable combinatorial layers for practical, computationally demanding problems.

Conclusion

The work introduces a theoretically-principled method for incorporating inexact combinatorial solvers, specifically neighborhood-based local search heuristics, into differentiable learning frameworks. By reinterpreting local search neighborhood systems as MCMC proposal distributions, the paper enables the construction of differentiable combinatorial layers compatible with gradient-based optimization. This approach offers a significant computational advantage over methods relying on exact solvers, making it suitable for large-scale NP-hard problems. The effectiveness of the proposed framework is demonstrated through its application to a large-scale dynamic vehicle routing problem with time windows.