Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Design (2501.08603v3)

Abstract: Handcrafting heuristics for solving complex optimization tasks (e.g., route planning and task allocation) is a common practice but requires extensive domain knowledge. Recently, LLM-based automatic heuristic design (AHD) methods have shown promise in generating high-quality heuristics without manual interventions. Existing LLM-based AHD methods employ a population to maintain a fixed number of top-performing LLM-generated heuristics and introduce evolutionary computation (EC) to iteratively enhance the population. However, these population-based procedures cannot fully develop the potential of each heuristic and are prone to converge into local optima. To more comprehensively explore the space of heuristics, this paper proposes to use Monte Carlo Tree Search (MCTS) for LLM-based heuristic evolution. The proposed MCTS-AHD method organizes all LLM-generated heuristics in a tree structure and can better develop the potential of temporarily underperforming heuristics. In experiments, MCTS-AHD delivers significantly higher-quality heuristics on various complex tasks. Our code is available.

• The paper proposes Monte Carlo Tree Search (MCTS) integrated with Large Language Models (LLMs) to improve automatic heuristic design for combinatorial optimization, addressing the premature convergence of prior population-based methods.
• The MCTS-AHD method explores the heuristic design space comprehensively using a tree structure and incorporates novel techniques like thought-alignment and exploration-decay.
• Empirical evaluation shows MCTS-AHD consistently outperforms baseline LLM-based methodologies on various NP-hard tasks, such as TSP and Knapsack Problems, demonstrating improved heuristic quality and convergence efficiency.

The paper "Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Design" introduces an innovative approach for designing heuristics using LLMs in combinatorial optimization (CO) tasks. The current landscape of CO solutions often involves manually crafted heuristics, demanding significant domain expertise. LLM-based Automatic Heuristic Design (AHD) methods have offered a way to autonomously generate high-quality heuristic functions, yet they frequently rely on population-based evolutionary strategies that can converge prematurely to local optima due to their inherently greedy nature.

The authors propose an alternative methodology leveraging Monte Carlo Tree Search (MCTS) integrated with LLMs to enhance heuristic generation. This method, termed MCTS-AHD, utilizes a tree structure to comprehensively explore the heuristic space, maintaining all LLM-generated functions. Unlike traditional population methods, MCTS's ability to preserve diverse heuristic candidates allows it to potentially escape local optima. The proposed framework introduces novel techniques within the MCTS process, including a thought-alignment procedure for aligning heuristic function descriptions with their implementations and an exploration-decay technique that dynamically adjusts exploration-exploitation considerations over time.

Key contributions of the paper include:

1. Initial Function Generation: The framework uses an LLM to generate initial heuristic functions, seeding the MCTS process without the necessity of pre-defined or manually developed seed functions.
2. Inference and Tree Expansion: Action strategies such as mutation and crossover are employed within the MCTS to explore new nodes. They introduce specific actions tailored to exploit the tree structure, facilitating the cross-pollination of diverse solutions.
3. Progressive Widening and Exploration Decay: The method applies a progressive widening strategy, where the breadth of exploration expands dynamically with accumulated experience. An exploration-decay mechanism is incorporated to balance exploration and exploitation, ensuring richer exploration initially with convergence towards optimal solutions over time.
4. Empirical Evaluation: The MCTS-AHD method was extensively tested across various NP-hard CO tasks, such as the Traveling Salesman Problem (TSP) and Knapsack Problems (KP), using step-by-step construction, Ant Colony Optimization (ACO), and Guided Local Search (GLS) frameworks. It consistently outperformed baseline LLM-based methodologies in both quality and convergence efficiency.

The implementation of MCTS in the context of LLM-based AHD demonstrates improved exploration of potential heuristic designs and provides a robust methodology adaptable to diverse problem settings. The empirical results affirm the efficacy of the MCTS framework in achieving significant performance gains over existing heuristic approaches, suggesting its applicability in broader heuristic-driven problem-solving landscapes. The paper opens avenues for further exploration into the synergy between AI models and traditional optimization algorithms, particularly in applications requiring adaptive and autonomous decision-making capabilities.