BCTuner: LLM-Guided Monte Carlo Tree Search for Efficient Blockchain Knob Tuning

Abstract: Knob tuning plays a critical role in improving the performance of permissioned blockchains. However, efficient tuning remains challenging due to the architectural complexity of blockchains and the semantic gap between knob-specific logic and the numerical optimization requirements of tuning tools. In addition, configuration changes are often coupled across different stages of the transaction pipeline, making their performance impact difficult to isolate and predict. Since each trial requires deployment and distributed benchmarking, ineffective exploration incurs substantial cost. These challenges motivate BCTuner, a LLM-guided framework that combines knowledge-guided reasoning with structured search. BCTuner organizes multi-source tuning knowledge to support LLM-based reasoning over knob semantics, constraints, and deployment context. It formulates tuning as a Monte Carlo Tree Search (MCTS) process over structured action trajectories, where configurations are incrementally constructed, validated, evaluated, and refined rather than generated in one step. BCTuner further applies adaptive pruning to discard infeasible or low-potential branches before system evaluation. We evaluate BCTuner on Hyperledger Fabric and ChainMaker under diverse workloads and network settings. Experimental results show that BCTuner achieves up to 211.38% throughput improvement over default configurations and outperforms the state-of-the-art blockchain tuning method by up to 20% in performance, while requiring up to 8x fewer interactions with the blockchain system.

• The paper introduces BCTuner, an LLM-guided Monte Carlo Tree Search framework that combines system knowledge, staged tuning actions, validation, feedback refinement, and cost-aware pruning for blockchain configuration optimization.
• BCTuner improves throughput by up to 211.38% over default settings, delivers average gains of 61.61% on SmallBank and 106.67% on Simple, and converges up to 8.27 times faster than competing tuners.
• The paper shows that structured knob knowledge and feedback refinement are critical for reliable tuning, while remaining challenges include hand-tuned pruning thresholds, uneven baseline search spaces, and limited evaluation beyond Fabric and ChainMaker.

Problem and motivation

Permissioned blockchains such as Hyperledger Fabric and ChainMaker expose large configuration spaces spanning consensus, networking, execution, validation, storage, and resource management. The paper identifies three obstacles that distinguish blockchain knob tuning from conventional DBMS tuning. First, knobs are heterogeneous in type, unit, range, and special values, and their effects depend on hardware, node roles, and topology, so tuning requires semantic and contextual reasoning rather than purely numerical search. Second, knobs across the transaction pipeline (proposal execution, endorsement, ordering, validation, commit) are coupled; a local adjustment such as increasing block size can shift the bottleneck from ordering to validation or storage. Third, each trial requires deployment or reconfiguration plus distributed benchmarking, making ineffective exploration prohibitively expensive.

The authors position BCTuner as the first LLM-based framework for blockchain knob tuning. Existing LLM-assisted database tuners—DB-BERT, GPTuner, MCTuner, $\lambda$-Tune, E2ETune, AgentTune—are argued to be unsuitable because they either treat the LLM as a preprocessing aid within iterative Bayesian optimization, depend on offline training data, or rely on SQL-specific workload abstractions that do not transfer to blockchains (2605.23280).

Framework design

BCTuner operates in two phases. In an offline phase, it constructs structured knowledge from system manuals and administrator-provided metadata at three levels: knob-level knowledge (semantics, types, units, special values, functional clustering along the transaction pipeline), hardware-level knowledge (CPU, memory, storage constraints), and network-level knowledge (node roles and topology). In the online phase, tuning is formulated as MCTS over structured action trajectories: each tree node is a configuration state, edges are LLM-generated tuning actions, and configurations are incrementally constructed, validated, evaluated, and refined rather than produced in one shot.

The action space comprises eight action types with explicit transition rules constraining valid orderings:

• A1 Global Tuning Plan: selects which pipeline clusters to prioritize.
• A2 Cluster-wise Tuning: jointly tunes semantically related knob groups.
• A3 Single-knob Tuning: fine-grained refinement of individual critical knobs.
• A4 Knob Validation / A5 Knob Fix: checks feasibility (ranges, formats, inter-knob consistency) and repairs violations before deployment.
• A6 Performance Evaluation: deploys and benchmarks on the real system.
• A7 Feedback-driven Refinement: uses runtime metrics and execution errors to target bottleneck knobs iteratively.
• A8 Terminal: stops under predefined conditions.

Action-specific prompts constrain the LLM's reasoning scope per decision type, mitigating attention dispersion over many knobs in one-shot generation. Search follows standard UCT selection, expansion with multiple sampled LLM actions per node, simulation to terminal states, and backpropagation of rewards. Rewards are sparse ($\pm 1$) for validation outcomes and exponential in relative throughput improvement for evaluation actions, with feedback actions amplified by $1.5\times$.

Pruning strategy

Because deployment and benchmarking dominate cost, BCTuner applies rule-based pruning during expansion along three principles: stage-aware progression (restricting early search to cluster-level tuning, deferring validation and suppressing frequent evaluations), outcome-aware pruning (terminating branches after repeated invalid validations, restricting successors post-validation, terminating branches whose throughput falls below 90% of initial performance), and performance-guided continuation (entering feedback tuning only if throughput reaches at least 80% of the baseline-to-best interval, and halting refinement on >10% degradation). This shifts pruning decisions ahead of costly system evaluation.

Experimental results

Evaluation covers Hyperledger Fabric v2.5.12 (Raft consensus) and ChainMaker v2.3.7 on six-server deployments, using SmallBank and Simple workloads via Hyperledger Caliper, against SMAC, GP, GPTuner, and Athena (PB-MADDPG). Baselines were restricted to 50 expert-selected Fabric knobs because applying them to all 120 knobs frequently yields invalid configurations that prevent network startup—a notable asymmetry favoring BCTuner, which tunes all 120.

Headline results include up to 211.38% throughput improvement over default configurations (Simple workload, 4 peers), average gains of 61.61% (SmallBank) and 106.67% (Simple) across 4–24 peer networks, and convergence within as few as 6–21 iterations. Relative to baselines, BCTuner improves TPS by roughly 19–20% over GPTuner and Athena and 23–41% over SMAC and GP, while converging approximately $8\times$ faster (e.g., $8.27\times$ vs. GPTuner on Simple). On ChainMaker, BCTuner achieves 123.71–144.58% improvement over defaults and 11.22–20.94% over Athena with $4.15\times$ faster convergence, indicating cross-system adaptability without system-specific customization.

The ablation study isolates component contributions. Removing knob knowledge causes the largest drop ($\Delta T$ from 102.85% to 68.00%, with nearly half of configurations invalid); removing feedback-driven refinement drops $\Delta T$ by 27.2%; removing cluster-wise tuning costs 11.5%; disabling pruning expands the search from 24 to 43 steps with 39 evaluations required. Time breakdowns confirm that deployment and evaluation dominate total tuning time, while MCTS search overhead remains small—consistent with the claim that the framework's value lies in reducing expensive trials rather than accelerating search computation.

Across ten LLM backbones, all improve throughput over defaults; Gemini-3.1-Pro achieves the highest TPS (1445.56), while lightweight models such as GPT-5-mini attain competitive results at $0.84 per run. Total token consumption ranges from 2.3M to 4.4M tokens ($0.77–$9.34), supporting the cost-effectiveness claim.

Limitations and open questions

Several caveats bear directly on the reported results. The baseline comparison is asymmetric: baselines tune only 50 expert-curated Fabric knobs versus BCTuner's full 120-knob space, so part of the advantage may reflect search-space differences rather than method quality alone. The pruning rules embed several hand-tuned thresholds (90%, 80%, 10%), and the paper does not report sensitivity analysis for these constants. Hardware and network knowledge must be supplied by administrators in structured form, so full automation depends on the accuracy of this manual input. The Global Tuning Plan action shows limited ablation impact, suggesting its contribution may be partially redundant. Finally, evaluation is confined to two permissioned platforms with fixed workloads; generalization to other consensus protocols, heterogeneous hardware, or adversarial workloads remains untested.

Conclusion

BCTuner reformulates blockchain knob tuning as LLM-guided MCTS over structured action trajectories grounded in multi-source system knowledge, with adaptive pruning to avoid costly evaluations. Experiments on Fabric and ChainMaker show substantial throughput gains over defaults and consistent improvements over Bayesian optimization, reinforcement learning, and prior LLM-based tuners, with markedly fewer system interactions. The main open questions concern sensitivity to hand-tuned pruning thresholds, the fairness of the reduced-knob baseline setup, and robustness beyond the two evaluated platforms.