Neuro-Symbolic Software Verification: Hyper-charging Local Language Models with Symbolic Reasoning at Scale

Abstract: Loop invariant synthesis remains a central and pivotal bottleneck in formal software verification. Recent LLM-based Neuro-Symbolic tools have achieved impressive solve rates. However, these tools rely on proprietary, often expensive cloud APIs, which constitute a hurdle for privacy-sensitive industrial deployments where the source code cannot leave the organisation or where cost is a factor. We present VerIbmc, a neuro-symbolic pipeline that pairs symbolic invariant generation with locally deployable open-weight LLMs with the ESBMC verification tool. Our pipeline combines a deterministic symbolic invariant synthesis phase with an iterative LLM refinement loop driven by structured verifier feedback. In addition, we provide two types of pipelines that differ in their prompting strategy: Chain-of-Thought vs. Tree-of-Thought. We conduct an extensive experimental evaluation with five open-weight models (ranging from 7B to 120B parameters) across five benchmark families comprising of 520 problems (499 after excluding 21 with unavoidable overflow). Overall, the best single configuration (GPT-OSS-120B) solves 431 of 499 problems (86.4%). Additionally, on the four benchmark suites shared with the strongest cloud-API tools, VerIbmc is competitive running only on a single local machine. The evaluation shows symbolic invariant synthesis solves 75 problems without any LLM call and yields up to +35 additional problems for the weakest model. Importantly, all inference runs entirely on a single local machine using open-weight models -- no cloud API or proprietary model is required. Overall, we demonstrate that a neuro-symbolic approach based on LLMs can be used effectively for invariant synthesis in a privacy-preserving and energy-efficient manner, without having to resort to expensive proprietary frontier models locked behind APIs.

• The paper presents VerIbmc, a neuro-symbolic verification pipeline that merges deterministic symbolic invariant synthesis with LLM-based refinement for loop invariants.
• The method utilizes a phased approach that first applies symbolic verification, then calls LLM synthesis only when symbolic methods fail, ensuring soundness through ESBMC checks.
• Experimental results reveal that integrating symbolic feedback notably improves solve rates, especially for weaker models, while preserving privacy and enabling local deployment.

Neuro-Symbolic Software Verification with Local LLMs and Symbolic Reasoning

Introduction and Motivation

The paper "Neuro-Symbolic Software Verification: Hyper-charging Local LLMs with Symbolic Reasoning at Scale" (2606.16886) addresses an acute limitation in formal software verification: the synthesis of inductive loop invariants. While recent LLM-based neuro-symbolic tools have achieved notable solve rates, their reliance on proprietary, API-gated LLMs hinders applicability in privacy-critical or resource-limited environments. This work presents VerIbmc, a neuro-symbolic software verification pipeline that combines deterministic symbolic invariant synthesis with locally servable, open-weight LLMs integrated via a structured interactive refinement loop with the ESBMC verification backend.

The approach explicitly targets security-, safety-, and privacy-sensitive production environments where transmitting source code to an external API is non-viable. Rather than relying solely on neural guess-and-check, VerIbmc introduces a sound pipeline that first applies symbolic methods, invoking LLM-based synthesis only when symbolic methods fail, and verifying all LLM outputs through ESBMC.

VerIbmc Pipeline Overview

VerIbmc consists of a three-phase architecture, escalating from symbolic reasoning to neural refinement only as necessary (Figure 1):

• Phase 0: Baseline verification using ESBMC attempts to prove or disprove safety with no loop invariants supplied.
• Phase 1: Systematic, deterministic symbolic generation and verification of atomic invariants for each loop.
• Phase 2: Iterative LLM-based invariant proposal and refinement, guided by structured counterexample feedback and atomic decomposition, with each candidate (or its provable atoms) verified via ESBMC’s loop-invariant-checking mode.

  Figure 1: The VerIbmc flow diagram, illustrating phased escalation from symbolic verification to LLM-guided invariant refinement, with ESBMC as the verification oracle.

Four inference strategies arise from the combination of symbolic feedback (LLM-Only vs. Basic) and prompt reasoning structure (Chain-of-Thought (CoT) vs. Tree-of-Thoughts (ToT)). Basic always leverages Phase 1 symbolic prior; LLM-Only does not. Each can run with linear CoT or ToT prompting over multiple derivation styles, and the pipeline’s architecture is fully model-agnostic.

Symbolic and Neuro-Symbolic Integration

ESBMC serves as the formal verification oracle throughout the entire process, supporting both atomic and whole-program invariant validation under $k$-induction. The symbolic stage is realized through systematic enumeration of variable- and constant-based atomic invariants, each individually classified (provable/disprovable/unknown) by ESBMC. For nontrivial problems, this often yields a verified partial invariant, which acts as structured context to the LLM in subsequent prompts.

The neural refinement phase leverages LLMs for invariant synthesis that is robust to model hallucinations and errors. LLM proposals are tokenized, parsed, and decomposed to atomic statements, each being verified for inductiveness before recombination and re-verification at the program level. The inclusion or exclusion of the symbolic phase is critical: the symbolic prior both reduces LLM search burden and improves performance, especially for weaker models.

Prompting Paradigms: CoT and ToT

Prompt engineering is systematically explored along two axes: Chain-of-Thought (CoT), in which the LLM is guided through step-by-step derivations (ranging from few-shot inductive reasoning to abstract Horn clause construction), and Tree-of-Thought (ToT), which orchestrates parallel multi-style exploration and heuristic-based refinement over more than one derivation style in search space.

The ToT approach consists of a scouting phase (one LLM proposal per derivation style) and a refinement phase focusing resources on the most promising styles, with selection and recombination tightly coupled with ESBMC’s oracle feedback. This explicit search over the prompt configuration leads to improved coverage on some benchmarks, at a compute cost that depends significantly on model strength.

Experimental Evaluation and Numerical Results

A systematic evaluation spanning five open-weight models (7B to 120B parameters), four inference strategies, and five established benchmark families (499 effective problems) yields several notable outcomes:

• The best single local model/strategy configuration (GPT-OSS-120B/Basic) achieves 431/499 (86.4%) successful solves, with performance strongly correlated to model size.
• The symbolic Phase 1 mechanism solves 75 problems without any LLM invocation; for weaker models (e.g., Llama-3.1-8B), inclusion of the symbolic phase yields up to 35 additional solves over LLM-only runs.
• Across all five models, the neuro-symbolic Basic strategy consistently outperforms or matches the LLM-Only baseline. For weaker LLMs, symbolic feedback is a material advantage.
• The pipeline matches or exceeds single-task solve rates of prior SOTA, while maintaining full local deployment without external API calls.

Figure 2: Contribution per pipeline phase by benchmark family. Phase 0 (baseline ESBMC) and Phase 1 (symbolic) solve significant fractions before entering the LLM-guided phase.

• Comparison with traditional and neural tools (CPAchecker, 2LS, ESBMC-Kind, Code2Inv) demonstrates a consistently superior aggregate solve rate for VerIbmc’s best configurations.
• Against neuro-symbolic SOTA tools requiring external closed APIs (LaM4Inv, Clause2Inv, LORIS), VerIbmc is competitive on overlapping suites and uniquely generalizes to the full SV-COMP loop set, including complex programs with arrays, nested/multi-loops, and pointers.

  Figure 3: Pairwise Jaccard similarity across the (model, strategy) grid on nontrivial problems, indicating robustness of solvable frontier to strategy/model choice once baseline is fixed.

Additional analysis reveals that while ToT-based strategies yield minor aggregate solve gains for strong models, their true advantage is in coverage diversity—reaching a few exclusive (model, problem) pairs not accessible via standard CoT, at a moderate wall-clock computational overhead.

Discussion and Theoretical Implications

The iterative, sound integration of symbolic and neural methods addresses the largest bottleneck in deductive program verification under general language semantics. Unlike pure neural approaches, VerIbmc is immune to hallucinated invariant acceptance. Unlike prior cloud-reliant neuro-symbolic tools, it admits deployment in privacy-sensitive, on-premise, or resource-constrained environments, and its energy and cost footprint aligns with the host environment rather than external API policies.

Model capability is the primary driver for accuracy, with the impact of pipeline strategy most pronounced for weaker LLMs. Symbolic feedback and atomic invariant decomposition/selection provide deterministic gains and improve the sample efficiency of the neural proposal mechanism. ToT prompting further increases diversity, although at a cost that is only justified for the smallest models or the most hard-to-crack instances.

These findings have broad implications for the integration of LLMs in program verification and, by extension, in any symbolic domain where soundness and privacy are critical. The systematic architecture and experimental design underscore that scale, modularity, and rigorous feedback are all necessary (but individually insufficient) for robust neuro-symbolic verification.

Practical Implications and Future Directions

Practically, VerIbmc demonstrates that fully local LLM-augmented neuro-symbolic verification pipelines are viable for large-scale industrial usage, and do not require dependency on API-gated, cost-prohibitive, or non-auditable LLMs. The modular structure enables direct integration with arbitrary verification backends, and the phased escalation ensures computational resources are conserved.

Unresolved problems at the hard SOTA frontier involve multi-loop, recursive, non-linear, or heap-associated invariant synthesis, which will require further advances in both symbolic candidate enumeration (e.g., non-linear, array, or separation logic templates) and neural feedback integration (e.g., fine-grained proof step correction as in LORIS). Further refinement of ToT scoring, adaptive iteration policies, and constrained decoding will also impact efficiency and reliability.

Conclusion

VerIbmc establishes a scalable, sound, and privacy-respecting neuro-symbolic framework for loop invariant synthesis and deductive verification built over open-weight LLMs and sound symbolic feedback. It advances the practical boundaries of full-pipeline software verification and provides a blueprint for scalable, auditable, and cost-effective integration of neural reasoning into critical formal methods pipelines. Future work will entail extending symbolic candidate domains, adaptive search/budget policies, and integrating fine-grained error feedback for even tighter symbolic–neural synergy.