VERUS-LM: Neurosymbolic Framework

• VERUS-LM is a neurosymbolic framework that integrates large language models with symbolic solvers to perform varied logical reasoning tasks.
• It employs a modular two-phase architecture with generic prompting and knowledge-query separation to enhance efficiency and reusability.
• The framework demonstrates robust performance on benchmarks by leveraging self-refinement and modular symbolic solving for tasks like verification and optimization.

VERUS-LM is a neurosymbolic framework that systematically integrates LLMs with symbolic solvers to perform a broad range of logical reasoning tasks. The framework addresses key limitations of prior approaches, including poor generalizability due to reliance on task-specific prompts, inefficiencies from repeated domain knowledge extraction, and restricted inferential capacity. VERUS-LM employs generic prompting, strict separation of reusable knowledge bases from queries, and supports a spectrum of inference tasks—including verification, explanation, optimization, and constraint satisfaction—through a modular architecture that leverages both LLMs and the IDP-Z3 solver under a variant of first-order logic (FO($\backslash$)) (Callewaert et al., 24 Jan 2025).

1. Objectives and Motivations

VERUS-LM is developed to advance neurosymbolic reasoning by combining the semantic understanding of LLMs with the transparency and correctness of symbolic solvers. The framework aims to:

• Enhance versatility: Facilitate a range of reasoning tasks (e.g., verification, explanation, optimization) without per-task customization.
• Promote knowledge reusability: Decouple the domain knowledge base (KB) from individual queries, allowing efficient multi-query operation.
• Eliminate prompt engineering: Operate without hand-crafting dataset- or task-specific in-context examples for prompt construction.

These objectives directly respond to the inefficiency, limited generalizability, and narrow inferential scope identified in previous pipelines (Callewaert et al., 24 Jan 2025).

2. Modular Two-Phase Architecture

VERUS-LM comprises a rigorously modularized, two-phase pipeline:

Phase I: Knowledge Base Creation

• Symbol Extraction: An LLM receives natural-language domain descriptions and extracts types, predicates, and functions, producing a logical vocabulary $V$ annotated in FO($\backslash$).
• Formula Extraction: The LLM generates a logical theory $T$ over $V$, guided by FO($\backslash$) grammar sketches and generic example pairs mapping NL to logic.
• Self-Refinement: Iteratively applies two subprocedures:
  • Syntax refinement: LLM corrects formulaic syntax using errors from FOLINT.
  • Semantic refinement: If $T$ is unsatisfiable under IDP-Z3, LLM repairs it by referencing the minimal unsatisfiable core.

Phase II: Inference

• Task Detection: A BERT classifier parses the NL query $Q$ and assigns it one of eight supported reasoning tasks.
• Information Extraction: Extracts partial structures ($S_0$), optimization terms ($t$), or target formulas ($V$0) via either a small LLM (SLM, e.g., Phi 3.5 mini) or a larger LLM as required by query complexity.
• Symbolic Solving: IDP-Z3 is invoked on $V$1, with results translated back to NL.

This architecture ensures strict decoupling of domain knowledge from queries and modular adaptation to task diversity (Callewaert et al., 24 Jan 2025).

3. Generic Prompting and Knowledge/Query Separation

Prompts for symbol and formula extraction are generic, domain-agnostic, and explicitly avoid task-specific exemplars. Each prompt specifies:

• The FO($V$2) syntax and typing rules;
• Three generic examples for symbol extraction (type, function, predicate);
• Two logic/NL example sentence pairs for formula extraction;
• The NL segment to be formalized.

At inference, prompts only request extraction of values or constraints, never direct reasoning—delegating formal inference to the symbolic solver. This ensures generalizability across domains without further engineering, as well as reusability of the constructed KB for multiple queries (Callewaert et al., 24 Jan 2025).

4. Supported Reasoning Tasks

VERUS-LM supports the following eight reasoning tasks (via IDP-Z3 under FO($V$3)), operating over a KB $V$4 and structures $V$5:

Task Name	Description
Model Generation	Find $V$6 such that $V$7
Satisfiability	Check if $V$8
Optimization	$V$9 subject to $\backslash$0
Propagation	For each atom $\backslash$1, test if $\backslash$2 true/false
Explanation	Given $\backslash$3 true in all $\backslash$4, or $\backslash$5 inconsistent—return minimal proof/explanation
Determine Range	Compute $\backslash$6
Relevance	Identify symbols $\backslash$7 where differing $\backslash$8 in $\backslash$9 and $T$0 yields $T$1
Logical Entailment	Check if $T$2

The spectrum of tasks extends from classical logic evaluation (entailment, satisfiability) to non-classical queries such as optimization and relevance (Callewaert et al., 24 Jan 2025).

5. Experimental Evaluation and Results

VERUS-LM’s performance is assessed on both a custom “DivLR” dataset (115 NL queries spanning six domains) and standard reasoning benchmarks, using metrics such as BERT-based task-detection accuracy and execution accuracy (successfully formed, satisfiable KB plus gold answer from IDP-Z3).

DivLR Dataset: Average Execution Accuracy

System	Avg. Execution Accuracy
SLM alone	39.5%
LLM alone	66.7%
VERUS-LM (V-SLM)	80.3%
VERUS-LM (V-LLM)	91.8%

VERUS-LM (V-LLM) markedly surpasses standalone LLMs and SLMs, especially on novel domains (BMI*), illustrating the benefit of explicit knowledge-query separation and symbolic integration.

Standard Benchmarks: Task Accuracy (%)

System	PrOntoQA	ProofWriter	FOLIO	LogicalDeduction	AR-LSAT
VERUS-LM	95.8	93.8	78.4	88.7	68.4
SymbCoT	99.6	82.5	83.3	93.0	43.9
Logic-LM	83.2	78.8	68.6	87.6	23.4
LINC	—	98.0	72.5	—	—
GPT-4	77.4	52.7	69.1	71.3	33.3
GPT-4+CoT	98.8	68.1	70.6	75.3	35.1

VERUS-LM is competitive on standard semantic benchmarks and outperforms all alternatives by at least 25 points on AR-LSAT, a diverse multi-task dataset.

Effect of Self-Refinement

• Syntax refinement yields +10–20 percentage points in execution rates.
• Semantic refinement yields an additional +10 point gain.
• Net improvement: 9–15 percentage points overall (Callewaert et al., 24 Jan 2025).

6. Efficiency, Generalizability, and Comparative Analysis

VERUS-LM’s architecture achieves cost efficiency by constructing the KB once and reusing it for multiple queries within the same domain. The use of SLMs (e.g., Phi 3.5 mini) constrained by grammars for extraction tasks enables fully local deployments, obviating the need for cloud LLM calls for routine queries.

In contrast to prior systems:

• LINC, Logic-LM: Rely on per-task in-context examples and manage knowledge formalization separately for each task.
• SymbCoT: Utilizes LLMs to reason step-wise without external solvers, enabling robustness but limited for tasks demanding optimization or model enumeration.
• VERUS-LM: Achieves hybrid integration, balancing LLM interpretability with the provable correctness and structured inference of symbolic solvers (Callewaert et al., 24 Jan 2025).

The framework’s generic prompting avoids re-engineering across domains, supporting eight logical tasks natively. Expressivity in FO($T$3) encompasses aggregates, definitions, and inductive rules.

7. Limitations and Prospects

Several constraints circumscribe VERUS-LM’s applicability:

• Expressiveness: FO($T$4), although rich, does not subsume higher-order or complex planning logics.
• World Assumption Handling: Closed-World Assumption (CWA) is default; Open-World Assumption (OWA) is only empirically approximated using an “unknown” element, lacking semantic guarantees.
• Scalability: Large domains or high-arity predicates may limit IDP-Z3’s back-end solvability.
• FO($T$5) Fluency Dependency: Prompt-based KB creation’s robustness varies with LLM proficiency in FO($T$6).

Identified avenues for future development include specialized LLMs for FO($T$7), expansion to alternate logics (temporal, probabilistic), and empirical validation on extensive real-world knowledge bases with interactive, multi-query demands (Callewaert et al., 24 Jan 2025).