KBSpec: LLM-Driven Spec Generation

Updated 6 July 2026

KBSpec is a multi-lineage framework that externalizes domain knowledge for formal specification generation and repair using LLMs and evolving knowledge bases.
It integrates techniques from symbolic execution, interactive configuration, and formal verification to infer contracts and optimize specification quality.
Empirical results show significant improvements in verification pass rates, demonstrating the practicality of combining external documentation with automated repair strategies.

KBSpec is an overloaded designation spanning several research lineages centered on explicit specification, externalized knowledge, and reusable inference. In current arXiv usage, the most direct referent is "KBSpec: LLM-driven Formal Specification Generation with Evolving Domain Knowledge Base" (Wang et al., 19 Jun 2026). Earlier work uses closely related names such as KindSpec 2.0 and KINDSPEC 2.0 for K-framework-based contract inference over KernelC (Alpuente et al., 2016, Alpuente et al., 2015). A broader antecedent is the knowledge base paradigm for interactive configuration, in which a single declarative knowledge base supports multiple inference tasks without changing the domain theory (Hertum et al., 2016).

1. Terminological scope and principal referents

The label has not denoted a single, stable formalism. Instead, it has referred to several specification-centered architectures that share a family resemblance: domain knowledge is made explicit, inference is separated from representation, and the resulting system is designed to support multiple downstream reasoning tasks.

Usage	Domain	Core characterization
KBSpec	Formal specification generation	LLM-driven formal specification generation with a self-evolving knowledge base
KindSpec 2.0 / KINDSPEC 2.0	Contract inference for C/KernelC	Symbolic execution, lazy initialization, abstract subsumption, observer-based axioms
Knowledge Base paradigm	Interactive configuration	FO(.) knowledge base reused by model expansion, propagation, optimization, and explanation

In the 2026 formulation, KBSpec targets Java Modeling Language generation and repair by combining external documentation with internal verifier-grounded memories (Wang et al., 19 Jun 2026). In the K-framework lineage, the same general name is associated with automatic inference of preconditions, postconditions, and frame conditions for heap-manipulating C-like programs using symbolic execution over an executable semantics (Alpuente et al., 2016, Alpuente et al., 2015). In the interactive-configuration lineage, KBSpec is best understood as a "knowledge-base specification" style: vocabulary, theory, and structures are specified once, and functionalities such as completion, checking, propagation, and explanation are obtained by different logical inferences over the same representation (Hertum et al., 2016).

2. K-framework specification inference for KernelC

The K-based lineage treats specification synthesis as a semantics-driven analysis problem. Programs are written in KernelC, a non-trivial fragment of C supporting functions, local variables, structs, heap allocation, pointer types, pointer dereferencing and update, and control structures such as conditionals and loops. Execution state is represented as a K configuration with cells such as k, env, heap, and, in the symbolic setting, path-condition, init-heap, aSubFlag, and locations (Alpuente et al., 2016, Alpuente et al., 2015).

The central methodological move is to explain a modifier function in terms of observer routines from the same program. Symbolic execution starts from symbolic inputs and explores feasible paths, with lazy initialization handling first access to symbolic heap addresses and abstract subsumption folding loops or recursive patterns when an abstract state is subsumed by an earlier one. Contracts are inferred as logical axioms of the form

$p \Rightarrow q,$

where $p$ is a conjunction of equations over observer calls in the initial state and $q$ is a conjunction of equations over observer calls in the final state together with the modifier’s return value. At the method level, the contract is written as

$\langle P, Q, \mathcal{L} \rangle,$

with $P$ the disjunction of antecedents, $Q$ the conjunction of per-path implications, and $\mathcal{L}$ the union of modified locations (Alpuente et al., 2016).

The later system refines the earlier 2015 formulation in two ways. First, it replaces fixed loop unrolling with abstract subsumption plus heap abstraction, allowing finite but more general symbolic exploration. Second, it introduces a refinement stage for paths flagged by aSubFlag = true: candidate axioms derived from abstracted paths are tested and, if necessary, specialized or discarded. The running linked-list set example shows the intended outcome. For insert(struct set* s, int x), the system infers observer-based rules for the null case, full set case, already-contains case, and the generalized non-null, non-empty, non-full, not-contains case in which length increases by $1$ and the return value is $1$ (Alpuente et al., 2016).

This line of work is explicitly semantics-relative. Its guarantees depend on the correctness of the underlying K semantics, the symbolic execution engine, and the refinement procedure. The 2016 paper presents the unabstracted-path axioms as semantically correct for the explored behaviors, while abstracted-path axioms remain candidates until validated by testing and specialization (Alpuente et al., 2016).

3. Knowledge-base specification in interactive configuration

The knowledge base paradigm formulates a different but related view of specification. A knowledge base is a repository of pure domain information expressed in a rich logical language; it is neither a program nor a single-purpose solver. Problem solving is delegated to multiple forms of inference over the same representation. In the FO(.) setting used in the paper, a knowledge base consists of a vocabulary $\Sigma$ , a theory $p$ 0, and one or more structures $p$ 1. The logic is FO(T, ID, Agg, arit, PF): first-order logic with types, inductive definitions, aggregates, arithmetic, and partial functions (Hertum et al., 2016).

A partial structure assigns truth values in $p$ 2 and is ordered by a precision relation $p$ 3. This makes incomplete user interaction a first-class object rather than an implementation artifact. The configuration process becomes a chain

$p$ 4

where each user action refines the current partial structure. The same theory then supports several generic inferences: Modelexpand, Modelcheck, Minimize, Propagate, and Query. On top of these, the paper defines configuration-specific derived inferences such as GetOpenTerms, GetConsistentValues, PosConsequences, NegConsequences, CheckConsistency, and explanation operators based on UnsatSubstructure, UnsatSubtheory, MinimalUnsatTheory, and MinimumUnsatTheory (Hertum et al., 2016).

The software-configuration example is intentionally simple but structurally representative. The vocabulary includes types such as software, employee, and int; predicates such as Install, IsOS, and PreReq; and functions such as PriceOf, MaxCost, Cost, and Requester. The theory contains rules for prerequisites, aggregate cost, budget respect, mandatory operating systems, and dual-boot dependencies. The initial structure fixes the domains and known prices, budgets, and prerequisite facts, while the uninterpreted atoms and terms become the user-facing parameters $p$ 5 (Hertum et al., 2016).

The practical contribution is that interactive configuration functionality is not hard-coded separately. In the banking proof of concept, the system handled approximately $p$ 6 parameters and approximately $p$ 7 constraints, with typical response times around $p$ 8 seconds and occasional responses up to $p$ 9 seconds. The paper presents this as evidence that a multi-inferential knowledge-base architecture can be feasible without specialized configuration algorithms, although it also notes that efficiency may lag systems with dedicated propagators or global constraints (Hertum et al., 2016).

4. LLM-driven formal specification generation with an evolving knowledge base

The 2026 KBSpec framework applies the same externalization principle to formal specification generation for Java and JML. Its starting point is that specification languages are low-resource for LLMs: syntax is frequently wrong, verifier-accepted idioms are underrepresented in training corpora, and generated specifications often fail semantic verification even when they parse. KBSpec addresses this by augmenting the LLM with dual-source knowledge: external knowledge from official documentation, tutorials, exercises, and repair guides, and internal knowledge distilled from successful generation and repair trajectories validated by OpenJML (Wang et al., 19 Jun 2026).

A knowledge item is represented as

$q$ 0

where $q$ 1 is the full content, $q$ 2 is an LLM-produced summary, and $q$ 3 is a set of extracted keywords. Each item also carries helpfulness scores

$q$ 4

and a derived rating

$q$ 5

The initial knowledge base is built from OpenJML examples, tutorials, exercises, and human-written repair guidelines from FormalBench. During training, the system generates baseline specifications without knowledge, retrieves relevant items, forms random groups of $q$ 6 items, generates new specifications, verifies them, updates helpfulness scores, and then attempts up to $q$ 7 rounds of repair on failures. After each epoch it retains the top $q$ 8 items by $q$ 9, with $\langle P, Q, \mathcal{L} \rangle,$ 0, and adds new internal items whose similarity to existing entries is below $\langle P, Q, \mathcal{L} \rangle,$ 1 (Wang et al., 19 Jun 2026).

Retrieval is embedding-based: $\langle P, Q, \mathcal{L} \rangle,$ 2 with an additional LLM-based selector at inference time choosing up to $\langle P, Q, \mathcal{L} \rangle,$ 3 retrieved items as prompt context. The resulting system is model-agnostic across GPT‑5.2, GPT‑5‑mini, and DeepSeek‑v3.2, and it requires neither fine-tuning nor labeled ground-truth specifications (Wang et al., 19 Jun 2026).

Conceptually, this KBSpec is not merely retrieval-augmented generation. The verifier is used as an oracle that scores memories by actual impact on verification outcomes, so the evolving knowledge base becomes a compact record of what the specification language and the verifier jointly accept.

5. Empirical behavior, repair patterns, and knowledge evolution

On FormalBench-Base, which contains $\langle P, Q, \mathcal{L} \rangle,$ 4 Java programs, and with training on a $\langle P, Q, \mathcal{L} \rangle,$ 5-program subset of FormalBench-Diverse sampled over $\langle P, Q, \mathcal{L} \rangle,$ 6 epochs, KBSpec improves verification pass rates substantially over strong LLM-based baselines. The paper reports the following comparison against SpecGen (Wang et al., 19 Jun 2026).

Backend	SpecGen PR	KBSpec PR	KBSpec FR
GPT‑5.2	43.35%	58.94%	20.60%
GPT‑5‑mini	24.03%	49.64%	33.33%
DeepSeek‑v3.2	32.76%	43.35%	31.76%

The abstract summarizes this as a $\langle P, Q, \mathcal{L} \rangle,$ 7– $\langle P, Q, \mathcal{L} \rangle,$ 8 improvement in verification pass rates over state-of-the-art LLM-based approaches, while the distributional analysis shows that KBSpec yields the largest number of medium-to-high completeness specifications. The average completeness score can decrease globally because the system converts many previously failing cases into verifying but sometimes weaker specifications; however, among the $\langle P, Q, \mathcal{L} \rangle,$ 9 mutually verified programs compared with SpecGen, KBSpec is equal or better on GPT‑5.2 and GPT‑5‑mini and nearly identical on DeepSeek‑v3.2 (Wang et al., 19 Jun 2026).

Ablation results isolate the major components. Removing knowledge-base evolving drops pass rate from $P$ 0 to $P$ 1 on GPT‑5.2, from $P$ 2 to $P$ 3 on GPT‑5‑mini, and from $P$ 4 to $P$ 5 on DeepSeek‑v3.2. Removing iterative repair is similarly damaging. External initialization matters less than these two components but still provides a measurable gain, while knowledge filtering mainly controls quality and size rather than defining the system’s core behavior (Wang et al., 19 Jun 2026).

The learned knowledge base becomes predominantly internal. In the final composition, external items account for about $P$ 6 for GPT‑5.2, about $P$ 7 for GPT‑5‑mini, and about $P$ 8 for DeepSeek‑v3.2; the rest is dominated by successful generation and especially repair trajectories. Repair-derived knowledge is particularly important: for GPT‑5.2 it accounts for about $P$ 9 of the final KB, and for GPT‑5‑mini about $Q$ 0 (Wang et al., 19 Jun 2026).

The repair taxonomy is equally revealing. The main failure causes are arithmetic obligations at about $Q$ 1– $Q$ 2, unsupported constructs such as \num_of, \sum, \max, and \min at about $Q$ 3– $Q$ 4, and loop reasoning issues at up to $Q$ 5. The main repair strategies are removing unsupported constructs at about $Q$ 6– $Q$ 7, adding numeric conditions at about $Q$ 8– $Q$ 9, and adding safety guards at about $\mathcal{L}$ 0– $\mathcal{L}$ 1. The outcome is often weaker but still meaningful: this category accounts for about $\mathcal{L}$ 2– $\mathcal{L}$ 3 of repairs, while domain-restricted repairs account for about $\mathcal{L}$ 4– $\mathcal{L}$ 5, safety-only repairs for about $\mathcal{L}$ 6– $\mathcal{L}$ 7, and semantics-preserving repairs for about $\mathcal{L}$ 8– $\mathcal{L}$ 9 (Wang et al., 19 Jun 2026).

6. Conceptual synthesis, adjacent uses, and limitations

A plausible unifying reading is that KBSpec consistently externalizes knowledge and then subjects that knowledge to reusable inference. In the K-framework lineage, the externalized object is an executable semantics plus observer vocabulary; in the interactive-configuration lineage, it is an FO(.) theory and partial structures; in the 2026 framework, it is a verifier-conditioned memory of specification idioms and repair trajectories. This suggests a shared architectural principle: knowledge should be represented in a form that can be queried, propagated, repaired, or verified without rewriting the domain logic for each task (Alpuente et al., 2016, Hertum et al., 2016, Wang et al., 19 Jun 2026).

The same family resemblance appears in adjacent, more interpretive uses of the term. A specification-oriented exposition of KBLRN presents a template for a multi-feature knowledge-base representation and completion system, integrating latent, relational, and numerical features through a Product of Experts model; this is framed as guidance for a "KBSpec" design rather than as the paper’s own title or system name (Garcia-Duran et al., 2017). Likewise, a specification-oriented treatment of kernel-based conditional moment testing explicitly interprets "KBSpec" as a general framework for kernel-based specification tests centered on spectral decomposition, truncation, and signal-to-noise-based kernel selection, again as an interpretive label rather than the paper’s canonical name (Rui et al., 5 Jun 2025).

Across the main lineages, the limitations are also recurrent. The K-based systems are constrained by symbolic-execution cost, observer dependence, and the need for suitable heap abstraction or refinement (Alpuente et al., 2016, Alpuente et al., 2015). The knowledge-base paradigm trades some efficiency for maintainability because it does not rely on domain-specific propagators or custom configuration algorithms (Hertum et al., 2016). The 2026 KBSpec is tightly coupled to OpenJML’s capabilities and quirks, focuses on JML rather than a broad range of formal languages, and sometimes repairs toward weaker but verifier-friendly contracts rather than semantics-preserving ones (Wang et al., 19 Jun 2026).

Taken together, KBSpec denotes not a single algorithmic object but a specification-centered research program. Its canonical contemporary form is LLM-driven formal specification generation with an evolving knowledge base, yet its broader significance lies in a persistent design commitment: explicit knowledge representation, strong separation between representation and inference, and continuous feedback from symbolic execution, logical reasoning, or formal verification.