Evontree: Ontology-Driven LLM Evolution

• Evontree is a framework for ontology rule–guided self-evolution of LLMs that extracts, validates, and reinforces domain-specific knowledge without relying on external labeled datasets.
• It systematically obtains reliable ontological triples via structured prompts, logical reasoning, and self-distillation, achieving up to a 3.7% accuracy boost in downstream tasks.
• Designed for data-sensitive, low-resource domains, Evontree enables LLM adaptation using expert-curated ontologies in fields like healthcare and finance.

Evontree is a framework for ontology rule–guided self-evolution of LLMs, designed to systematically extract, validate, and reinforce domain-specific knowledge encoded implicitly within LLMs using a small set of high-quality ontology rules. It targets adaptation of LLMs to data-sensitive, low-resource domains, particularly where extensive domain-specific training corpora are unavailable but expert-curated ontologies exist. Evontree operates without requiring external supervised datasets, relying instead on self-extraction and logical reasoning informed by fundamental ontological principles, and achieves measurable gains in downstream domain-specific tasks (Tu et al., 30 Oct 2025).

1. Motivation and Problem Scope

In domains such as healthcare and finance, privacy regulations and sparsity of labeled data hinder the fine-tuning of LLMs on large, high-quality, domain-specific corpora. Domain experts typically capture knowledge as formal ontology rules, which encode relationships like synonymy and subclass hierarchies. While pre-trained LLMs inherently internalize broad human and world knowledge, their ontological structures are often incomplete or inconsistent at the domain level. Evontree addresses this gap by:

• Extracting implicit ontological triples (e.g., subclass and synonym relationships) from a raw LLM via structured prompts.
• Detecting and correcting inconsistencies using two universally valid ontology rules.
• Reinforcing reliable and extrapolated knowledge directly through self-distillation and parameter-efficient adaptation.

This enables substantial accuracy improvements on domain benchmarks, with a reported increase of up to 3.7% in end-task accuracy, without using labeled external datasets.

2. Framework Architecture and Workflow

Evontree is organized into a three-stage pipeline:

Stage I: Ontology Extraction

• Manually chosen root concepts $\mathcal{C}_0$ are expanded by recursively prompting the LLM to generate subclass and synonym trees to depth $\geq 3$.
• Extracted candidate triples are of form $(c, \mathtt{SubclassOf}, c_\text{child})$ and $(c_\text{child}, \mathtt{SynonymOf}, s)$, collectively $\mathcal{T}_{\text{raw}}$.
• For each triple $t$ under a prompt $p$, a ConfirmValue score,

$$\mathtt{ConfirmValue}(t,p) = \frac{\operatorname{sign}(\mathrm{PPL}_{\rm False}-\mathrm{PPL}_{\rm True})}{\min\{\mathrm{PPL}_{\rm True},\,\mathrm{PPL}_{\rm False}\}},$$

is computed using the next-token perplexity scores on “True” and “False” statements. Triples exceeding a learned threshold $\tau^*$ (using Youden’s index) are retained as confirmed knowledge $\mathcal{T}_{\rm conf}$.

Stage II: Rule-Driven Examination and Refinement

• Two ontology rules are enforced:
  • R1 (Synonym–Subclass Transitivity):

  $\geq 3$0 - R2 (Subclass Transitivity):

  $\geq 3$1

• The rules select mutually corroborating triples and extrapolate new reliable subclass triples.

• Newly inferred triples are scored; those with $\geq 3$2 constitute the gap triples $\geq 3$3.

Stage III: Self-Distillation Fine-Tuning

• Gap triples are converted into synthetic question–answer (QA) pairs:

  • Explicit: “Is it true that $\geq 3$4 is a subclass of $\geq 3$5? Answer: True/False”
  • Implicit: “Outline the primary functions of $\geq 3$6. Hint: $\geq 3$7 is a subclass of $\geq 3$8, and $\geq 3$9 is a subclass of $(c, \mathtt{SubclassOf}, c_\text{child})$0.”
• Fine-tuning is performed on this synthetic data $(c, \mathtt{SubclassOf}, c_\text{child})$1 with LoRA adapters, optimizing cross-entropy loss over responses.

3. Formal Processes and Algorithms

The extraction and validation process is formalized as follows:

Ontology Extraction Algorithm

$(c, \mathtt{SubclassOf}, c_\text{child})$9

ConfirmValue Calculation

$(c, \mathtt{SubclassOf}, c_\text{child})$2 uses the difference in model perplexity between “True” and “False” statements to ascertain the confidence of the extracted triple, averaging or minimizing across multiple prompt paraphrases.

4. Empirical Evaluation and Results

Evontree was evaluated on three medical QA datasets:

• MedMCQA (~193K questions)
• MedQA (~11K USMLE-style questions)
• PubMedQA (1K expert-annotated test set)

Baseline models included Llama3-8B-Instruct, Med42-v2, TaxoLLaMA, and OntoTune$(c, \mathtt{SubclassOf}, c_\text{child})$3. Core evaluation metrics were zero-shot accuracy per dataset, average accuracy across all datasets, general capability retention (MMLU, ARC, TriviaQA), and safety metrics (AdvBench “raw safe” & “jailbreak safe” rates).

Model / Setting	MedQA	MedMCQA	PubMedQA	Average
Llama3-8B (raw)	51.7	51.7	70.3	57.9
TaxoLLaMA	50.5	46.1	73.4	56.7
OntoTune$(c, \mathtt{SubclassOf}, c_\text{child})$4	51.5	56.7	72.0	60.1
Llama3-8B + Evontree (mix)	51.0	57.4	74.7	61.0
Δ vs. raw	+1.0%	+5.7%	+6.1%	+3.1%
Med42-v2 (raw)	57.8	58.1	74.6	63.5
Med42-v2 + Evontree (mix)	57.2	57.7	74.9	63.3
Δ vs. raw	+2.5%	+4.3%	+4.3%	+3.7%

These results demonstrate consistent improvements by Evontree over both unmodified and supervised baselines. Scatter plots show that higher ConfirmValue scores correlate with factual accuracy, while gap triples—though less confident—still achieve approximately 75% factual validity.

5. Efficiency, Robustness, and Limitations

Evontree incurs minimal computational overhead:

• No external corpus is required; only two simple ontology rules and $(c, \mathtt{SubclassOf}, c_\text{child})$51.4K gap triples (≈0.1% of raw triples) are processed.
• Fine-tuning is performed with LoRA adapters (rank 8, $(c, \mathtt{SubclassOf}, c_\text{child})$6, dropout=0.05) over 3 epochs, using an 8GB GPU, and incurs cost similar to a small LoRA fine-tuning step.
• Model robustness is maintained: general capability degradation (MMLU/ARC) is $(c, \mathtt{SubclassOf}, c_\text{child})$70.2%; safety metrics remain stable or improve under adversarial “jailbreak” conditions.

Principal limitations include:

• The fixed ConfirmValue threshold $(c, \mathtt{SubclassOf}, c_\text{child})$8 may not generalize optimally across all domains.
• Only two relation types (Synonym, Subclass) are currently supported; handling more expressive ontological relations remains open.
• Reliance on self-confirmation may propagate existing model weaknesses in underrepresented subdomains.

6. Future Extensions and Research Directions

Key avenues identified for advancing Evontree include:

1. Adaptive or multi-objective calibration of ConfirmValue to optimize gap-triple recall and noise control.
2. Incorporation of additional ontology axioms (disjointness, equivalence classes, role restrictions).
3. Cross-model ensemble approaches: extracting and reconciling ontologies from multiple LLMs to bolster reliability and coverage.
4. Extension to alternative data-sensitive domains (e.g., law, finance) and to multimodal LLM architectures.

A plausible implication is that automating ontology-guided self-evolution via Evontree could generalize to diverse settings where high-quality ontological constraints are available but labeled data is scarce.

7. Significance and Outlook

Evontree exemplifies a paradigm in which LLMs are treated as implicit repositories of human knowledge whose internal ontological structures can be systematically exposed, scrutinized, and enhanced by a minimal set of domain-expert rules. The approach elevates LLM performance in knowledge-intensive, data-constrained domains without recourse to massive external supervision. Its strategies—ontology extraction via structured prompts, rule-based reasoning, and self-distillation—align with broader interests in factually controlled and interpretable LLM adaptation (Tu et al., 30 Oct 2025). Limitations suggest ample research opportunities in scaling relation coverage, calibrating confidence metrics, and expanding applicability to new modalities and composite model settings.