Optimal ConfirmValue threshold for gap triple selection in Evontree

Determine the optimal ConfirmValue threshold for identifying gap ontology triples—namely, extrapolated SubclassOf triples produced by applying ontology rule R2 to reliable triples—in the Evontree ontology rule-guided self-evolution framework, such that fine-tuning on the resulting injected data maximizes performance on medical question answering benchmarks.

Background

Evontree extracts an LLM’s implicit ontology knowledge (SynonymOf and SubclassOf relations), applies ontology rules (R1 and R2) to extrapolate additional triples, and uses a perplexity-based confidence score called ConfirmValue to assess whether the model has already internalized these facts. Triples with ConfirmValue below a threshold τ* are designated as gap triples and injected via fine-tuning to improve domain performance.

In the hyperparameter sensitivity analysis, the authors vary the ConfirmValue threshold used to select gap triples and observe performance changes. While the approach is robust to threshold variation, the authors explicitly state that the optimal threshold remains uncertain, leaving open the precise choice of ConfirmValue cutoff that yields maximal downstream gains.