Semantic Grounding Index (SGI)

• Semantic Grounding Index (SGI) is a geometric metric that quantifies context engagement in RAG systems by measuring angular distances between queries, responses, and contexts.
• SGI leverages a ratio of geodesic distances on the unit hypersphere to distinguish valid, context-grounded responses from hallucinated, semantically lazy outputs.
• Empirical benchmarks across multiple embedding models confirm SGI's efficacy, demonstrating robust discrimination and calibration for hallucination detection in production pipelines.

The Semantic Grounding Index (SGI) is a geometric metric for quantifying context engagement in retrieval-augmented generation (RAG) systems. Defined as the ratio of angular distances in embedding space between a generated response, the original question, and the retrieved context, SGI provides both theoretical and empirical guarantees for detecting hallucinated outputs. Its utility is rooted in the phenomenon of "semantic laziness:" hallucinated responses remain angularly close to the question rather than moving meaningfully toward the context. SGI offers computationally efficient, model-agnostic assessments of grounding and enables calibrated hallucination risk estimates for production RAG pipelines (Marín, 15 Dec 2025).

1. Formal Definition and Geometric Foundations

SGI operates on the unit hypersphere $S^{d-1} = \{ x \in \mathbb{R}^d : \|x\| = 1 \}$, where RAG queries, retrieved contexts, and generated responses are embedded as $\hat{q}$, $\hat{c}$, and $\hat{r}$, respectively. The natural distance metric for these normalized embeddings is the geodesic (angular) distance: $$\theta(x, y) = \arccos(\hat{x}^\top \hat{y}) \in [0, \pi]$$ For any RAG triple $(q, c, r)$, the Semantic Grounding Index is defined: $$\mathrm{SGI}(r; q, c) = \frac{\theta(r, q)}{\theta(r, c)} = \frac{\arccos(\hat{r}^\top \hat{q})}{\arccos(\hat{r}^\top \hat{c})}$$ Interpretation:

• $\mathrm{SGI} > 1$: The response is angularly closer to context than question (evidence of valid grounding).
• $\mathrm{SGI} < 1$: The response is closer to the question, indicating semantic laziness (hallucination signature).

The metric’s geometric properties are governed by the spherical triangle inequality: $$|\theta(q, c) - \theta(r, c)| \leq \theta(r, q) \leq \theta(q, c) + \theta(r, c)$$ Yielding bounds on SGI: $\hat{q}$0 As the question-context angle $\hat{q}$1 increases, the allowed range of SGI widens, providing better potential discrimination between grounded and hallucinated responses.

2. Empirical Validation and Performance Benchmarks

Empirical analyses were conducted on HaluEval ($\hat{q}$2), benchmarking SGI across five sentence-transformer embedding models:

• all-mpnet-base-v2 (768d)
• all-MiniLM-L6-v2 (384d)
• bge-base-en-v1.5 (768d)
• e5-base-v2 (768d)
• gte-base (768d)

Key findings are summarized in the table below:

Metric	Valid Responses	Hallucinated Responses	Effect Size / Correlation
Mean SGI	1.188	0.913	Cohen’s $\hat{q}$3 (large)
ROC-AUC (per model)	–	–	0.776–0.824 (mean $\hat{q}$4)
Cross-model Pearson $\hat{q}$5	–	–	$\hat{q}$6 (off-diagonal mean)
Spearman $\hat{q}$7	–	–	$\hat{q}$8

When stratifying results by question-context angle as predicted by the theoretical bounds, discriminative power increases monotonically with $\hat{q}$9:

• Low $\hat{c}$0: $\hat{c}$1, AUC=0.721
• Medium $\hat{c}$2: $\hat{c}$3, AUC=0.768
• High $\hat{c}$4: $\hat{c}$5, AUC=0.832

These results confirm that SGI’s effectiveness grows with divergent question and context embeddings.

3. Subgroup Analyses and Calibration

Subgroup analyses demonstrate robustness and highlight scenarios where SGI is particularly discriminative:

• Question length:
  • Short: $\hat{c}$6, AUC=0.812
  • Medium: $\hat{c}$7, AUC=0.781
  • Long: $\hat{c}$8, AUC=0.714
• Context length:
  • All: $\hat{c}$9–$\hat{r}$0, AUC $\hat{r}$1–$\hat{r}$2
• Response length:
  • Short: $\hat{r}$3, AUC=0.771
  • Medium: $\hat{r}$4, AUC=0.804
  • Long: $\hat{r}$5, AUC=0.893

SGI is especially effective for detecting hallucinations on long responses and short, focused questions.

Calibration analysis via reliability diagrams supports the use of SGI as a probabilistic hallucination risk estimator (Expected Calibration Error ECE=0.10)—SGI scores can be min–max normalized to estimate probability. Hallucination rates decline from ~100% in the lowest SGI decile to ~65% in the highest.

4. Distinction from Factual Accuracy

A critical negative result on TruthfulQA ($\hat{r}$6) demonstrates that SGI measures semantic (topical) engagement, not factual correctness:

• Truthful and false answers to the same question cluster with nearly identical $\hat{r}$7 (mean $\hat{r}$8)
• Cohen’s $\hat{r}$9, AUC=0.478 (worse than chance)

This outcome indicates that angular embedding geometry is sensitive to topical alignment rather than verifying truth, and cannot discriminate factuality once the topic is held constant.

5. Practical Implementation in RAG Systems

Production integration of SGI is efficient and low-overhead:

• Precompute $$\theta(x, y) = \arccos(\hat{x}^\top \hat{y}) \in [0, \pi]$$0 to estimate overall discriminative capacity; low $$\theta(x, y) = \arccos(\hat{x}^\top \hat{y}) \in [0, \pi]$$1 predicts weaker SGI signal.
• Recommended embedding models include all-MiniLM-L6-v2 for efficiency, with mpnet, bge, e5, and gte as alternatives.
• Standard workflow: For each $$\theta(x, y) = \arccos(\hat{x}^\top \hat{y}) \in [0, \pi]$$2 tuple, compute embeddings, apply L2 normalization, obtain two dot products, then two $$\theta(x, y) = \arccos(\hat{x}^\top \hat{y}) \in [0, \pi]$$3 evaluations. Add $$\theta(x, y) = \arccos(\hat{x}^\top \hat{y}) \in [0, \pi]$$4 to denominators to avoid division by zero.
• Raw SGI $$\theta(x, y) = \arccos(\hat{x}^\top \hat{y}) \in [0, \pi]$$5 flags likely hallucination; for risk stratification or probabilistic calibration, bin SGI and calibrate on held-out data.
• SGI is most impactful on long responses and short, information-dense questions—contexts where manual verification cost is highest.
• Combining SGI with complementary techniques, such as NLI-based entailment, extends coverage to both semantic disengagement and logical inconsistency.

6. The Semantic Laziness Hypothesis

SGI formalizes the "semantic laziness" hypothesis: responses flagged as hallucinated by human raters systematically display $$\theta(x, y) = \arccos(\hat{x}^\top \hat{y}) \in [0, \pi]$$6, while grounded responses exhibit higher SGI. This is captured: $$\theta(x, y) = \arccos(\hat{x}^\top \hat{y}) \in [0, \pi]$$7 This regularity, validated by large effect sizes and consistent cross-model correlations, underpins SGI’s practical and theoretical merit for detecting context disengagement in RAG systems (Marín, 15 Dec 2025).

7. Summary of LaTeX Expressions

$$\theta(x, y) = \arccos(\hat{x}^\top \hat{y}) \in [0, \pi]$$8

SGI provides a theoretically justified and empirically validated index for measuring the degree to which generated responses are semantically grounded in context in RAG systems. Its geometric formulation, strong association with manual hallucination annotations, and efficient implementation collectively substantiate its relevance for large-scale, production-level LLM pipelines (Marín, 15 Dec 2025).