Entity-Enhanced Cognitive Alignment (EECA)

• Entity-Enhanced Cognitive Alignment (EECA) is a framework that improves semantic coherence by leveraging dual hypergraph structures and staged retrieval strategies.
• It employs multi-granular supervision in LVLMs to align visual embeddings with language outputs, boosting referential reasoning and factuality.
• EECA bridges thematic and high-order entity gaps, demonstrating state-of-the-art performance in both retrieval-augmented generation and vision-language contexts.

Entity-Enhanced Cognitive Alignment (EECA) is a principled framework for improving semantic coherence and factual alignment in both retrieval-augmented generation scenarios and vision-LLM reasoning. EECA broadly addresses cognitive misalignment between model interpretation spaces by (a) leveraging high-order entity and thematic structures in symbolic data (Hu et al., 17 Nov 2025) and (b) employing multi-granularity supervision for visual token embedding within a LLM’s cognitive manifold (Zhao et al., 2024). In both contexts, EECA achieves superior alignment between input modalities and model outputs by explicit entity- and theme-level representation, staged reasoning, and matched loss functions.

1. Motivation and Problem Statement

In symbolic RAG, cognitive misalignment emerges when retrieval and response pipelines fail to preserve thematic and high-order entity relations, leading to poor generation factuality and coherence. Graph-based augmentation models predominantly capture only pairwise relationships, discarding latent cross-chunk or group semantics. In LVLMs, misalignment is quantified by the discrepancy between vision encoder outputs and the LLM’s text-based semantic space; images with ambiguous embeddings (VE-Unknown) cannot be reliably interpreted by the LLM, while rich, discriminative features (VE-Known) greatly facilitate referential reasoning.

EECA introduces mechanisms to (i) construct a theme-aligned, dual-hypergraph retrieval system in RAG, and (ii) infuse multi-granular, entity-aware supervision into visual representation learning for multimodal models. This dual approach closes alignment gaps at both the retrieval and encoding levels.

2. Formal Representation: Dual-Hypergraph and Multi-Granular Supervision

In the Cog-RAG instantiation (Hu et al., 17 Nov 2025), EECA formalizes knowledge by splitting a data corpus $\mathcal{D}$ into overlapping chunks and extracting two hypergraphs:

• Theme hypergraph $$\mathcal{G}_{\rm theme} = (V_{\rm key},\,E_{\rm theme})$$ models key entities as nodes and themes (storylines) as hyperedges. Incidence is stored in $H^{(\rm theme)}\in\{0,1\}^{n_t\times m_t}$.
• Entity hypergraph $$\mathcal{G}_{\rm entity} = (V, E_{\rm low}\cup E_{\rm high})$$ represents fine-grained entities as nodes and both pairwise ($E_{\rm low}$) and higher-order ($E_{\rm high}$) entity interactions as hyperedges, with incidence matrices $H^{(\rm low)}$ and $H^{(\rm high)}$.

In LVLM cognitive alignment (Zhao et al., 2024), multi-granular supervision is imbued via a dual-branch adapter:

• Low-resolution visual branch: Downsampled image features $X_v^L$.
• High-resolution branch: Patch-level visual tokens $X_v^H$ resampled via Perceiver modules.

Each image is annotated with coarse hierarchical labels $h_i$ and fine-grained entity tags $\{e_{i,1},...,e_{i,E_i}\}$, with entity-aware contrastive losses ($\mathcal{L}_e$) and classification losses ($\mathcal{L}_h$) aligning visual feature embeddings to LLM tokens.

3. Mechanisms: Two-Stage Retrieval and Alignment Losses

Cog-RAG employs a cognitive-inspired, two-stage retrieval:

1. Thematic Activation: Extract thematic keywords $\mathcal X_{\rm theme}$ from the query, embed all theme hyperedges and keywords, compute relevance scores $s_{\rm theme}(e_j)$ via cosine similarity, and select top-$K_t$ theme hyperedges. Diffuse to key-entity vertices $V_{\rm dif}$ within the theme hypergraph, forming an initial context $\mathcal C_{\rm theme}$ and provisional answer $\mathcal A_{\rm theme}$.
2. Theme-Aligned Entity Recall: For each entity keyword $x \in \mathcal X_{\rm entity}$, generate an alignment prompt $\eta(x)$ that fuses the provisional answer embedding and the entity keyword. Entities are scored $s_{\rm ent}(u_i)$, top-$K_e$ are selected as $V_{\rm rel}$, and one-hop entity-diffusion yields hyperedges $E_{\rm dif}$. The final answer $\mathcal A$ incorporates both global theme and local entity evidence.

EECA in LVLMs trains with composite objectives:

$$\mathcal{L} = \lambda\mathcal{L}_g + \mu_e\mathcal{L}_e + \mu_h\mathcal{L}_h$$

where $\mathcal{L}_g$ is the autoregressive generation loss, $\mathcal{L}_e$ the entity-level contrastive loss, and $\mathcal{L}_h$ the cross-entropy for hierarchical classification. Pseudocode formalizes the multi-step training pipeline for visual token extraction, entity weighting, and backpropagation.

4. Key Components and Hyperparameter Choices

In Cog-RAG (Hu et al., 17 Nov 2025):

• Chunking: Sliding window length $L$ and overlap $o$.
• Retrieval: $K_t$ (themes) and $K_e$ (entities).
• Embedding dimension $d$.
• Prompt templates: $\mathcal P_{\rm ext\_theme}$, $\mathcal P_{\rm ext\_key}$, $\mathcal P_{\rm ext\_entity}$, $\mathcal P_{\rm ext\_low}$, $\mathcal P_{\rm ext\_high}$.
• Diffusion: Depth (default one-hop, extendable).

In LVLM EECA (Zhao et al., 2024):

• Adapter architecture: Dual-branch, depth, MLP dimensions, Perceiver configuration.
• Loss weights: $\lambda$, $\mu_e$, $\mu_h$.
• Token granularity: Number of hierarchical and entity tags per sample.

Both frameworks avoid end-to-end training, and the implicit objective in retrieval maximizes semantic alignment via learned cosine similarities.

5. Quantitative Performance and Ablation Studies

Cog-RAG (Selection-Based Win Rates)

Benchmark	NaiveRAG	GraphRAG	LightRAG	HiRAG	Hyper-RAG	Cog-RAG
Mix	15.5%	41.0%	35.2%	42.0%	46.8%	84.5%
CS	7.5%	36.3%	27.5%	42.2%	45.5%	92.5%
Neurology	3.2%	33.0%	25.8%	32.5%	39.5%	96.0%

Ablation (Score-Based)

Model	Mix	CS	Neurology
Cog-RAG full	85.39	87.07	86.55
– w/o Entity Hypergraph	76.58	84.58	84.49
– w/o Theme Hypergraph	84.82	85.88	85.41
– w/o Two-Stage Retr.	84.88	86.41	86.18

Removing the entity hypergraph degrades local detail (-8.8 on Mix), while omitting the theme hypergraph impairs cross-chunk alignment (-1.19 on CS). Skipping two-stage retrieval induces further drop (-0.98 overall).

EECA in LVLMs (Landmark Recognition Accuracy)

Method	Strongly Known	Known	Accuracy
Baseline	4.12%	4.56%	8.68%
Entity Prompt	19.52%	9.32%	28.84%
EECA	8.52%	7.00%	15.52%

Ablation (HSS-50k): HR branch alone +0.04 pp; +$\mathcal{L}_e$ +0.52 pp; +$\mathcal{L}_h$ +1.12 pp.

6. Dataset Construction and Evaluation Protocols

In EECA LVLM, the Multi-Granularity Landmark Dataset (MGLD) is built on Google Landmarks v2 (4.1M images, 203k labels), annotated with GPT-4o for both coarse hierarchical categorization (e.g., "church," "mountain") and fine-grained entities (e.g., "Gothic arches"). VE-Known and VE-Unknown splits are generated using CLIP similarities ($\text{Sim}_\text{CLIP}$, Relative Similarity Rank). Evaluation involves multi-response GPT-4o scoring for four answer levels (Strongly Known, Known, Weakly Unknown, Unknown), reported as aggregate accuracy.

7. Comparative Analysis and Theoretical Contributions

Compared to baseline RAG approaches and vision-language alignment strategies, EECA demonstrates unique methodological advances:

• Cog-RAG integrates both theme and entity hypergraphs for top-down and bottom-up semantic recall, surpassing entity-only, graph-only, or single-stage retrieval models.
• LVLM EECA introduces entity-aware visual contrastive supervision and hierarchical loss, fostering robust multimodal cognitive alignment especially in ambiguous (VE-Unknown) regimes.

Conventional RAG and GraphRAG neglect high-order or global thematic links; Hyper-RAG ignores theme-driven activation. EECA/Cog-RAG unifies macro (theme) and micro (entity) reasoning stages, mirroring human cognitive structuring and yielding state-of-the-art results in factuality, coherence, and reasoning depth (Hu et al., 17 Nov 2025, Zhao et al., 2024).

A plausible implication is that further refinement of EECA via deeper multi-hop diffusion, adaptive entity granularity, and interpretable alignment dynamics may generalize its benefits to a broader class of multimodal, generative, and retrieval-centric systems.