HyperABSA: Dynamic Hypergraph for ABSA

• HyperABSA is a dynamic hypergraph framework that uses sample-specific hierarchical clustering to capture complex aspect-opinion relations in short texts.
• It employs an adaptive clustering strategy with an acceleration-fallback cutoff to optimize hyperedge granularity, yielding up to 7% F1 improvements over baselines.
• Empirical evaluations on datasets like Lap14 and Rest14 show robust performance and reduced generalization gaps, particularly in low-resource scenarios using RoBERTa embeddings.

HyperABSA is a dynamic hypergraph framework for Aspect-Based Sentiment Analysis (ABSA) that leverages sample-specific hierarchical clustering to induce higher-order relational structures among tokens in short texts. The approach replaces traditional pairwise graph modeling—in which multiple graphs carry syntactic, semantic, or external relations—with a single, adaptively constructed hypergraph tailored to each input instance. This yields more robust performance in both standard and low-resource ABSA benchmarks, particularly when paired with contextual embeddings such as those from RoBERTa. HyperABSA is characterized by its dynamic resolution of hyperedge granularity, using an acceleration-fallback cutoff to balance the expressivity and parsimony of the induced hypergraph (Kashyap et al., 18 Nov 2025).

1. Dynamic Hypergraph Construction

HyperABSA processes an input sentence of $n$ tokens through a pretrained encoder (RoBERTa-base), producing contextual token embeddings $$\mathbf{X}^{(0)} = [\mathbf{x}_1,\dots,\mathbf{x}_n] \in \mathbb{R}^{n \times d}$$. Instead of constructing multiple pairwise graphs—which introduce redundancy, increase model parameterization, and can propagate errors during graph fusion—HyperABSA creates a single sample-specific hypergraph $\mathcal{G} = (\mathcal{V},\mathcal{E})$. Nodes $\mathcal{V}=\{1,\dots,n\}$ correspond to input tokens; hyperedges $\mathcal{E}$ are variable-sized clusters of tokens, discovered automatically using hierarchical agglomerative clustering (HAC) on the encoder outputs. This construction enables HyperABSA to capture complex aspect-opinion structures and multi-token opinion spans without relying on explicit syntactic/dependency annotations.

2. Hierarchical Clustering and Acceleration-Fallback Cutoff

Hyperedge formation is guided by an adaptive clustering procedure:

• Hierarchical Agglomerative Clustering (HAC): Token embeddings are $\ell_2$-normalized and clustered using Ward’s linkage. At each agglomerative step, the merge dissimilarity $\delta_t$ is stored.
• Adaptive Cutoff Strategy: A window of recent merge dissimilarities, with window size $r=\max\{1,\lfloor\rho(n-1)\rfloor\}$ and hyperparameter $\rho\in(0,1]$, determines the set $$\delta_{\text{recent}} = [\delta_{n-r},\dots,\delta_{n-1}]$$.
• Acceleration (Elbow) Criterion: When $|\delta_{\text{recent}}|>3$, the method computes second-order finite differences $\kappa_j = \delta_{j+2} - 2\delta_{j+1} + \delta_j$ to detect the “elbow,” representing an optimal granularity point (merge level). The cutoff value $\delta_{\text{elbow}}$ is set at the maximally accelerated difference.
• Fallback Criterion: If the acceleration signal is weak or too few recents are present, a variance-based threshold is used: $$\delta_{\text{fallback}} = \bar{\delta} + \lambda \sigma_{\text{recent}}$$, with $\lambda>0$.
• Final Cutoff Selection: The minimum of $\delta_{\text{elbow}}$ and $\delta_{\text{fallback}}$ is taken when the criterion is triggered; otherwise, fallback is used. Cutting the dendrogram at $\delta_{\text{cut}}$ yields $|E|$ clusters, and the hypergraph incidence matrix $H \in \{0,1\}^{n \times |E|}$ is defined by cluster membership.

This dynamic approach adapts the granularity of the hypergraph to each sentence, improving the model’s flexibility in capturing relational structure.

3. Hypergraph Neural Network and Representation Aggregation

Node embeddings $\mathbf{X}^{(0)}$ are propagated through a Hypergraph Attention Network (HyperGAT):

• Multi-head Attention: For each head, embeddings are linearly projected and intra-hyperedge attention weights $A^{(h)}=\mathrm{softmax}(H^\top\tilde X^{(h)})$ are computed.
• Hyperedge Feature Aggregation: For each hyperedge $e$, features are aggregated as $$E_{e,:}^{(h)} = \sum_{i=1}^n H_{i,e} A^{(h)}_{e,i} X_{i,:}^{(0)}$$. All heads are concatenated: $E = \mathrm{Concat}(E^{(1)},\dots,E^{(H)})$.
• Sentence-level Representation: Mean pooling across hyperedges yields $\bar{E} = |E|^{-1} \sum_e E_{e,:}$.
• Classification: Final prediction is made via $\hat{y} = W_2 \bar{E} + b_2$, with class probabilities from $\mathrm{softmax}(\hat{y})$.

This architecture exploits the adaptive clustering to condition attention on complex, higher-order substructures, improving the aggregation of aspect and opinion information.

4. Training Objective, Optimization, and Implementation

The model is trained using cross-entropy loss with $\ell_2$ regularization: $$\mathcal{L} = -\sum_{c\in\mathcal{C}}\mathbf{1}(y=c)\log P(y=c\mid\bar E) + \beta\|\Theta\|_2^2,$$ where $\Theta$ denotes all parameters and $\beta=2\times10^{-5}$. Optimization is performed using Adam (learning rate $10^{-2}$), batch size 16, and dropout (0.2–0.3). Sentence embeddings leverage RoBERTa-base encoder outputs. Cluster granularity hyperparameters $(\rho,\lambda)$ are tuned on the development set. Experiments are conducted on a single NVIDIA RTX 4090, and clustering (HAC) is performed via Ward linkage over Euclidean distance on $\ell_2$-normalized contextual embeddings (Kashyap et al., 18 Nov 2025).

5. Empirical Evaluation and Benchmark Performance

HyperABSA demonstrates statistically significant improvements over strong graph-based ABSA systems on multiple datasets:

Dataset	Prior Graph Baseline (Acc/F1 %)	HyperABSA (Acc/F1 %)	Improvement (Acc/F1 %)
Lap14	YORO: 77.45 / 73.21	83.44 / 80.60	+5.99 / +7.39
Rest14	R-GAT: 84.64 / 77.14	87.76 / 82.76	+3.12 / +5.62
MAMS	CHGMAN: 83.23 / 82.66	84.56 / 83.74	+1.33 / +1.08

HyperABSA’s dynamic hypergraph achieves consistent 4–7% absolute F1 improvements versus graph neural baselines, especially when paired with RoBERTa (Table 1, (Kashyap et al., 18 Nov 2025)). Its generalization gap under low-resource regimes is $\lesssim$3%, while traditional multi-graph models exhibit 6–8% gaps.

6. Ablation Studies and Analytical Findings

Ablation on the acceleration-fallback cutoff strategy indicates that the dynamic cutoff delivers up to +6 F1 over fixed or fallback-only clustering (e.g., on Rest14, F1 82.76 vs. 76.89 or 74.61, Table 4). HAC with adaptive cutoff outperforms KMeans and GMM clustering on Lap14 and Rest14 (e.g., Lap14: HAC 83.44/80.60 vs. KMeans 81.71/80.67). Qualitative case studies show that HyperABSA’s clustering captures multi-token opinion expressions (positive and negative subspans) without explicit dependency guidance. This suggests improved modeling of nuanced aspect-opinion relations in short texts.

7. Limitations and Prospects for Extension

Key limitations include the lack of explicit syntactic or semantic interpretation for induced hyperedges, potentially complicating error analysis. HAC is $\mathcal{O}(n^2)$ but remains tractable for ABSA sentence lengths ($n\le50$). Overfitting is possible in very low-resource domains. Planned extensions include applying dynamic hypergraph induction to related NLP tasks (aspect extraction, question classification, dialog intent), integrating interpretability modules (post-hoc labeling of hyperedges), meta or reinforcement learning for hyperparameters, and combining hypergraph and sparse local graph modules to scale to longer contexts. These directions are expected to broaden the utility of dynamic hypergraph ABSA beyond sentiment classification for short texts (Kashyap et al., 18 Nov 2025).