Head Information Aggregation (HIA) Overview

• HIA is a family of techniques that combine diverse neural network heads into a single representation to enhance reliability and preserve specialization.
• It is applied in multi-head attention, RLHF reward modeling, federated learning, and vision tasks using methods like entropy-guided weighting and cross-attention fusion.
• HIA addresses limitations of uniform head aggregation by leveraging structured fusion approaches, such as routing-by-agreement and interleaved head mixing, to maintain key head-specific insights.

Searching arXiv for the cited HIA-related papers and topic variants to ground the article in current arXiv records. Head Information Aggregation (HIA) denotes a family of techniques that combine information produced by multiple heads into a single representation, prediction, reward, or control signal. The “head” being aggregated varies by domain: it may be an attention head in a Transformer, a reward head corresponding to a safety rule in RLHF, a personalized model head in federated learning, or a head representation extracted from another camera view in gaze target estimation. Across these settings, the common premise is that heads are not uniformly informative, and that indiscriminate averaging, simple concatenation, or purely local use of heads can preserve noise, suppress specialization, or discard globally useful information (Li et al., 26 Mar 2025, Wang et al., 2020, Li et al., 2019, Zhou et al., 2024, Miao et al., 7 Aug 2025).

1. Conceptual scope and recurring motivation

In the cited literature, HIA is not a single standardized algorithm but a recurring design problem: how to combine head-level signals while preserving the heterogeneity that produced those signals in the first place. In multi-head attention, the standard practice of concatenation followed by a linear transformation is described as potentially failing to “fully exploit the expressiveness of multi-head attention,” because it does not explicitly model relationships among heads (Li et al., 2019). In textual sequence matching, simple matching between pairwise sequence representations is described as bringing “unnecessary noise,” whereas head-wise matching preserves the factorization already present in the Transformer (Wang et al., 2020). In RLHF safety alignment, equal or random weighting of rule-specific reward heads ignores the fact that some rules are less reliable than others for identifying human-preferred responses (Li et al., 26 Mar 2025).

A parallel motivation appears in interpretability and systems work. In diffusion-based visual interpretation, averaging cross-attention maps across all heads and timesteps, as in DAAM, can blur concept-specific signals that are concentrated in a subset of heads (Park et al., 7 Apr 2026). In long-context language modeling, HydraHead reports that layers exhibit block-wise functional similarity while individual heads within the same layer display distinct functional specialization, implying that the head axis can be a more precise granularity than the layer axis for mixing heterogeneous mechanisms (Tan et al., 18 Jun 2026). In long autoregressive video generation, Head Forcing identifies local heads, anchor heads, and memory heads, each associated with a different temporal role and therefore a different optimal KV-cache policy (Tian et al., 14 May 2026).

Taken together, these works treat aggregation not merely as a dimensionality-reduction step but as a problem of reliability estimation, causal importance, geometric alignment, or functional specialization. A plausible implication is that HIA is best understood as a structured fusion problem rather than a uniform pooling problem.

2. Canonical mathematical forms

Representative HIA mechanisms differ substantially in what is being combined and how the aggregation weights are determined, but several recurring forms appear: entropy-guided weighting, cross-attention fusion, element-wise interpolation, and cross-head linear mixing.

Setting	Representative formula	Aggregation role
RLHF multi-head reward aggregation	$$w_k = \frac{\exp(-H(\psi_k)/\tau)}{\sum_{j=1}^R \exp(-H(\psi_j)/\tau)}$$, $\phi(x,y)=\sum_{k=1}^R w_k \psi_k(x,y)$	Downweights high-entropy reward heads
Personalized federated learning	$$\hat{h}_i^{t}:=h_i^{t-1}+(h^t-h_i^{t-1})\odot W_i^t$$	Interpolates local and global heads element-wise
Multi-view gaze target estimation	$$\tilde{\mathbf{F}}^h_1=\mathbf{F}^h_1+\text{CrossAtt}(\mathbf{Q}^h_1,\mathbf{K}^h_1,\mathbf{V}^h_1)$$	Fuses head features across views
Interleaved Head Attention	$$\widetilde{\mathbf{Q}}_{h,j}=\sum_{m=1}^H \alpha^Q_{m,h,j}(\mathbf{X}\mathbf{W}_Q^{(m)})$$	Mixes Q/K/V information across heads

These forms correspond to different operational interpretations of “aggregation.” In ENCORE, aggregation is a weighted linear combination over rule-specific reward heads, with the weights determined by empirical rating entropy (Li et al., 26 Mar 2025). In FedAH, the aggregation target is the personalized classifier head itself, with one learnable interpolation weight per head parameter (Zhou et al., 2024). In multi-view gaze estimation, HIA is a cross-attention module that uses head features from a reference camera view together with the relative camera rotation $\mathbf{R}_{21}=\mathbf{R}_1\mathbf{R}_2^{-1}$ to enhance the primary-view head embedding (Miao et al., 7 Aug 2025). In Interleaved Head Attention, aggregation occurs before attention output collapse: pseudo-queries, pseudo-keys, and pseudo-values are learned linear combinations of original heads, enabling communication among heads during the attention computation rather than only after it (Duvvuri et al., 24 Feb 2026).

A notable structural distinction follows from this comparison. Some HIA methods are post hoc, in the sense that they combine already computed head outputs; others alter the computation graph so that heads interact before the final output is formed. This suggests a broad taxonomy of HIA into output-level aggregation, parameter-level aggregation, feature-level cross-view aggregation, and in-attention cross-head mixing.

3. Reward modeling and RLHF

In RLHF safety alignment, HIA is formulated explicitly as the problem of composing multiple reward heads, each associated with a distinct safety rule or attribute, into a single scalar reward. “Multi-head Reward Aggregation Guided by Entropy” introduces ENCORE, an entropy-guided, training-free, and interpretable aggregation method built on the empirical observation that safety rules with higher rating entropy are generally less reliable in identifying responses preferred by humans (Li et al., 26 Mar 2025).

The method first estimates the discrete entropy of each rule head,

$$H(\psi_k)=-\sum_{z\in \text{supp}(\psi_k)} p_{\psi_k}(z)\log p_{\psi_k}(z),$$

and then assigns a softmax weight penalized by that entropy,

$$w_k=\frac{\exp(-H(\psi_k)/\tau)}{\sum_{j=1}^R \exp(-H(\psi_j)/\tau)}.$$

The aggregated reward is

$\phi(x,y)=\sum_{k=1}^R w_k\psi_k(x,y).$

The temperature parameter $\tau$ controls the sharpness of selection: as $\phi(x,y)=\sum_{k=1}^R w_k \psi_k(x,y)$0, the method recovers uniform weighting; as $\phi(x,y)=\sum_{k=1}^R w_k \psi_k(x,y)$1, the weighting approaches a top-$\phi(x,y)=\sum_{k=1}^R w_k \psi_k(x,y)$2 regime in which only the lowest-entropy rules receive weight (Li et al., 26 Mar 2025).

The same work provides a Bradley–Terry justification for this weighting scheme. With pairwise preference probability modeled by a Bradley–Terry objective and with the aggregated margin

$\phi(x,y)=\sum_{k=1}^R w_k \psi_k(x,y)$3

the per-sample loss derivative with respect to $\phi(x,y)=\sum_{k=1}^R w_k \psi_k(x,y)$4 is proportional to the headwise score difference. If a rule head is maximally entropic and therefore nearly random over the dataset, then

$\phi(x,y)=\sum_{k=1}^R w_k \psi_k(x,y)$5

so its gradient contribution vanishes and it naturally receives negligible weight under Bradley–Terry optimization. The paper states this formally in Theorem 1 (Li et al., 26 Mar 2025).

On RewardBench safety tasks with a Llama3.1-8B base model, ENCORE achieves the highest overall safety accuracy among the evaluated multi-head aggregation schemes, with 88.5% versus 84.2% for random weighting, 85.5% for uniform weighting, and 86.0% for Mixture-of-Experts; the cited single-head or LLM-based judging baselines range from 64.0% to 81.6% depending on the model (Li et al., 26 Mar 2025). The reported ablations also show that selecting low-entropy rules is better than random selection, but that full entropy-weighted aggregation is best. Within this line of work, HIA is therefore not merely a fusion heuristic: it is a rule-reliability estimator tied to preference learnability.

4. Transformer representation matching and cross-head interaction

A second major strand of HIA concerns the aggregation of attention-head representations in Transformers. “Multi-level Head-wise Match and Aggregation in Transformer for Textual Sequence Matching” argues that simple sequence-level matching of precomputed Transformer representations can dilute informative head-level signals and introduce noise. The proposed alternative computes a head-wise match representation

$\phi(x,y)=\sum_{k=1}^R w_k \psi_k(x,y)$6

for corresponding heads of the source and target sequences, followed by aggregation across heads and then across layers (Wang et al., 2020).

Three match functions are considered: cosine similarity, bilinear matching, and an element-wise matching function based on concatenating $\phi(x,y)=\sum_{k=1}^R w_k \psi_k(x,y)$7, $\phi(x,y)=\sum_{k=1}^R w_k \psi_k(x,y)$8, their difference, and their product before a nonlinear projection. For head aggregation, the paper studies both max pooling and concatenation, and for multi-level aggregation it computes a per-layer head-wise aggregate $\phi(x,y)=\sum_{k=1}^R w_k \psi_k(x,y)$9, transforms it to $$\hat{h}_i^{t}:=h_i^{t-1}+(h^t-h_i^{t-1})\odot W_i^t$$0, and aggregates over layers to obtain the final sequence-pair representation (Wang et al., 2020). The hierarchical structure is important: the reported SNLI ablation shows Match_element + Agg_maxpooling at 88.1, compared with 88.0 for Match_element + Agg_concat and 87.6 for Match_element + Agg_concat without hierarchical aggregation. On benchmark tasks, classic match yields 83.2 on SNLI, 73.8 on MNLI-match, 74.2 on MNLI-mismatch, 63.3 on QQP, and 61.5 on SQuAD-binary, whereas the multi-level head-wise model reaches 88.1, 79.2, 79.3, 69.0, and 62.9, respectively (Wang et al., 2020).

Whereas the preceding work aggregates matched head outputs, “Information Aggregation for Multi-Head Attention with Routing-by-Agreement” modifies the aggregation mechanism itself. It recasts head aggregation as assigning “parts” to “wholes”: each head output is transformed into an input capsule, each input capsule produces vote vectors for output capsules, and routing iteratively updates assignment probabilities according to agreement (Li et al., 2019). With simple routing, the assignments are obtained by a softmax over logits and the logits are incremented by the dot-product agreement between vote vectors and current output capsules. With EM routing, the output capsules are treated as Gaussian mixture components, with means and variances estimated from the head votes. The result is an aggregation mechanism that is nonlinear, dynamic, and explicit about head relationships.

The reported empirical impact is twofold. On linguistic probing tasks, EM routing outperforms the baseline on most syntactic and semantic probes; for example, TreeDepth improves from 44.48% to 47.72% and Tense from 84.20% to 88.17%. On WMT14 En→De machine translation, Transformer-Base improves from 27.31 BLEU to 28.26, and Transformer-Big from 28.58 to 28.96; the gains are described as statistically significant (Li et al., 2019).

A third development pushes HIA inside the attention operator. “Interleaved Head Attention” observes that standard multi-head attention produces $$\hat{h}_i^{t}:=h_i^{t-1}+(h^t-h_i^{t-1})\odot W_i^t$$1 independent attention matrices with no communication between heads during attention computation, which is limiting for multi-step reasoning. IHA therefore constructs $$\hat{h}_i^{t}:=h_i^{t-1}+(h^t-h_i^{t-1})\odot W_i^t$$2 pseudo-heads per head, typically with $$\hat{h}_i^{t}:=h_i^{t-1}+(h^t-h_i^{t-1})\odot W_i^t$$3, where each pseudo query, key, and value is a learned linear combination of all original heads. This yields up to $$\hat{h}_i^{t}:=h_i^{t-1}+(h^t-h_i^{t-1})\odot W_i^t$$4 attention patterns per head with extra parameters scaling as $$\hat{h}_i^{t}:=h_i^{t-1}+(h^t-h_i^{t-1})\odot W_i^t$$5 (Duvvuri et al., 24 Feb 2026). The paper proves that IHA strictly generalizes standard MHA and gives parameter-efficiency results on synthetic tasks. Empirically, it improves Multi-Key retrieval on RULER by 10–20% from 4k to 16k, improves GSM8K by 5.8% after fine-tuning for reasoning, and improves MATH-500 by 2.8% under Majority Vote (Duvvuri et al., 24 Feb 2026).

Across these works, HIA in Transformers spans a spectrum from post-attention combination to routing-based structured fusion to full cross-head mixing of Q/K/V representations. The common target is the same: retaining complementary head specialization without collapsing it prematurely.

5. Vision, diffusion interpretability, and multi-view gaze estimation

In diffusion-based visual interpretation, HIA appears as selective aggregation of attention maps. “Selective Aggregation of Attention Maps Improves Diffusion-Based Visual Interpretation” contrasts DAAM’s all-head average,

$$\hat{h}_i^{t}:=h_i^{t-1}+(h^t-h_i^{t-1})\odot W_i^t$$6

with a selective alternative that averages only over a subset of heads most relevant to the target concept,

$$\hat{h}_i^{t}:=h_i^{t-1}+(h^t-h_i^{t-1})\odot W_i^t$$7

Head selection is based on a Head Relevance Vector method; in Stable Diffusion v1.4, the paper typically selects 30 out of 128 heads, corresponding to roughly 20–25% of the total (Park et al., 7 Apr 2026).

The reported mean IoU results on the “Animals” category show consistent improvements over DAAM. At threshold 0.4, DAAM achieves 0.7540 while selective aggregation reaches 0.7765; at 0.3, the values are 0.7490 and 0.7698; at 0.5, 0.6261 and 0.6785 (Park et al., 7 Apr 2026). Using the least relevant 30 heads performs much worse, with 0.6172 at threshold 0.4 and 0.4649 at 0.5, which the paper uses as evidence that head selection isolates concept-specific information rather than simply reducing noise through sparsity (Park et al., 7 Apr 2026). The same study also uses selective aggregation to diagnose prompt ambiguity, as in the case of the word “mouse,” where different head subsets correspond to different semantic senses.

In gaze target estimation, HIA is instead a cross-view fusion module. “Multi-view Gaze Target Estimation” processes a pair of camera views and includes a Head Information Aggregation module that uses head crops from both views together with the relative camera rotation to enhance the primary-view head representation (Miao et al., 7 Aug 2025). If $$\hat{h}_i^{t}:=h_i^{t-1}+(h^t-h_i^{t-1})\odot W_i^t$$8 and $$\hat{h}_i^{t}:=h_i^{t-1}+(h^t-h_i^{t-1})\odot W_i^t$$9 denote head features extracted by a ResNet-18 backbone, then the HIA update is

$$\tilde{\mathbf{F}}^h_1=\mathbf{F}^h_1+\text{CrossAtt}(\mathbf{Q}^h_1,\mathbf{K}^h_1,\mathbf{V}^h_1)$$0

with

$$\tilde{\mathbf{F}}^h_1=\mathbf{F}^h_1+\text{CrossAtt}(\mathbf{Q}^h_1,\mathbf{K}^h_1,\mathbf{V}^h_1)$$1

and $$\tilde{\mathbf{F}}^h_1=\mathbf{F}^h_1+\text{CrossAtt}(\mathbf{Q}^h_1,\mathbf{K}^h_1,\mathbf{V}^h_1)$$2. The aggregated tokens are then average-pooled to produce the final head embedding (Miao et al., 7 Aug 2025).

The paper reports that adding HIA to a single-view baseline reduces gaze target estimation distance error from 0.151 to 0.135 when the head is visible in the reference view and the target is in view; adding HIA together with UGS reduces it further to 0.130, and HIA + UGS + ESA yields 0.129 (Miao et al., 7 Aug 2025). The same report states that the mean angular error in gaze vector estimation decreases substantially with HIA, “e.g., from ~27° (no HIA) → ~21° (with HIA) in head-visible cases” (Miao et al., 7 Aug 2025). Here, HIA is not a generic pooling layer but a geometry-aware cross-view transfer mechanism that allows one view to compensate for head occlusion or weak pose cues in another.

These two vision settings are structurally different—one aggregates attention maps for interpretability, the other aggregates head features for prediction—but both reject all-head or single-view uniformity in favor of head relevance and alignment.

6. Personalized, long-context, and head-specialized systems

HIA also appears in system-level settings where the goal is not only better prediction but better allocation of global information, computation, or memory. In personalized federated learning, “FedAH: Aggregated Head for Personalized Federated Learning” observes that personalized-head methods keep the head local and therefore prevent the model from learning global knowledge in the head. FedAH addresses this through an Aggregated Head initialized at each round by element-wise interpolation between the previous local head and the current global head,

$$\tilde{\mathbf{F}}^h_1=\mathbf{F}^h_1+\text{CrossAtt}(\mathbf{Q}^h_1,\mathbf{K}^h_1,\mathbf{V}^h_1)$$3

where $$\tilde{\mathbf{F}}^h_1=\mathbf{F}^h_1+\text{CrossAtt}(\mathbf{Q}^h_1,\mathbf{K}^h_1,\mathbf{V}^h_1)$$4 is learned by gradient descent on local data while the other parameters are frozen (Zhou et al., 2024). The resulting local head is then fine-tuned, followed by feature-extractor adaptation, and the server applies FedAvg-style aggregation over uploaded client models. On five benchmark datasets, FedAH is reported to outperform ten state-of-the-art FL methods by up to 2.87% in test accuracy, while maintaining its advantage under client dropout (Zhou et al., 2024).

In long-context language modeling, HydraHead starts from the observation that head-level contributions are highly diverse. Among 448 query heads in Qwen3-1.7B, only about 6.5% are crucial for retrieval, and the per-layer Gini coefficient of head importance averages 0.622 (Tan et al., 18 Jun 2026). The method therefore hybridizes Full Attention and Linear Attention along the head axis rather than the layer axis, assigning Full Attention only to the top-ranked heads and using a scale-normalized fusion module to reconcile the distributional gap between FA and LA head outputs. Under a unified training setup, it matches a 3:1 layer-wise hybrid’s long-context performance at a 7:1 LA-to-FA ratio, and when trained on only 15B tokens it achieves over 69% improvement over the baseline at 512K context length (Tan et al., 18 Jun 2026). This is not HIA in the narrow sense of averaging or linear combination, but it is a closely related head-aware allocation strategy grounded in interpretability.

A similar principle appears in autoregressive video generation. Head Forcing identifies three stable head categories in AR video diffusion transformers: local heads, anchor heads, and memory heads. Local heads attend to the current and neighboring frames; anchor heads focus on the first-frame sink bucket; memory heads distribute attention broadly across the temporal history (Tian et al., 14 May 2026). The method assigns each head type a distinct KV-cache strategy and adds head-wise RoPE re-encoding so that temporal relative positions remain within the pretrained range. Without additional training, it extends generation from 5 seconds to minute-level duration. For 60-second generation, the reported Dynamic Degree is 41.37 for Head Forcing versus 31.92 for Self Forcing, while the method also uses less total KV cache than uniform caching baselines, with 3096 versus 7560 frame-slots in the cited comparison (Tian et al., 14 May 2026).

A plausible implication of these works is that once head heterogeneity is established empirically, aggregation can generalize into head-specific scheduling, mechanism assignment, or memory management. In this broader systems sense, HIA becomes a principle for exploiting non-uniform head utility rather than merely combining head outputs.

7. Terminological boundaries and recurrent misconceptions

The acronym “HIA” is not unique to Head Information Aggregation. In wireless communications, it also denotes “hierarchical information accessibility,” a physical-layer security model for downlink MU-MIMO in which decoding feasibility is determined by user priority. That literature studies secrecy-rate maximization under layered access constraints and introduces GPI-HIA, a generalized power iteration method for a functional eigenvalue problem (Lee et al., 2021). Despite the acronym overlap, this is a distinct concept from head-level aggregation in neural models.

Within machine learning, a recurring misconception is that all heads should be treated symmetrically. The surveyed papers repeatedly report the opposite. ENCORE shows that high-rating-entropy rule heads are less reliable and should be downweighted (Li et al., 26 Mar 2025). Selective diffusion-map aggregation shows that the most relevant heads localize concepts more accurately than the least relevant heads and that averaging all heads can mask prompt ambiguity (Park et al., 7 Apr 2026). HydraHead reports that only a sparse subset of heads is retrieval-critical (Tan et al., 18 Jun 2026). Head Forcing finds stable role specialization into local, anchor, and memory heads, with each category benefiting from a different cache policy (Tian et al., 14 May 2026). In textual sequence matching, max pooling is effective partly because it naturally suppresses matches from heads that attend to stopwords or punctuation (Wang et al., 2020).

A second misconception is that HIA is equivalent to simple concatenation. The routing-by-agreement line explicitly challenges that assumption by arguing that concatenation followed by a linear map may not capture the relationships among head outputs (Li et al., 2019). Interleaved Head Attention extends the challenge further by showing that if head isolation during attention computation is the bottleneck, then aggregation after attention is structurally insufficient; head interaction must occur in Q/K/V space (Duvvuri et al., 24 Feb 2026).

These boundary cases indicate that HIA is best treated as a general research theme centered on how to combine non-uniform head-level signals without erasing the specialization that makes them useful. A plausible implication is that future work will continue to move from uniform head treatment toward explicit head scoring, causal importance estimation, geometry-aware alignment, and mechanism-specific fusion.