Human-Aware Mixture-of-Experts (MoE)

• Human-aware MoE is a family of architectures where expert decomposition is aligned with human cues, rules, or anatomical structures rather than latent clusters.
• It employs techniques such as preferential gating, semantic and cross-attention routing, and task-specific labeling to integrate human guidelines for enhanced interpretability.
• Empirical results across clinical, perceptual, and multi-task applications show that aligning model specialization with human-relevant structures improves both accuracy and transparency.

Human-Aware Mixture-of-Experts (MoE) denotes a family of MoE architectures in which expert decomposition, routing, supervision, or interpretation is tied to human-relevant structure rather than treated as a purely latent partition of parameter space. In the literature represented here, that structure appears in several distinct forms: explicit reliance on human guidelines in decision making, human-centric perceptual variables such as gaze targets, body-part-aware specialization for subtle social motions, task- and style-labeled experts for high-stakes instruction following, and semantically or socially informed priors that bias expert selection (Pradier et al., 2021). Across these variants, the common theme is that the MoE mechanism is used not only for conditional computation, but also for aligning specialization with human rules, human anatomy, human behavior, or human-interpretable task structure.

1. Conceptual scope and defining variants

A narrow definition of human-aware MoE is given by Preferential Mixture-of-Experts, where one expert is explicitly a human expert $g$, the other is a trainable ML expert $f_\theta$, and the gating function $\rho_w(x)$ is trained to maximize the use of human guidelines subject to a performance constraint relative to an unconstrained MoE baseline (Pradier et al., 2021). In this setting, human awareness is not metaphorical: the architecture directly encodes the question of when to follow or override human rules. The predictive distribution is

$$\hat{y}_{\theta,w}(x) = \begin{cases} (1 - \rho_w(x))\, f_\theta(x) + \rho_w(x)\, g(x) & \text{if } g(x) \neq -1, \ f_\theta(x) & \text{if } g(x) = -1. \end{cases}$$

The model is therefore human-aware through a hard preference over experts rather than through post hoc interpretation.

A broader definition appears in perception models whose expert structure is organized around human cues. GazeMoE treats gaze target estimation as a human-aware problem because it predicts where a person is attending and whether the target is in-frame or out-of-frame, which is described as core for joint attention, intention prediction, engagement estimation, and social signal interpretation (Dai et al., 6 Mar 2026). B-MoE makes the human body itself the organizing principle of the expert set: experts are bound to head, body, upper limbs, and lower limbs, and routing is conditioned on inter-region relationships for micro-action recognition (Poddar et al., 25 Mar 2026). In both cases, the MoE is human-aware because experts are aligned with human-centric perceptual variables rather than arbitrary latent clusters.

A third variant is task-aware and instruction-aware specialization. AT-MoE constructs task-specific LoRA experts, a general-purpose LoRA expert, and a layer-wise adaptive grouped routing module over expert groups such as function, domain knowledge, and style (Li et al., 2024). This is not a human-in-the-loop model in the clinical-rule sense, but it is human-aware in that experts correspond to human-understandable task categories and routing over those categories is intended to be controllable and interpretable. T-REX is adjacent to this line: its core contribution is semantic-aware routing via clustering-derived intuition vectors, and the paper explicitly describes how this mechanism could be extended toward human-aware or task-aware priors by replacing semantic clusters with user/task signals (Zhang et al., 2024).

This suggests that “human-aware MoE” is best treated as an umbrella category rather than a single architecture. The literature spans at least four recurring formulations: human-rule-preferential gating, human-centric perception MoE, body- or cue-structured expert decomposition, and human-interpretable task- or style-aware routing.

2. Human structure as an organizing principle for experts

One major design axis is whether experts are semantically grounded before routing is learned. In B-MoE, grounding is explicit: each expert specializes in a distinct body region and is instantiated with a Macro–Micro Motion Encoder (M3E), while SAPIENS segmentation is used to construct region-specific masked crops $R_k$ (Poddar et al., 25 Mar 2026). The expert feature extraction is

$$z_k = \mathrm{M3E}_k\big(\mathrm{VideoMAE}(R_k)\big),$$

and the paper reports that removing any one expert degrades performance, with the head expert producing the largest drop on MA-52. This anchors specialization in human anatomy rather than in emergent expert identities.

GazeMoE uses a less explicit but still human-centric decomposition. Its routed experts are not hard-wired to eyes, head, gestures, or context, yet the design choice $N=4$ is explicitly motivated by those four principal human-centric cues (Dai et al., 6 Mar 2026). The MoE feed-forward layer contains one shared expert and four routed experts, with top-$K$ sparse routing using $K=2$. The final MoE output is

$x' = x_{shared} + x_{routed}.$

Here specialization is implicit rather than anatomically enforced, but the architecture is designed around the observation that real scenes contain missing or unreliable cues such as occluded eyes or extreme head pose.

AT-MoE extends semantic grounding to instruction following by defining experts at the level of task/domain/style. It describes three broad kinds of experts in a medical setting: “case generation, pharmacy prescription, triage, and guidance”; domain-knowledge experts such as “surgery, radiology, pathology”; and style experts such as “clear conclusions” or “rigorous” (Li et al., 2024). Because each expert is a separate LoRA trained on task-specific data, expert identities are labeled and documentable. This is a direct contrast with MoE systems in which expert specialization is emergent but opaque.

T-REX takes a different route: experts are ultra-lightweight rank-1 LoRA factors,

$f_\theta$0

and can be mixed via a router into an effective low-rank update $f_\theta$1 (Zhang et al., 2024). Its human relevance arises less from fixed expert identities than from the paper’s observation that semantic clustering of training embeddings provides a more useful routing prior than human-annotated task categories. For Llama 2 7B, the reported average accuracy is 79.90% with no router reference, 77.40% with task category embedding, and 81.02% with “Intuition.” The paper’s interpretation is that human task labels do not align with the latent semantic organization of the data, whereas cluster-derived priors do.

A plausible implication is that human-aware expert design does not require all experts to be hand-labeled. Some architectures impose human semantics directly on the expert set, while others align routing with human-relevant structure only at the level of priors, constraints, or interpretability.

3. Routing mechanisms and human-aware gating

Routing is the central locus of human awareness in these systems. Preferential MoE defines perhaps the clearest human-aware gate: the gating function is optimized to maximize reliance on the human expert, using the objective

$f_\theta$2

subject to the constraint

$f_\theta$3

where $f_\theta$4 is the unconstrained MoE optimum (Pradier et al., 2021). Human coverage is quantified by soft coverage $f_\theta$5 and hard coverage $f_\theta$6. The gate is thus interpretable not only as a routing function but as a quantitative map of when the model trusts human rules.

T-REX uses soft routing and augments the gate with a semantic prior derived from clustering. The router is parameterized as

$f_\theta$7

and the prior $f_\theta$8 is defined by cosine similarities between the sample embedding and $f_\theta$9-means centroids (Zhang et al., 2024). The resulting gate is

$\rho_w(x)$0

The paper explicitly states that this additive prior could be generalized to human/task information, for example by defining $\rho_w(x)$1 over user or task embeddings. This is one of the clearest bridges from semantic-aware to human-aware MoE: the architecture remains unchanged, while the prior becomes human-conditioned.

AT-MoE introduces grouped routing over expert groups rather than a flat expert pool. Group-level weights are obtained from $\rho_w(x)$2, within-group weights from $\rho_w(x)$3, and the fused LoRA update is a weighted mixture over expert groups and experts within groups (Li et al., 2024). The grouped router is described as first performing overall weight allocation from the dimension of the expert group and then conducting local weight normalization adjustments within the group. Because different transformer layers have separate routing modules, the design is explicitly layer-wise adaptive and intended to reflect that lower layers emphasize foundational knowledge while higher layers emphasize formal or functional features.

B-MoE uses cross-attention routing rather than a scalar gate. Given a global semantic embedding $\rho_w(x)$4 and stacked expert embeddings $\rho_w(x)$5, the fused semantic representation is

$\rho_w(x)$6

which can be interpreted as an MoE with soft expert weights $\rho_w(x)$7 over body-region experts (Poddar et al., 25 Mar 2026). The cross-attention heatmap is reported to show head classes activating the head expert, upper-limb classes activating the upper-limb expert, and joint actions activating multiple experts simultaneously.

ERMoE replaces the conventional router MLP with a geometric routing score tied directly to expert structure. For expert $\rho_w(x)$8, token $\rho_w(x)$9, and attention-weighted context $$\hat{y}_{\theta,w}(x) = \begin{cases} (1 - \rho_w(x))\, f_\theta(x) + \rho_w(x)\, g(x) & \text{if } g(x) \neq -1, \ f_\theta(x) & \text{if } g(x) = -1. \end{cases}$$0, the Eigenbasis Score is

$$\hat{y}_{\theta,w}(x) = \begin{cases} (1 - \rho_w(x))\, f_\theta(x) + \rho_w(x)\, g(x) & \text{if } g(x) \neq -1, \ f_\theta(x) & \text{if } g(x) = -1. \end{cases}$$1

where $$\hat{y}_{\theta,w}(x) = \begin{cases} (1 - \rho_w(x))\, f_\theta(x) + \rho_w(x)\, g(x) & \text{if } g(x) \neq -1, \ f_\theta(x) & \text{if } g(x) = -1. \end{cases}$$2 and $$\hat{y}_{\theta,w}(x) = \begin{cases} (1 - \rho_w(x))\, f_\theta(x) + \rho_w(x)\, g(x) & \text{if } g(x) \neq -1, \ f_\theta(x) & \text{if } g(x) = -1. \end{cases}$$3 are projections into the expert’s learned orthonormal basis (Cheng et al., 14 Nov 2025). Routing is then thresholded and top-$$\hat{y}_{\theta,w}(x) = \begin{cases} (1 - \rho_w(x))\, f_\theta(x) + \rho_w(x)\, g(x) & \text{if } g(x) \neq -1, \ f_\theta(x) & \text{if } g(x) = -1. \end{cases}$$4. This directly ties token assignment to the expert’s representation space, which the paper presents as a route to stable routing and interpretable specialization without explicit balancing losses.

These mechanisms differ substantially, but they share a common pattern: routing is no longer only an efficiency device. It is the place where human rules, human anatomy, semantic priors, task plans, or expert geometry enter the model’s conditional computation.

4. Interpretability, controllability, and internal analysis

Human-aware MoE is often motivated not only by accuracy but by auditable specialization. Preferential MoE emphasizes this most directly. Its gating function is typically an $$\hat{y}_{\theta,w}(x) = \begin{cases} (1 - \rho_w(x))\, f_\theta(x) + \rho_w(x)\, g(x) & \text{if } g(x) \neq -1, \ f_\theta(x) & \text{if } g(x) = -1. \end{cases}$$5-regularized logistic regression, and the paper reports inspection of the learned weights $$\hat{y}_{\theta,w}(x) = \begin{cases} (1 - \rho_w(x))\, f_\theta(x) + \rho_w(x)\, g(x) & \text{if } g(x) \neq -1, \ f_\theta(x) & \text{if } g(x) = -1. \end{cases}$$6 to identify which features push the model toward human rules and which indicate the guideline should be overridden (Pradier et al., 2021). In HIV therapy prediction, the preferential gate identifies more emphasis on comorbidities and side effects than the standard MoE gate; in MDD prescription prediction, cardiovascular-risk-related features appear with positive association to human rules. The model therefore exposes an interpretable partition of the input space into “follow human” and “use ML.”

AT-MoE builds interpretability through task-labeled experts and structured routing. Routing weights can be visualized per instruction at both the group and within-group levels, making it possible to trace whether an instruction engaged domain experts, functional experts, or style experts (Li et al., 2024). The paper frames this as credibility, controllability, and interpretability in settings such as medicine, where a single instruction may contain multiple intents.

ERMoE advances interpretability by coupling the router to expert geometry. Each expert weight is reparameterized as

$$\hat{y}_{\theta,w}(x) = \begin{cases} (1 - \rho_w(x))\, f_\theta(x) + \rho_w(x)\, g(x) & \text{if } g(x) \neq -1, \ f_\theta(x) & \text{if } g(x) = -1. \end{cases}$$7

with orthonormality regularization on $$\hat{y}_{\theta,w}(x) = \begin{cases} (1 - \rho_w(x))\, f_\theta(x) + \rho_w(x)\, g(x) & \text{if } g(x) \neq -1, \ f_\theta(x) & \text{if } g(x) = -1. \end{cases}$$8 and $$\hat{y}_{\theta,w}(x) = \begin{cases} (1 - \rho_w(x))\, f_\theta(x) + \rho_w(x)\, g(x) & \text{if } g(x) \neq -1, \ f_\theta(x) & \text{if } g(x) = -1. \end{cases}$$9 (Cheng et al., 14 Nov 2025). Because the router uses the same basis $R_k$0, routing explanations can be phrased as geometric alignment between a token and an expert’s learned subspace. In the 3D MRI variant ERMoE-ba, region experts for white matter, gray matter, and CSF receive the highest Eigenbasis Scores on region-isolated inputs by late training, which the paper presents as anatomically interpretable expert specialization.

Beyond individual architectures, "Beyond Benchmarks: Understanding Mixture-of-Experts Models through Internal Mechanisms" develops a mechanism-level analysis toolkit for MoE systems (Ying et al., 28 Sep 2025). Its Model Utilization Index (MUI) measures the fraction of all possible MoE neurons that are ever among the top 0.1% contributors for a task dataset. It further defines key experts as those active in at least 60% of samples. The reported findings are that neuron utilization decreases as models evolve, training shows an Accumulating-to-Evolving trajectory, task completion emerges from collaborative contributions of multiple experts, shared experts drive concentration, and neuron-level activation patterns provide a proxy for data diversity. This reframes interpretability from “what expert fired” to “how much of the model’s internal capacity was recruited and how that recruitment changes across training and tasks.”

A common misconception is that interpretability in MoE is exhausted by reading router probabilities. The cited work suggests a broader view: expert labels, gating sparsity, geometric alignment, neuron-level contribution scores, and coverage metrics all provide distinct but complementary forms of human legibility.

5. Empirical domains and representative results

Human-aware MoE has been evaluated in clinical decision support, gaze estimation, micro-action recognition, multi-task recommendation, and multi-task LLM finetuning. Preferential MoE is evaluated on HIV therapy outcome prediction and antipsychotic prescription in major depressive disorder (Pradier et al., 2021). On the HIV task, ML-only achieves AUC $R_k$1, standard MoE achieves AUC $R_k$2 with soft coverage $R_k$3, and Preferential MoE achieves AUC $R_k$4 with soft coverage $R_k$5–$R_k$6. On the MDD task, ML-only gives AUC $R_k$7, standard MoE AUC $R_k$8 with soft coverage $R_k$9, and Preferential MoE AUC $$z_k = \mathrm{M3E}_k\big(\mathrm{VideoMAE}(R_k)\big),$$0 with soft coverage $$z_k = \mathrm{M3E}_k\big(\mathrm{VideoMAE}(R_k)\big),$$1–$$z_k = \mathrm{M3E}_k\big(\mathrm{VideoMAE}(R_k)\big),$$2. These results show that increased reliance on human heuristics need not coincide with lower predictive performance under the paper’s optimization scheme.

GazeMoE is evaluated on GazeFollow, VideoAttentionTarget, ChildPlay, GazeFollow360, and EYEDIAP (Dai et al., 6 Mar 2026). On VideoAttentionTarget, it reports AUC 0.939, L2 0.097, and AP$$z_k = \mathrm{M3E}_k\big(\mathrm{VideoMAE}(R_k)\big),$$3 0.917, compared with Gaze-LLE ViT-L at AUC 0.937, L2 0.103, and AP 0.903. On ChildPlay, GazeMoE reports AUC 0.945, L2 0.106, and AP 0.994. On EYEDIAP zero-shot, it reports AUC 0.618, L2 0.312, and AP 0.702. The paper attributes robustness partly to the MoE decoder and partly to the focal loss and augmentation pipeline.

B-MoE is evaluated on MA-52, MPII-GroupInteraction, and SocialGesture (Poddar et al., 25 Mar 2026). On MA-52, MANet reports 60.90 Top-1 and 48.98 macro F1, while B-MoE reports 64.54 Top-1 and 53.30 macro F1. On MPII-GI, MANet reports 66.86 and 41.63, whereas B-MoE reports 69.43 and 44.98. The per-class analysis emphasizes gains on ambiguous and underrepresented classes such as “Covering mouth,” “Covering face,” “Scratching feet,” and “Shrugging.”

T-REX reports results across 14 datasets and multiple LLM backbones (Zhang et al., 2024). Intuition-MoR1E improves mean accuracy over LoRA, MoLoRA, and SiRA on Llama 2 13B, Llama 2 7B, Mistral 7B, Yi 6B, Bloom 3B, Phi-2 2B, Gemma 2B, and TinyLlama 1B. For example, on Llama 2 7B the reported averages are 79.90 for LoRA, 79.71 for MoLoRA, 80.29 for SiRA, and 81.11 for Intuition-MoR1E. The abstract states up to 1.78% mean accuracy improvement with around 30%–40% less trainable parameters across 14 public datasets.

HoME is evaluated at Kuaishou scale for multi-task recommendation (Wang et al., 2024). Relative to MMoE, HoME improves GAUC by +0.44 on Effective view, +0.38 on Long view, +0.60 on Click, +0.38 on Like, +0.48 on Comment, +0.65 on Collect, +0.67 on Forward, and +0.57 on Follow. Online A/B tests report play-time gains of +0.636%, +0.735%, and +1.283% across three scenarios.

These studies do not share a common benchmark, but they collectively show that human-aware design is not confined to one modality or task family. It appears in tabular clinical data, RGB images, video, recommender systems, and LLM adapters.

6. Design patterns, tensions, and limitations

Several reusable design patterns recur across the literature. One is semantically meaningful expert partitioning: body parts in B-MoE, human cues in GazeMoE, task/domain/style in AT-MoE, and explicit human vs ML experts in Preferential MoE (Poddar et al., 25 Mar 2026). Another is router-plus-prior composition, exemplified by T-REX’s $$z_k = \mathrm{M3E}_k\big(\mathrm{VideoMAE}(R_k)\big),$$4, where the prior can be semantic today and potentially human-conditioned in a later extension (Zhang et al., 2024). A third is frozen foundation encoder plus lightweight MoE adapters, which appears in GazeMoE and several LoRA-based LLM methods. A fourth is structured interpretation of expert usage, either through sparse linear gates, expert labels, cross-attention maps, Eigenbasis Scores, or MUI-based internal metrics.

The literature also identifies recurring tensions. HoME reports three concrete anomalies in industrial multi-task MoE: expert collapse, expert degradation, and expert underfitting (Wang et al., 2024). It attributes them to gate-driven misallocation of training signal and addresses them with BatchNorm plus Swish experts, hierarchy masks, feature-gates, and self-gates. ERMoE identifies a different tension: standard auxiliary load-balancing losses can reduce load disparities but often weaken expert specialization and hurt downstream performance, so it removes explicit balancing losses and ties routing to expert geometry instead (Cheng et al., 14 Nov 2025). The MUI analysis further shows that shared experts can become concentration hubs, whereas routed-only architectures exhibit a more dispersed “many-hands” collaboration (Ying et al., 28 Sep 2025).

A second tension concerns human labels versus latent structure. T-REX explicitly reports that using human task categories as router reference harms performance relative to embedding-cluster intuition on Llama 2 7B (Zhang et al., 2024). This does not imply that human task structure is useless; AT-MoE shows that task-labeled experts can be valuable for interpretability and controllability (Li et al., 2024). Rather, it suggests that human-aware routing may work best when human concepts are encoded at the right level of abstraction, or combined with learned semantic structure rather than imposed naively.

The limitations are equally consistent. Preferential MoE assumes that human rules are available and meaningful; when explicit rules are unavailable, a surrogate human expert must be learned from historical decisions, which may itself be noisy or biased (Pradier et al., 2021). GazeMoE has no explicit interpretability of which expert corresponds to which cue and no explicit temporal modeling (Dai et al., 6 Mar 2026). B-MoE depends on segmentation quality and uses a fixed four-expert body configuration (Poddar et al., 25 Mar 2026). T-REX uses static embedding-based intuition that may require recomputing clusters under domain shift (Zhang et al., 2024). AT-MoE does not provide explicit safety guarantees, and expert taxonomy design remains hand-crafted (Li et al., 2024). ERMoE notes open questions about scale, deeper analysis of eigenbasis formation, and fairness-oriented analysis of residual routing biases (Cheng et al., 14 Nov 2025).

This suggests that human-aware MoE is not defined by any single mechanism, but by a design objective: to make expert specialization and routing correspond more closely to structures that humans can specify, inspect, trust, or audit. The current literature shows several workable routes to that objective, but it also shows that human awareness introduces additional modeling burdens—expert taxonomy design, prior construction, interpretability validation, and bias analysis—that do not disappear when MoE is made more structured.