MAMEX: Adaptive Multimodal Recommender

• MAMEX is a multimodal recommendation framework designed to tackle cold-start challenges by adaptively weighting diverse input modalities.
• It employs a dual-level MoE architecture with a Modality Extraction Module and a Modality Fusion Module to specialize and integrate modality signals.
• The framework enhances robustness to noisy or missing modalities and outperforms prior baselines on Amazon Reviews datasets.

Searching arXiv for the MAMEX paper and closely related work on multimodal cold-start recommendation. I’ll verify the primary MAMEX paper and scan for nearby arXiv work to ensure the terminology and context are current. MAMEX, short for Multi-modal Adaptive Mixture of Experts for Cold-start Recommendation, is a recommendation framework for item cold-start settings in which new items have little or no interaction history. It is designed to compensate for the weakness of collaborative filtering under sparse feedback by exploiting multimodal item content such as images, text, and audio, while making multimodal fusion adaptive rather than fixed. Architecturally, MAMEX is a dual-level Mixture of Experts (MoE) model composed of a Modality Extraction Module and a Modality Fusion Module, with modality-specific expert adaptation followed by learnable cross-modal weighting (Nguyen et al., 11 Aug 2025).

1. Problem formulation

The paper formulates cold-start recommendation as learning a multimodal recommendation function

$$y_{u,i'} = g(u, i', \{x_{i'}^1, x_{i'}^2, \ldots, x_{i'}^{|\mathcal{M}|}\}, \mathcal{X}),$$

where $\mathcal{U}$ and $\mathcal{I}$ denote the sets of users and items, $$\mathcal{X} \in \mathbb{R}^{|\mathcal{U}| \times |\mathcal{I}|}$$ is a sparse interaction matrix, $x_{i'}^m$ is the raw feature of modality $m$ for item $i'$, and $\mathcal{I}' \subset \mathcal{I}$ denotes cold-start items (Nguyen et al., 11 Aug 2025).

The central difficulty is that no or very few interactions are available for newly introduced items, so collaborative signals are weak or absent. The paper further emphasizes that modalities differ in informativeness, that simple fusion methods such as concatenation or averaging are frequently suboptimal, and that modality quality can vary because some modalities are weak, noisy, or missing. In this setting, MAMEX is motivated by the premise that multimodal signals should not be fused uniformly; rather, the model should learn which modalities to trust for each item.

This framing distinguishes MAMEX from approaches that treat multimodal recommendation as a static representation-learning problem. A plausible implication is that MAMEX is intended not merely to aggregate features, but to perform content-dependent routing and content-dependent fusion, thereby making the representation itself conditional on the item’s modality profile.

2. Dual-level Mixture-of-Experts architecture

MAMEX has two main modules: the Modality Extraction Module and the Modality Fusion Module. The first refines each modality separately through a modality-specific MoE layer; the second adaptively fuses the resulting modality representations into a final item embedding (Nguyen et al., 11 Aug 2025).

Within the Modality Extraction Module, each modality is first encoded by a pretrained modality-specific encoder,

$h_m = E_m(x_m),$

where $E_m$ is the feature extractor for modality $\mathcal{U}$0, $\mathcal{U}$1 is the raw modality input, and $\mathcal{U}$2 is the extracted modality embedding. The paper gives CLIP as an example for image and text and wav2vec 2.0 as an example for audio. Each modality is then passed through a modality-specific MoE layer with $\mathcal{U}$3 experts. The paper writes the adapted representation as

$\mathcal{U}$4

where $\mathcal{U}$5 is the $\mathcal{U}$6-th expert for modality $\mathcal{U}$7, $\mathcal{U}$8 is the gating weight, and $\mathcal{U}$9 is the set of top-$\mathcal{I}$0 experts selected by the router. This yields sparse routing, specialization, and efficiency.

To prevent expert collapse, MAMEX adds a balance regularization term, denoted $\mathcal{I}$1, that encourages the average routing distribution over a batch to remain close to uniform across the $\mathcal{I}$2 experts. In the paper’s description, this improves expert utilization, training stability, and adaptability across items.

The Modality Fusion Module then combines the adapted modality embeddings. The paper uses a learnable sparse gating function $\mathcal{I}$3 over modality embeddings and writes, for the illustrated case,

$\mathcal{I}$4

where $\mathcal{I}$5 denotes concatenation and $\mathcal{I}$6 contains the modality weights. The final item representation is

$\mathcal{I}$7

This mechanism assigns per-item modality weights rather than using fixed fusion.

Two additional regularizers are used at this stage. The first is a balanced fusion regularization term, $\mathcal{I}$8, which discourages systematic collapse onto a single modality. The second is a modality alignment loss,

$\mathcal{I}$9

which keeps the fused embedding close to the modality-specific adapted representations. In the paper’s interpretation, this helps preserve modality-specific semantics while still enabling adaptive fusion.

3. Recommendation scoring and optimization

For top-$$\mathcal{X} \in \mathbb{R}^{|\mathcal{U}| \times |\mathcal{I}|}$$0 recommendation, MAMEX uses user and item embeddings with a dot-product scorer,

$$\mathcal{X} \in \mathbb{R}^{|\mathcal{U}| \times |\mathcal{I}|}$$1

where $$\mathcal{X} \in \mathbb{R}^{|\mathcal{U}| \times |\mathcal{I}|}$$2 is the user embedding and $$\mathcal{X} \in \mathbb{R}^{|\mathcal{U}| \times |\mathcal{I}|}$$3 is the final item embedding (Nguyen et al., 11 Aug 2025).

Training uses Bayesian Personalized Ranking (BPR): $$\mathcal{X} \in \mathbb{R}^{|\mathcal{U}| \times |\mathcal{I}|}$$4 where $$\mathcal{X} \in \mathbb{R}^{|\mathcal{U}| \times |\mathcal{I}|}$$5 is a triplet consisting of a positive item $$\mathcal{X} \in \mathbb{R}^{|\mathcal{U}| \times |\mathcal{I}|}$$6 and a negative item $$\mathcal{X} \in \mathbb{R}^{|\mathcal{U}| \times |\mathcal{I}|}$$7, and $$\mathcal{X} \in \mathbb{R}^{|\mathcal{U}| \times |\mathcal{I}|}$$8 is the sigmoid function. The full objective combines ranking, alignment, adapter balancing, fusion balancing, and $$\mathcal{X} \in \mathbb{R}^{|\mathcal{U}| \times |\mathcal{I}|}$$9 regularization: $x_{i'}^m$0 where $x_{i'}^m$1 denotes all trainable parameters.

This objective makes clear that MAMEX is not a plain MoE recommender. Its optimization target jointly enforces ranking quality, expert utilization, modality balance, and semantic coherence between the fused embedding and the modality-adapted embeddings. A plausible implication is that the model’s empirical behavior depends on the interaction among these terms rather than on sparse routing alone.

4. Robustness to heterogeneous and weak modalities

A central claim of MAMEX is robustness to less informative or missing modalities. The paper attributes this robustness to three mechanisms: modality-specific experts that can specialize within a modality, learnable gates that assign higher weight to informative modalities and lower weight to noisy or less useful ones, and balance regularization that prevents over-reliance on one modality during training (Nguyen et al., 11 Aug 2025).

In the paper’s illustrative interpretation, if an item’s image is noisy but the text is rich, the model can place more weight on text; if a modality is absent or less useful, the learned fusion weights can downweight it. This is presented as a way to improve robustness and adaptability in heterogeneous cold-start scenarios.

At the same time, the conclusion identifies explicitly handling missing modalities as a future direction. This is an important qualification. A common misconception would be to treat MAMEX’s robustness as equivalent to a dedicated missing-modality mechanism. The reported design supports adaptive downweighting and improved tolerance to modality imbalance, but the paper still lists missing-modality handling as an open challenge, alongside cross-modal generation, temporal MoE layers, optimizing expert routing for scalability, and adapting to evolving user preferences.

5. Experimental evaluation

The empirical study evaluates MAMEX on three Amazon Reviews datasets—Amazon Baby, Amazon Clothing, and Amazon Sport—which include product images, textual descriptions, and user reviews. The data is split into 80% train, 10% validation, and 10% test. To simulate cold-start, interactions for items in the development and test sets are completely removed, making these items effectively new. Evaluation uses Recall@10, Recall@20, NDCG@10, and NDCG@20. Reported hyperparameters include expert numbers $x_{i'}^m$2, top-$x_{i'}^m$3 routing $x_{i'}^m$4, learning rate $x_{i'}^m$5, balancing coefficient $x_{i'}^m$6, and the Adam optimizer (Nguyen et al., 11 Aug 2025).

The comparison set includes MTPR, AlignRec, MetaEmbed, DropoutNet, CLCRec, GAR, and MILK. The paper states that MAMEX achieves the best performance across all reported metrics and datasets.

Dataset	MAMEX results	Best prior baseline
Amazon Baby	Recall@10 0.0512; Recall@20 0.0771; NDCG@10 0.0292; NDCG@20 0.0363	MILK: 0.0465; 0.0730; 0.0271; 0.0344
Amazon Clothing	Recall@10 0.1048; Recall@20 0.1501; NDCG@10 0.0606; NDCG@20 0.0729	MILK: 0.0991; 0.1436; 0.0571; 0.0691
Amazon Sport	Recall@10 0.0776; Recall@20 0.1152; NDCG@10 0.0470; NDCG@20 0.0574	MILK: 0.0668; 0.0998; 0.0390; 0.0483

For Amazon Baby, the reported relative improvements over the best baseline are +10.11% Recall@10, +5.62% Recall@20, +7.75% NDCG@10, and +5.52% NDCG@20. For Amazon Clothing, the gains are +5.75% Recall@10, +4.53% Recall@20, +6.13% NDCG@10, and +5.50% NDCG@20. For Amazon Sport, they are +16.17% Recall@10, +15.43% Recall@20, +20.51% NDCG@10, and +18.84% NDCG@20. The paper interprets the strongest gains on Sports as suggesting that adaptive routing and fusion are especially useful when modality informativeness varies substantially.

6. Ablations, interpretation, and reproducibility

The ablation study evaluates three variants: w/o MoE, which replaces modality-specific MoE layers with standard neural networks; w/o Align, which removes the alignment loss; and w/o MMF, which replaces adaptive multimodal fusion with simple averaging (Nguyen et al., 11 Aug 2025).

The full MAMEX model consistently outperforms all variants. The paper reports that removing MoE hurts performance, indicating that modality-specific expert specialization matters; removing adaptive fusion also lowers performance, confirming that learned modality weighting is better than naive averaging; and removing the alignment loss causes the largest degradation, indicating that cross-modal semantic alignment is crucial. It explicitly notes that omitting MoE or adaptive fusion causes meaningful NDCG drops, while excluding alignment loss leads to the largest decline.

The paper also compares three MoE routing designs: Joint Router, with one router over concatenated modalities and shared experts; Modality-Specific Router, with separate routers but shared experts; and MAMEX, with separate routers and separate experts per modality. MAMEX performs best, supporting the claim that maximal specialization—dedicated routing together with dedicated experts—produces the strongest results. It further reports that text performs better than image on its own, while multimodal fusion outperforms either single modality, which the paper treats as evidence for the value of adaptive multimodal integration.

In reproducibility terms, the paper states that code is publicly available at https://github.com/L2R-UET/MAMEX. Taken together, the architectural design, ablation results, and open problems position MAMEX as a multimodal cold-start recommender in which the main contributions are adaptive modality weighting, modality-specific expert specialization, and an explicitly regularized fusion objective rather than fixed multimodal aggregation.