---
title: 'MoAE: Asymmetric Experts in Multi-Attribution CVR'
url: https://www.emergentmind.com/topics/mixture-of-asymmetric-experts-moae
type: topic
---

# MoAE: Asymmetric Experts in Multi-Attribution CVR

Mixture of Asymmetric Experts (MoAE) is a Multi-Attribution Learning (MAL) architecture for conversion rate (CVR) prediction that was introduced together with the Multi-Attribution Benchmark (MAC), the first public CVR dataset providing conversion labels under multiple attribution mechanisms for the same ad click [2603.02184]. MoAE is designed for the setting in which one attribution mechanism is selected as the target prediction objective while labels from other attribution mechanisms are used as auxiliary supervision. In the MAC study, MoAE is presented as a model that combines a Mixture-of-Experts backbone for multi-attribution knowledge learning with a main-task-centric asymmetric transfer module for main-task-oriented knowledge utilization, and it substantially surpasses the existing state-of-the-art MAL method on that benchmark [2603.02184].

## 1. Position within multi-attribution learning

MoAE arises in the context of MAL for CVR prediction. In the formulation used by MAC, each click instance $\mathbf{x}$ is associated with four attribution labels, corresponding to Last-click, First-click, Linear, and Data-Driven Attribution (DDA) via the CausalMTA model [2603.02184]. For each mechanism $\mathcal{A}$, the label is a continuous attribution weight $w^{\mathcal{A}} \ge 0$, interpreted as the conversion credit attributable to that click. A target mechanism $\mathcal{A}_t$ defines the primary task, and the remaining mechanisms define auxiliary tasks. The modeling goal is to train a predictor $f_\theta(\mathbf{x})$ such that
\[
\hat{y}^{\mathcal{A}_t} = f_\theta(\mathbf{x}) \approx w^{\mathcal{A}_t}.
\]

This setting differs from standard public CVR datasets, which use a single attribution mechanism and therefore expose only one label per click [2603.02184]. MAC was constructed precisely to remove that limitation. It provides four attribution “views” of the same user journey, enabling a model to predict CVR under a target attribution mechanism while jointly learning from alternative attribution labels [2603.02184].

The paper situates MoAE against two strands of prior work. One strand consists of general multi-task learning architectures adapted to MAL, including Shared-Bottom, MMoE, PLE, and HoME [2603.02184]. Another is the MAL-specific NATAL baseline, which uses AKA, PTP, and CAT and emphasizes asymmetric knowledge transfer from auxiliary tasks to the primary task [2603.02184]. MoAE is explicitly motivated by the observation that these two strands capture different desirable properties: MoE-based models learn multi-attribution knowledge well, whereas NATAL prioritizes main-task performance through asymmetric transfer [2603.02184].

## 2. Benchmark setting and empirical motivation

MAC is built from Taobao ad traffic over 21 consecutive days, with stratified user sampling and a final scale of approximately 0.8M users, 79M clicks, and tens of millions of items [2603.02184]. Each click contains categorical features, behavior sequence features, and four multi-attribution labels. Behavior sequence features include a user’s purchase item sequence truncated to max length 20, with item ID, shop ID, category ID, and a visual similarity score to the current ad item computed from an image encoder trained by contrastive learning [2603.02184].

The four attribution mechanisms in MAC induce markedly different label sparsities. Positive sample ratios are 1.6% for Last-click, 1.8% for First-click, 4.3% for DDA, and 5.3% for Linear [2603.02184]. The benchmark therefore exposes an intrinsic heterogeneity in supervision density: Linear and DDA produce more positive samples because they spread conversion credit across multiple clicks, whereas Last-click and First-click place all credit on a single touchpoint [2603.02184].

The empirical analyses on MAC motivate MoAE through three findings. First, MAL brings consistent performance gains across different attribution settings, especially for users featuring long conversion paths [2603.02184]. Second, performance growth generally scales with objective complexity, but when predicting First-click conversion targets, simply adding auxiliary objectives is counterproductive [2603.02184]. Third, two architectural design principles are identified as paramount: fully learning the multi-attribution knowledge, and fully leveraging this knowledge to serve the main task [2603.02184]. MoAE is presented as a direct implementation of these two principles.

A further motivation is the observed tradeoff among baseline models. PLE and HoME, which incorporate MoE structure, improve all tasks but do not prioritize the main task as strongly as NATAL. NATAL maximizes main-task performance via asymmetric transfer but sacrifices auxiliary task performance [2603.02184]. This suggests that a successful MAL architecture must combine broad attribution-specific representation learning with explicitly asymmetric main-task-centric transfer.

## 3. Architecture

MoAE consists of three main components: a MoE backbone, a main-task-centric asymmetric transfer module, and per-attribution MLP predictors [2603.02184].

The MoE backbone contains one shared expert and several attribution-specific experts. The shared expert learns general conversion patterns common to all attribution mechanisms, while attribution-specific experts specialize to Last-click, First-click, DDA, and Linear [2603.02184]. For each attribution task, a gating network outputs a distribution over experts and forms a task-specific representation by weighted summation. The paper represents the experts as
\[
f_k(\mathbf{x}) : \mathbb{R}^d \to \mathbb{R}^{h}, \quad k = 1, \dots, K_{\text{exp}}
\]
and the task-specific gate for task $t$ as
\[
g_t(\mathbf{x}) = \text{softmax}(W_t \mathbf{x} + b_t) \in \mathbb{R}^{K_{\text{exp}}}.
\]
The corresponding task representation is
\[
\mathbf{h}_t = \sum_{k=1}^{K_{\text{exp}}} g_{t,k}(\mathbf{x}) \, f_k(\mathbf{x}).
\]

This backbone is described as being inspired by PLE and as encouraging both shared and private knowledge [2603.02184]. In the paper’s terminology, it is the mechanism by which MoAE “fully learn[s] the multi-attribution knowledge.”

The second component is the main-task-centric asymmetric transfer module. It takes auxiliary-task representations and injects them into the main-task tower, but not vice versa [2603.02184]. The transfer is therefore asymmetric: auxiliary tasks support the primary task, while auxiliary heads are not influenced by the main task’s internal features in the reverse direction. At a high level, the enriched primary representation is expressed as
\[
\tilde{\mathbf{h}}_{\text{primary}} = \phi\Big(\mathbf{h}_{\text{primary}}, T(\{\mathbf{h}_{\text{aux}^{(k)}}\})\Big),
\]
where $T(\cdot)$ is an aggregation or transfer function and $\phi(\cdot)$ is a fusion function [2603.02184]. The paper does not fix these abstract operators to a single concrete mechanism in the summary, but it states their roles as aggregation and fusion.

The third component consists of per-attribution MLP predictors. Each attribution mechanism has its own MLP head that outputs $p_i^{\mathcal{A}}$ [2603.02184]. For the primary task, the head consumes the enriched representation $\tilde{\mathbf{h}}_{\text{primary}}$; auxiliary heads use their own expert-based representations [2603.02184].

A concise architectural summary is as follows.

| Component | Function | Task orientation |
|---|---|---|
| MoE backbone | Learns shared and attribution-specific patterns | All attributions |
| Asymmetric transfer module | Injects auxiliary representations into the main tower | Main task only |
| Per-attribution MLP predictors | Outputs task-specific probabilities | One head per attribution |

Conceptually, MoAE blends the strengths of PLE or HoME, which emphasize expert-based knowledge learning, and NATAL, which emphasizes asymmetric knowledge transfer [2603.02184]. This suggests that MoAE should be understood less as a minor MoE variant than as a hybrid MAL architecture whose asymmetry is central rather than incidental.

## 4. Objectives and training formulation

On MAC, each attribution mechanism $\mathcal{A}$ defines a binary classification task derived from continuous attribution weights. A sample is positive if $w^{\mathcal{A}} > 0$ and negative if $w^{\mathcal{A}} = 0$ [2603.02184]. The per-task loss is binary cross-entropy:
\[
\mathcal{L}^{\mathcal{A}} = -\frac{1}{N} \sum_{i=1}^N \Big[ y_i^{\mathcal{A}} \log p_i^{\mathcal{A}} + (1 - y_i^{\mathcal{A}}) \log (1 - p_i^{\mathcal{A}}) \Big].
\]

The joint MAL objective uses one primary task and several auxiliary tasks:
\[
\mathcal{L} = \mathcal{L}_{\text{primary}} + \lambda \sum_k \mathcal{L}^{(k)}_{\text{aux}},
\]
where $\lambda$ is selected from $\{0.1, 0.2, 0.3, 0.4\}$ [2603.02184]. The paper also evaluates GCS and PCGrad as adaptive training plugins, but reports that their impact is relatively minor compared with architectural choices [2603.02184].

MoAE can additionally include Cartesian Auxiliary Training (CAT). For $N$ attribution mechanisms, each with binary label $A_i \in \{0,1\}$, CAT defines a multi-class auxiliary label
\[
O = \sum_{i=0}^{N-1} A_i \cdot 2^i, \quad O \in \{0, \dots, 2^N - 1\}.
\]
For MAC, where $N=4$, this yields 16 classes [2603.02184]. The total loss becomes
\[
\mathcal{L} = \mathcal{L}_{\text{primary}} + \lambda \sum_k \mathcal{L}^{(k)}_{\text{aux}} + \lambda_{\text{CAT}} \mathcal{L}_{\text{CAT}}.
\]

Auxiliary-task selection is not treated as trivial. The MAC study performs greedy search over combinations of auxiliary attributions: starting from no auxiliaries, it iteratively adds one auxiliary objective at a time and keeps it only if validation GAUC improves [2603.02184]. The resulting best configurations are asymmetric across target mechanisms. For Last-click, Linear, and DDA targets, the best configuration uses all other attribution labels plus CAT. For First-click, the best configuration uses only Last-click as auxiliary; adding more hurts performance [2603.02184].

This behavior is one of the clearest indications that MoAE is not merely a capacity expansion strategy. Its utility depends on how heterogeneous supervision sources are chosen and coupled. The paper interprets the First-click exception as a consequence of noisy labels caused by long time lag and more confounders [2603.02184].

## 5. Empirical performance on MAC

On MAC, MoAE is reported as the best MAL model for each target attribution mechanism [2603.02184]. The main primary-task GAUC results are:

| Target attribution | BASE GAUC | MoAE GAUC |
|---|---:|---:|
| Last-click | 0.7424 | 0.7636 |
| First-click | 0.6969 | 0.7003 |
| DDA | 0.7470 | 0.7644 |
| Linear | 0.7602 | 0.7682 |

These correspond to gains of +2.12 points for Last-click, +0.34 for First-click, +1.74 for DDA, and +0.80 for Linear relative to the single-attribution BASE model [2603.02184]. The gains are largest for Last-click and DDA and smallest for First-click.

The paper also reports a diagnostic comparison for Last-click targeting that isolates architectural ingredients [2603.02184]. Shared-Bottom achieves primary GAUC 0.7571; PLE 0.7583; HoME 0.7594; NATAL 0.7613; and MoAE 0.7636 [2603.02184]. In auxiliary-task GAUC, PLE and HoME are strong, NATAL is weaker, and MoAE remains strong while also maximizing the primary objective [2603.02184]. This is the core empirical argument that MoAE achieves the “best of both worlds.”

Performance improvements are particularly pronounced for users with long conversion paths. The paper measures conversion path complexity as
\[
\text{complexity} \approx \frac{\# \text{linear positives}}{\# \text{last-click positives}}.
\]
Higher values indicate more clicks per conversion [2603.02184]. For Last-click, Linear, and DDA targets, GAUC improvements from MAL increase with path complexity [2603.02184]. The trend is less clear for First-click, which the paper again links to noisier labels [2603.02184].

Objective complexity provides a second empirical axis. For a Last-click target, progressively adding auxiliary tasks and CAT improves GAUC from 0.7429 with no auxiliary objectives to 0.7636 with First-click, Linear, DDA, and CAT [2603.02184]. By contrast, for a First-click target, only Last-click as auxiliary improves performance, from 0.6971 to 0.7003, while adding DDA, Linear, or CAT reduces GAUC into the approximately 0.688–0.693 range [2603.02184].

The paper supplies additional evidence for First-click noisiness through MML, the mean of $w^{\text{DDA}} - w^{\text{Linear}}$. For Last-click positives, MML is +0.05; for First-click positives, MML is –0.03 [2603.02184]. Since DDA is causally informed, the disagreement for First-click positives is interpreted in the paper as evidence that many such positives are not truly influential [2603.02184].

## 6. Interpretation, significance, and limitations

MoAE is significant primarily as an architectural synthesis within MAL. It operationalizes two design principles extracted from MAC experiments: rich multi-attribution knowledge learning and main-task-first knowledge utilization [2603.02184]. In that sense, its contribution is not separable from the benchmark that motivated it. MAC supplies the multiple attribution labels needed to expose both the benefits and the failure modes of multi-attribution supervision, while PyMAL provides the standardized environment in which MoAE is compared with BASE, Shared-Bottom, MMoE, PLE, HoME, and NATAL [2603.02184].

Its broader importance lies in how it reframes CVR modeling under attribution ambiguity. Standard single-attribution datasets force models to treat one business rule as the sole target. MAC and MoAE instead treat last-click, first-click, linear, and DDA labels as complementary supervision signals for the same event [2603.02184]. This suggests a shift from single-view prediction toward learning across multiple attribution mechanisms.

At the same time, the reported results delimit MoAE’s applicability. The benchmark is domain-specific to Taobao display ads, and the attribution window is fixed but confidential [2603.02184]. Only four attribution mechanisms are exposed, and all conversions use direct conversion attribution, where conversions are attributed only to clicks on ads of the same item [2603.02184]. The paper also shows that more supervision is not universally beneficial: for First-click targeting, additional auxiliary objectives can degrade performance [2603.02184]. This directly counters a possible misconception that MoAE’s gains arise simply from adding more tasks or more labels.

From a methodological standpoint, MoAE belongs to a broader line of attribution-aware advertising models that includes sequence and causal attribution architectures such as DARNN [1808.03737] and CAMTA [2012.11403]. Those works focus on assigning credit across touchpoints or channels, whereas MoAE operates at the level of MAL for click-level CVR prediction under multiple attribution labels [2603.02184; 1808.03737; 2012.11403]. A plausible implication is that MoAE’s main conceptual novelty lies not in replacing attribution modeling itself, but in using multiple attribution mechanisms as coordinated supervision targets for a primary CVR task.

In practice, the MAC paper positions MoAE as the default strong MAL architecture within PyMAL, trained with Adam for one full pass over the 20-day training set, batch size 4096, and learning-rate search over $\{0.1x, x, 10x\}$ for $x \in \{0.0030, 0.0035, 0.0040\}$ [2603.02184]. That recommendation follows directly from the empirical pattern reported on MAC: MoAE consistently improves over single-attribution training and surpasses the existing MAL baselines on the benchmark [2603.02184].

Source: https://www.emergentmind.com/topics/mixture-of-asymmetric-experts-moae