SenseNova-U1-A3B-MoT: Unified Multimodal Model

• SenseNova-U1-A3B-MoT is a unified multimodal model that combines vision and language processing using a shared token space and specialized streams.
• It employs a sparse Mixture-of-Experts design with top-8 expert gating to efficiently scale capacity in both understanding (128 experts) and generation (32 experts) tasks.
• The model demonstrates state-of-the-art performance in reasoning, any-to-image synthesis, and multimodal tasks by integrating consistent training objectives and hybrid inference mechanisms.

SenseNova-U1-A3B-MoT is a large-scale unified multimodal model that advances vision-language modeling by integrating understanding and generation within a single architecture. Developed as part of the SenseNova-U1 family, it leverages the NEO-unify architecture and employs a Mixture-of-Experts (MoE) framework to efficiently scale capacity and performance. The “A3B” designation refers to a sparse MoE configuration with approximately 3 billion active parameters per token during inference, supporting sophisticated reasoning, vision-language perception, spatial intelligence, and generative capabilities such as any-to-image synthesis and infographic generation. The model is designed to seamlessly transition between perception, generation, decision-making, and even world-modeling tasks without decoupling modality representations (Diao et al., 12 May 2026).

1. Model Architecture and MoE Design

SenseNova-U1-A3B-MoT is built upon the NEO-unify primitive, which processes raw pixels and textual tokens within a shared representational space. Patch encoding is accomplished via two convolutional layers (stride 16→2), transforming $32\times32$ image patches into tokens, after which both image and text tokens are projected into a shared $2\,048$-dimensional space using token embedding.

The unified backbone consists of a Mixture-of-Transformers (MoT), maintaining distinct “understanding” and “generation” streams, but with shared self-attention and rotary embeddings. Its patch decoding branches implement a linear head for understanding tasks (outputting vocabulary logits) and an MLP head for generation, responsible for pixel-patch prediction.

The MoE configuration for A3B employs:

• Understanding stream: $M_u=128$ experts, each with $h_u=2\,048$ hidden size, totaling approximately $30$B parameters.
• Generation stream: $M_g=32$ experts, each with $h_g=2\,048$ hidden size, with $8.2$B parameters.
• Sparsity: For each token, only the top-$k=8$ experts are activated ($\approx3$B active parameters, yielding the “A3B” label).

Gating and Routing

Each MoE layer computes token-specific expert weights via $2\,048$0 and selects the top-$2\,048$1 experts per token. A load-balancing loss penalizes deviation from uniform expert usage: $2\,048$2 where $2\,048$3 during generation pretraining and $2\,048$4 during joint training.

Mixture-of-Tokens (MoT) routing distributes each token to the selected top-$2\,048$5 experts, yielding

$2\,048$6

with an optional routing entropy regularization.

2. Training Data, Preprocessing, and Filtering

Training leverages large-scale, meticulously curated datasets segmented for both understanding and generation. Corpus statistics include:

• Understanding pretrain (1.19T tokens): 32% image–text pairs, 17% captions, 14% infographic QA, 37% pure text.
• Mid-training (0.88T tokens): general (39.2%), agent/spatial (22.3%), knowledge (19.3%), pure text (19.2%).
• SFT (0.13T tokens): fine-tuned on mixes of spatial, reasoning, OCR, dialogue, code.

Data preprocessing employs cross-source deduplication (MinHash + CLIP clusters), safety and quality filters (NSFW, low-res, and corrupt data removal), CLIP-ratio re-captioning for relevance, and prompt augmentation with multi-criteria filtering against correctness, hallucination, and style.

For generation, the corpus comprises:

• Text-to-Image: nature (40.5%), people (26.7%), design (20.7%), infographics, and bilingual text.
• Editing: natural scenes (52.3%), people (14.7%), infographic/synthetic, with diverse operations (add/remove, style, composites).
• Interleaved: lifestyle (44%), infographics (29%), video (19%), reasoning (8%).

A multi-stage filtering flow (low-level cleanup, deduplication, VLM captioning, quality scoring) is used throughout.

3. Optimization Objectives and Losses

The global training objective integrates separate losses for understanding, generation, and MoE balance: $2\,048$7 with weights $2\,048$8 and $2\,048$9 during joint training.

• Understanding Loss: Negative log-likelihood over token prediction,

$M_u=128$0

• Generation Loss: Flow-matching, predicting denoising “velocity” between perturbed and target data,

$M_u=128$1

where $M_u=128$2, $M_u=128$3, and $M_u=128$4 is computed as specified in the architecture.

• Balance loss: As outlined above.

Inference incorporates classifier-free guidance (CFG),

$M_u=128$5

typically with $M_u=128$6.

4. Inference Mechanisms and Deployment

Deployment is disaggregated for efficiency using distinct LightLLM (multimodal prefill, text streaming) and LightX2V (pixel-space denoising) engines with shared memory for key-value caches, supporting independent scaling. For a $M_u=128$7 T2I task (with TP2+CFG2), per-step latencies are $M_u=128$8s on NVIDIA RTX 5090 and $M_u=128$9s on NVIDIA L40S with approximately $h_u=2\,048$0GB memory per engine.

Prompt and attention handling leverages a hybrid kernel (causal masks for text rows, full span for image rows post-$h_u=2\,048$1), and noise-scale conditioning combines task time-step $h_u=2\,048$2 and patch size normalization.

5. Empirical Performance and Benchmarking

Quantitative performance is reported on a suite of established multimodal benchmarks. SenseNova-U1-A3B-MoT offers strong or state-of-the-art results in the following domains:

Multimodal Understanding

Benchmark	U1-30BA3B-Think	Qwen3VL-30BA3B
MMMU (reasoning)	80.55	76.00
VSI-Bench (32f)	56.90	51.56*

Pure-Text Understanding (MMLU-Pro %)

Subtask	U1-30BA3B	Qwen3VL-30BA3B
knowledge	84.04	80.50
instr-follow	79.79	34.69

Text-to-Image Generation

GenEval Overall	U1-A3B-MoT	Qwen-Image
	0.91	0.87

DPG-Bench Overall	U1-A3B	Qwen-Image
	88.14	88.32

Text-Rich and Infographic Generation

CVTG-2K Avg. Acc	U1-A3B	Qwen-Image
	0.881	0.829

IGenBench Q-ACC	U1-A3B	Qwen-Image
	0.42	0.36

Additional Modalities

• Reasoning-centric generation (WISE, with CoT): 0.81 (U1-A3B-SFT) vs 0.63 (Qwen-Image)
• Interleaved Gen (OpenING, with CoT): 9.16 (U1-A3B-SFT) vs 8.20 (GPT-4o+DALL-E3)
• Vision–Language–Action (VBVR-Image preview): 68.9% (U1-A3B-SFT) vs 62.3% (Nano-Banana-2)
• Image Editing (ImgEdit Overall): 3.91 (U1-A3B), with higher Qwen-Edit (4.51)

6. Synergies, Trade-Offs, and Interpretations

Ablation studies indicate that joint parameter decoupling with shared attention minimizes conflict between tasks and supports rapid convergence (generation performance stabilizes within 120K steps without degrading text understanding). Understanding and generation streams are shown to co-adapt synergistically.

Comparisons between MoE and dense variants show the A3B configuration sustains approximately threefold active inference capacity, enhancing reasoning-oriented tasks (WISE: +0.03 gain versus dense). Dense 8B matches or slightly outperforms A3B in pure generation benchmarks due to the full parameter utilization. MoE’s larger capacity supports superior performance on knowledge-intensive benchmarks, while dense variants offer lower and more consistent latency.

Preliminary qualitative and quantitative results in vision-language-action (VLA) and world modeling (WM) suggest native ability for pixel-level future state prediction and multi-modal policy reasoning. A plausible implication is the feasibility of deploying unified agents for embodied tasks, real-time planning, and closed-loop world modeling using a single foundation model (Diao et al., 12 May 2026).

7. Implications and Future Prospects

SenseNova-U1-A3B-MoT exemplifies an architectural shift from fragmented, pipeline-based multimodal systems to singular unified paradigms. The approach departs from explicit modality translation, instead allowing model capabilities to emerge from unified token space processing. This suggests a foundational pathway where reasoning, synthesis, and actionable intelligent behavior arise natively within one architecture. Future applications are anticipated in embodied agents, dynamic world modeling, policy generation, and real-time multimodal planning. Potential future work includes further scaling of MoE configurations, extension to higher resolution video-modalities, and continued ablation to disentangle emergent behaviors enabled by unified architectures (Diao et al., 12 May 2026).