Alias1: Unified Multi-Modal Architecture

• Alias1 architecture is a unified deep learning framework that concurrently handles multiple data modalities including vision, speech, and text.
• It employs modality nets to translate raw inputs into a shared 1024-d latent space, processed uniformly through convolution, multi-head attention, and sparsely-gated MoE layers.
• Empirical results reveal that joint multi-task training boosts performance on low-resource tasks while preserving high-resource task efficacy.

The Alias1 model architecture is a unified deep learning framework capable of concurrent training and inference across multiple data modalities, including vision, speech, machine translation, image captioning, and parsing, without modification to its core structure. Alias1 integrates computational primitives from diverse domains—depthwise-separable convolutions, multi-head attention, and sparsely-gated mixture-of-experts layers—enabling all tasks to share a common body, representations, and parameter set. Modality-specific “modality nets” translate raw data (text, image, audio, or categorical) into and out of a shared variable-length latent space of depth 1024, after which the unified body processes all examples identically. Empirical results show effective multi-task transfer, with smaller tasks benefiting from joint training and minimal performance degradation on large data tasks (Kaiser et al., 2017).

1. Overall Architectural Flow

Alias1 structure comprises four “modality nets” connected to a shared processing body. Each modality net projects input data—language tokens, images, audio waveforms/spectrograms, or categorical classes—into a depth-1024 latent representation. The unified body then processes these representations through:

• An encoder: six convolutional blocks, with a single mixture-of-experts (MoE) layer after the third block.
• An I/O mixer: one multi-head attention block followed by two convolutional blocks, operating autoregressively.
• A decoder: four conv-attention blocks, with a second MoE interleaved between the second and third blocks.

The processing is strictly causal—left-padding ensures autoregressive behavior throughout mixing and decoding, preventing access to future positions during training or sampling. Task specification occurs by prepending a learned “command token” (e.g., “To-French”) to the output sequence, drawn from the shared vocabulary.

2. Modality Nets and Data Translation

Each “modality net” serves as an adapter between raw input/output and the shared latent space:

• Language input: An 8,000-token subword vocabulary maps to 1024-dimensional embeddings via $W_E$. Language output applies a $1024 \to 8000$ linear map $W_S$ and softmax.
• Image input: An entry flow aggregates four depthwise-separable ConvRes blocks, beginning with $3\times3$ stride-2 (32 channels), followed by $3\times3$ (64 channels), and two residual ConvRes (128/256 channels), ultimately producing a 1024-channel latent.
• Categorical output: Employs the reverse of the image entry flow, concluding with global average pooling and a $1\times1$ convolution projecting to the number of classes.
• Audio input: Raw 1D waveforms or 2D spectrograms pass through eight ConvRes blocks with exponentially increasing width (2, 4, 8…128 → 256 → 512 → 1024 channels), avoiding frequency-axis stride in the spectrogram case.

All data modalities thus map into or out of the same feature space, allowing seamless routing through the shared body.

3. Computational Primitives

a) Depthwise-Separable Convolution Blocks

The localized pattern extractor comprises four-step stacks, each performing ReLU activation, depthwise-separable convolution, layer normalization, and 40% dropout (during training):

$$\mathrm{ConvStep}_{d,s,f}(W,x) = \mathrm{LN}\bigl(\mathrm{SepConv}_{d,s,f}\bigl(W,\;\mathrm{ReLU}(x)\bigr)\bigr)$$

A full convolutional block at $D=1024$ width is defined by:

$$\begin{aligned} h_1 &= \mathrm{ConvStep}_{1,1,D}(W_{h1},\,x) \ h_2 &= x + \mathrm{ConvStep}_{1,1,D}(W_{h2},\,h_1) \ h_3 &= \mathrm{ConvStep}_{1,1,D}(W_{h3},\,h_2) \ h_4 &= x + \mathrm{ConvStep}_{8,1,D}(W_{h4},\,h_3) \ \mathrm{ConvBlock}(x) &= \begin{cases} \mathrm{Dropout}(h_4; 0.4), & \text{training} \ h_4, & \text{otherwise} \end{cases} \end{aligned}$$

b) Multi-Head Dot-Product Attention

Each attention block receives source $S \in \mathbb{R}^{B \times T_s \times D}$ and target $T \in \mathbb{R}^{B \times T_t \times D}$. The model first adds sinusoidal positional encoding:

$$\Delta(2k) = 10^{-2k/D}, \quad \mathrm{Pos}(t,2k)=\sin(t\,\Delta(2k)),\; \mathrm{Pos}(t,2k+1)=\cos(t\,\Delta(2k))$$

After context mixing (via two ConvBlocks), the representation is projected to queries $(Q)$, keys $(K)$, and values $(V)$ using learned pointwise convolutions:

$Q = T' W_Q, \quad K = S W_K, \quad V = S W_V$

These are split into $g=8$ heads, with scaled dot-product attention computed for each:

$$\alpha_{ij} = \frac{\exp(q_i \cdot k_j/\sqrt{D/g})}{\sum_{j'}\exp(q_i \cdot k_{j'}/\sqrt{D/g})}$$

Outputs are concatenated and passed through a $D \times D$ output projection $W_O$.

c) Sparsely-Gated Mixture-of-Experts

Midway through encoding and decoding, Alias1 applies a sparsely-gated MoE layer as in Shazeer et al. (2017). Each layer contains $E$ two-layer ReLU feed-forward MLP “experts.” Position-wise gating is computed:

$g(x) = \mathrm{softmax}(W_g\,x) \in \Delta^{E-1}$

During training, the top-$k=4$ experts contribute to each position's output:

$$\mathrm{MoE}(x) = \sum_{i=1}^{E} g_i(x)\,\mathrm{Expert}_i(x)$$

E is set to 240 for joint multi-task training and 60 for single-task, and load-balancing loss is added as in Shazeer et al.

4. Parameter Sharing and Hyper-parameters

All parameters—including convolutional filters, attention matrices, expert weights, and modality-net embeddings—are globally shared across tasks and modalities. Core hyper-parameters:

Component	Value	Role
Depth (D)	1024 features	ConvBlock and Attention width
Encoder Layers	6	Feature extraction
Mixer Layers	1 attention, 2 ConvBlocks	I/O autoregressive mixing
Decoder Layers	4 conv-attention blocks	Autoregressive output
Attention Heads (g)	8	Multi-head attention
Experts (E)	240 (joint), 60 (single-task)	Mixture-of-experts layers
Top-k (MoE)	4	Active experts per position
Dropout (p)	0.4	Post-ConvBlock regularization
Vocabulary	8,000 subword tokens	Language domains
Optimizer	Adam, gradient clipping	Training
Batch Size (modality)	64 (images), 4,096 (speech frames), 512 (tokens)	Data-specific

5. Multi-Task Training Paradigm

Alias1 is jointly trained across eight distinct tasks, each providing a standard cross-entropy loss $\ell_t$ (token-level or categorical/sequence likelihood as appropriate):

$$\mathcal{L} = \sum_{t=1}^8 \sum_{i=1}^{N_t} -\log p\left(y^{(t)}_i \mid x^{(t)}_{<i}\right) + \mathcal{L}_\mathrm{load}$$

No explicit per-task weighting is introduced. Training employs Adam optimizer with gradient clipping and asynchronous workers per task. Each worker draws batches from its domain, updates gradients on its task loss, and contributes to the shared parameter set. Efficient data balancing is achieved naturally since high-volume tasks (e.g., ImageNet, WMT English–French) submit many updates, while low-resource tasks (e.g., parsing) submit few.

6. Empirical Properties and Transfer Behavior

Jointly trained Alias1 models display minimal performance degradation on the largest data tasks and substantial performance improvement on smaller data tasks, highlighting the efficacy of inductive transfer between modalities. In practice, architectural blocks not strictly necessary for a given task were found not to harm and, in most cases, improved overall performance. This suggests a plausible implication that architectural over-specification for multi-domain universality is a favorable strategy for deep learning models seeking broad applicability across data types (Kaiser et al., 2017).

7. Significance Within Multi-Domain Models

Alias1 demonstrates that a carefully crafted unified architecture, built from modern computational primitives and modality adapters, can streamline model deployment and training across vision, audio, language, and structured data domains. Its empirical robustness across diverse tasks and high degree of parameter sharing offer a template for future research in universal models, scalable multi-task learning, and transfer learning. The sparsely-gated MoE component, referencing Shazeer et al. (2017), illustrates scalable capacity increases without prohibitive computational cost, and its use of command tokens for universal decoding provides a minimal method for dynamic task specification.