Time-Interval Disentangled Experts (TIDE)

• TIDE is a framework where time intervals serve as a key axis for disentangling expertise across models in recommendations, diffusion fine-tuning, and system inference.
• It is instantiated through domain-specific designs such as dual-expert next-basket recommendation, interval-specific LoRA in diffusion, and lossless MoE inference for LLMs.
• Empirical evaluations demonstrate that leveraging temporal structure leads to notable improvements in prediction accuracy, reconstruction, and throughput across various tasks.

Searching arXiv for the cited TIDE-related papers to ground the article in the primary sources. Time-Interval Disentangled Experts (TIDE) denotes a class of temporally structured modeling ideas in which time intervals are treated as a primary axis for specialization, routing, or decomposition. The phrase appears explicitly as the name of a next-basket recommendation model that separates habitual repurchase from exploratory interest while making both intents time-interval aware (Deng et al., 1 May 2026), and as the name of a lossless inference system for MoE diffusion LLMs that exploits temporal stability of expert activations within diffusion blocks (Chen et al., 19 May 2026). Closely related formulations also appear in diffusion-model research, where timestep intervals are used to train interval-specific LoRA experts or to interpret sparse temporally modulated units as expert-like components aligned with denoising phases (Zhuang et al., 10 Mar 2025, Huang et al., 10 Mar 2025). The acronym is therefore polysemous rather than canonical, and one database paper uses TIDE for the unrelated phrase “time intervals by duration and endpoint” (Wang et al., 22 Jun 2026).

1. Scope and conceptual basis

Across the expert-oriented usages of TIDE, the common technical premise is that temporal position is not merely an auxiliary conditioning variable. In diffusion models, timesteps correspond to different noise scales, different effective conditioning strengths, and different semantic versus low-level focus, so they “naturally suggest a decomposition into time-specific or time-interval-specific experts” (Zhuang et al., 10 Mar 2025). In interpretable DiT analysis, sparse units are described as specializing in early timesteps for coarse structure and global 3D layout, middle timesteps for object contours and mid-level semantics, and late timesteps for fine texture and local style (Huang et al., 10 Mar 2025). In MoE diffusion LLM inference, the decisive observation is different but structurally related: expert routing across successive denoising steps within a block is highly stable, so expert placement decisions can be amortized across intervals rather than recomputed at every step (Chen et al., 19 May 2026).

Paper	Official expansion of TIDE	Technical role
(Deng et al., 1 May 2026)	Time-Interval Disentangled Experts	Dual-expert next-basket recommendation
(Chen et al., 19 May 2026)	Time-Interval Disentangled Experts	Lossless MoE diffusion LLM inference system
(Zhuang et al., 10 Mar 2025)	TimeStep Master	“TIDE-style” interval-specific LoRA experts
(Huang et al., 10 Mar 2025)	Temporal-aware Sparse Autoencoders for Interpretable Diffusion transformErs	Sparse units interpreted as time-interval-disentangled experts
(Wang et al., 22 Jun 2026)	time intervals by duration and endpoint	Disk-based interval index, unrelated to expert modeling

This multiplicity matters because “TIDE” does not denote a single fixed architecture. In some papers it is an explicit model family, in others an interpretive description of learned specialization, and in one case an inference runtime. A recurring misconception is therefore to treat all TIDE papers as variations of the same MoE design. The literature instead supports a narrower statement: time intervals are being used as an organizing coordinate for disentanglement, specialization, or scheduling, but the mechanisms differ substantially across domains.

2. TIDE in next-basket recommendation

In next-basket recommendation, TIDE is defined as a model for predicting the set of items a user will purchase next from a history of transaction baskets with timestamps (Deng et al., 1 May 2026). The task is formulated over a user set $\mathcal{U}$ and item set $\mathcal{I}$, with basket sequence $\mathcal{S}_u=(B_1,B_2,\dots,B_k)$, where each basket $B_m\subseteq\mathcal{I}$ is purchased at time $t_m$. The paper identifies two distinct user intents: habitual repurchase, which depends on a user’s own recurring behavior and item-specific periodicity, and exploratory interest, which depends on global co-occurrence and collaborative patterns. Its central critique of prior NBR models is twofold: they often entangle these conflicting motives within a single latent space, and they usually ignore continuous-time intervals and item-specific periodicities.

TIDE addresses the temporal component with a Hawkes-enhanced Fourier Time Encoding. For an item $i$, the interval to the prediction target is defined as

$\Delta t_i=(k+1)-m_{last}^{(i)}.$

This scalar is encoded by a Fourier-like representation

$$\mathbf{p}_i=[\sin(\mathbf{w}\Delta t_i+\boldsymbol{\phi});\ \cos(\mathbf{w}\Delta t_i+\boldsymbol{\phi})],$$

which is then combined with the item embedding $\mathbf{v}_i$ and a frequency embedding $\mathbf{e}_i$:

$\mathcal{I}$0

A Hawkes-inspired dynamic decay is applied through

$\mathcal{I}$1

yielding the time-aware item representation $\mathcal{I}$2. This design is intended to capture both non-monotonic periodicity and item-specific decay.

The architectural disentanglement is implemented with two experts. The Habit Expert aggregates the time-aware vectors $\mathcal{I}$3 within each basket through additive attention and then sums basket representations into a habit intent vector,

$\mathcal{I}$4

The Exploration Expert uses a global item-pattern bipartite graph $\mathcal{I}$5 and LightGCN-style message passing to construct graph-enhanced item embeddings $\mathcal{I}$6, then projects $\mathcal{I}$7 into an exploration space as

$\mathcal{I}$8

An item-aware gating mechanism computes

$\mathcal{I}$9

so that the final prediction is a per-item mixture of habit and exploration scores:

$\mathcal{S}_u=(B_1,B_2,\dots,B_k)$0

Training combines a recommendation cross-entropy with an InfoNCE contrastive loss aligning $\mathcal{S}_u=(B_1,B_2,\dots,B_k)$1 and $\mathcal{S}_u=(B_1,B_2,\dots,B_k)$2.

The empirical evaluation uses four Amazon domains after pre-processing and 5-core filtering: Beauty, Sports, Grocery, and Home. Metrics are Recall@K and NDCG@K for $\mathcal{S}_u=(B_1,B_2,\dots,B_k)$3. The paper reports that TIDE is best or tied-best on all metrics, with representative gains including Beauty R@10 of 0.1276 versus 0.1078 for SemTHy, Grocery N@20 of 0.0605 versus 0.0550 for TREx, and Home R@20 of 0.0286 versus 0.0212 for SemTHy (Deng et al., 1 May 2026). Ablations show that removing the exploration graph causes especially large degradation: Beauty R@10 drops from 0.1276 to 0.0510, and Home R@10 drops from 0.0160 to 0.0017. The paper interprets these results as evidence that disentangling habit from exploration and making both interval-aware improves next-basket prediction.

3. Interval-disentangled experts in diffusion fine-tuning

A closely related but differently named formulation appears in “TimeStep Master,” which presents an asymmetrical mixture of timestep LoRA experts for diffusion models (Zhuang et al., 10 Mar 2025). The motivating observation is that a single LoRA update $\mathcal{S}_u=(B_1,B_2,\dots,B_k)$4 is shared across all diffusion timesteps in standard practice, even though internal activations and denoising behavior differ substantially across noise levels. The paper therefore proposes interval-specific LoRA experts by partitioning the timestep range $\mathcal{S}_u=(B_1,B_2,\dots,B_k)$5 into $\mathcal{S}_u=(B_1,B_2,\dots,B_k)$6 uniform intervals and training a separate low-rank update

$\mathcal{S}_u=(B_1,B_2,\dots,B_k)$7

for each interval, with training restricted to timesteps in that slice. This is presented as a concrete “TIDE-style” view in which timesteps are the natural axis of expert disentanglement.

The method has two stages. In the fostering stage, interval-specific experts are trained independently at one or more temporal resolutions. In the assembling stage, the finest-interval expert for timestep $\mathcal{S}_u=(B_1,B_2,\dots,B_k)$8 is taken as a core expert without gating, while experts from coarser interval partitions are treated as context experts with time-dependent gating:

$\mathcal{S}_u=(B_1,B_2,\dots,B_k)$9

The router is deliberately simple,

$B_m\subseteq\mathcal{I}$0

combining a linear projection of the current feature with a learnable timestep embedding. The asymmetry is hard-coded: the finest interval is always active, and only context experts are gated.

This design is evaluated on domain adaptation, post-pretraining, and model distillation across UNet, DiT, MM-DiT, and video backbones. The paper states that any $B_m\subseteq\mathcal{I}$1 significantly outperforms $B_m\subseteq\mathcal{I}$2, directly supporting the value of time-interval-disentangled experts. Representative results include, for domain adaptation on T2I-CompBench, SD2 + vanilla LoRA versus SD2 + TSM improving Color from 66.03 to 75.93 and Texture from 62.87 to 67.44; for PixArt-$B_m\subseteq\mathcal{I}$3, Color improves from 46.53 to 54.66 and Texture from 53.37 to 57.12 (Zhuang et al., 10 Mar 2025). In model distillation on COCO 30K, DMD2 + LoRA yields FID 14.58 and Patch-FID 15.43, whereas DMD2 + TSM yields FID 9.90 and Patch-FID 11.82 with only about 1M extra parameters and 0.1 A100 days extra training. The paper also distinguishes this approach from simple timestep-conditioned LoRA: the gains are attributed not merely to conditioning on $B_m\subseteq\mathcal{I}$4, but to training experts whose gradients come only from their assigned intervals.

4. Sparse temporal experts in interpretable Diffusion Transformers

In “TIDE: Temporal-aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generation,” the acronym is officially expanded as “Temporal-aware Sparse Autoencoders,” not Time-Interval Disentangled Experts (Huang et al., 10 Mar 2025). The paper nevertheless states that the behavior of its sparse units can naturally be interpreted as “time-interval–disentangled experts”: units that specialize in particular semantic features and behave differently across diffusion timesteps. This is an interpretive use rather than the official model name.

The framework is built on PixArt-XL-2-1024-MS and trains one SAE/TIDE per transformer block, for 28 DiT activation layers, using 80K image-text pairs from CC3M at 1024×1024 resolution via latent space. For an activation vector $B_m\subseteq\mathcal{I}$5, the encoder applies ReLU followed by exact TopK sparsification,

$B_m\subseteq\mathcal{I}$6

and the decoder reconstructs $B_m\subseteq\mathcal{I}$7. Temporal awareness is introduced through timestep-dependent modulation,

$B_m\subseteq\mathcal{I}$8

with parameters initialized from the pretrained DiT’s adaptive LayerNorm. The training loss combines reconstruction MSE, cosine similarity loss, temporal consistency loss, sparsity loss, and weight regularization.

The paper argues that direct SAEs on DiT activations converge slowly because activation distributions shift substantially across denoising steps, and that temporal modulation plus temporal consistency regularizes this shift into a more stationary form. The resulting sparse units are then described as behaving like expert features whose activity is strongly tied to time intervals. Empirically, the temporal analysis reports abrupt changes in cosine similarity and MSE at a specific timestep corresponding to contour emergence, marking a transition from a “pre-alignment” phase to a “fine-refinement” phase. Different subsets of TIDE units become active before and after this transition, showing coarse-to-fine roles aligned with time intervals (Huang et al., 10 Mar 2025).

Quantitatively, the abstract reports state-of-the-art reconstruction performance with an MSE of $B_m\subseteq\mathcal{I}$9 and cosine similarity of 0.97. The per-layer evaluation shows that TIDE with sampling attains train MSE $t_m$0 and $t_m$1, with validation MSE $t_m$2 and $t_m$3, outperforming non-temporal SAE variants in generalization and diffusion-loss preservation. Scaling experiments indicate that increasing latent dimension and TopK improves reconstruction toward MSE $t_m$4 and cosine similarity $t_m$5. The learned features are probed as 3D-level, class-level, and semantic-level features, including concepts such as “far,” “near,” “shadow,” “Rococo,” “eagle,” and “beak.” The same sparse latent space is used for sparse activation-guided image editing and style transfer, including erasing Rococo style, scaling age-related units, and replacing latents for architectural transformations.

5. TIDE as a lossless inference system for MoE diffusion LLMs

A second exact use of the phrase appears in “TIDE: Efficient and Lossless MoE Diffusion LLM Inference with I/O-aware Expert Offload,” where TIDE is a system-level optimization for serving MoE diffusion LLMs on a single GPU plus CPU without retraining and without changing model outputs (Chen et al., 19 May 2026). The setting is motivated by MoE dLLMs such as LLaDA2.0, where all tokens in a block are active at every denoising step, so a single step touches many more distinct experts than autoregressive decoding. This makes AR-oriented baselines either I/O-heavy, because experts are migrated every step, or CPU-heavy, because missing experts are executed on CPU while the GPU idles.

The key empirical observation is temporal stability of expert activations within a block. The paper reports mean cosine similarity of expert routing of about 0.985 for adjacent denoising steps and approximately 0.95 even for steps separated by five denoising steps. TIDE uses this stability to introduce an interval-based expert refresh strategy. At refresh steps $t_m$6, it recomputes expert placement on GPU using global hit counts; at skipped steps, it keeps placement fixed and executes GPU-resident experts on GPU while routing misses to CPU. The interval $t_m$7 is chosen by minimizing a mathematical program that balances expert-migration I/O against increased CPU work:

$t_m$8

Under the paper’s drift model, the expected migration cost is approximated by

$t_m$9

The system is explicitly described as lossless because router decisions, expert weights, diffusion schedule, block size, and decoding algorithm are unchanged; only expert placement and movement are altered. Evaluations are conducted on LLaDA2.0-mini and LLaDA2.0-flash using a single NVIDIA A100 40GB or H100 80GB GPU with a 48-core Intel CPU and 1024 GB DDR4 host memory. Throughput is measured in tokens per second on Sanitized MBPP program synthesis. Representative results include LLaDA2.0-mini with generation length 256 and GPU expert budget 128, where throughput improves from 1.74 tok/s for Fiddler and 1.76 tok/s for Mixtral-Offload to 2.36 tok/s for TIDE, and LLaDA2.0-flash with budget 64, where throughput improves from 1.14 and 1.35 tok/s to 1.73 tok/s (Chen et al., 19 May 2026). The paper summarizes the overall gain as up to $i$0 for the mini model and $i$1 for the flash model. It also states that choosing $i$2 by solving the mathematical program is necessary: random intervals underperform optimized ones.

6. Terminological ambiguity, limitations, and outlook

The literature makes clear that “Time-Interval Disentangled Experts” is not a single canonical research object. In recommendation, it denotes a dual-expert model for habit and exploration (Deng et al., 1 May 2026). In diffusion fine-tuning, it is an organizing idea instantiated by interval-specific LoRA experts and multi-scale core-context mixtures (Zhuang et al., 10 Mar 2025). In DiT interpretability, it is an interpretive description of sparse temporally modulated units rather than the official acronym expansion (Huang et al., 10 Mar 2025). In MoE diffusion LLM systems, it denotes an inference runtime that exploits temporal stability of expert usage (Chen et al., 19 May 2026). A separate database line uses “TIDE” for “time intervals by duration and endpoint,” a disk-based interval index under the Increasing Ending Time assumption; that usage is unrelated to expert modeling and should not be conflated with the expert-oriented papers (Wang et al., 22 Jun 2026).

The limitations are correspondingly domain-specific. The recommendation model uses a static item-pattern graph and a simplified Hawkes-inspired kernel rather than full continuous-event modeling (Deng et al., 1 May 2026). The diffusion fine-tuning framework relies on uniform interval partitions and a simple router, even though the paper notes that dynamic intervals and continuous expert indexing are plausible extensions (Zhuang et al., 10 Mar 2025). The interpretability framework requires large latent sizes, per-layer SAEs, and substantial training cost, and it does not implement an explicit mixture-of-experts over time bins (Huang et al., 10 Mar 2025). The inference system is currently limited to within-block similarity, single GPU–CPU settings, and the tested hardware platforms (Chen et al., 19 May 2026).

A plausible implication is that the enduring value of TIDE lies less in a specific module than in a reusable design principle: time intervals can be promoted from conditioning metadata to a structural coordinate for specialization. The papers themselves point toward several next steps, including dynamic sparse activation and explicit time-segment dictionaries in DiT interpretability, dynamic intervals and sparse MoE over time in diffusion fine-tuning, and block-level similarity analysis plus multi-GPU or multi-node deployment in MoE diffusion LLM inference (Huang et al., 10 Mar 2025, Zhuang et al., 10 Mar 2025, Chen et al., 19 May 2026). Under that reading, TIDE is best understood as a cross-domain pattern for temporal factorization whose concrete realization depends on whether the goal is recommendation, adaptation, interpretability, or systems optimization.