MixFT: Mixture-Based Fine-Tuning for TSFMs

• MixFT is a parameter-efficient fine-tuning method that re-partitions heterogeneous time series data into latent sub-domains using Bayesian Gaussian mixtures.
• It employs one LoRA module per sub-domain, enabling specialized adaptation that mitigates destructive interference present in shared tuning.
• Empirical evaluations show that MixFT consistently outperforms shared and per-dataset methods in zero-shot forecasting accuracy across diverse TSFMs.

MixFT is a parameter-efficient fine-tuning method for adapting time series foundation models (TSFMs) to improve zero-shot forecasting on a new but related domain. It is designed for the setting in which a TSFM has been pretrained broadly, but the target domain of interest is not fully covered by pretraining, so zero-shot performance can suffer. Rather than fine-tuning one LoRA on all available data or one LoRA per original dataset, MixFT re-partitions the fine-tuning data into latent sub-domains using a Bayesian mixture model fitted on TSFM embeddings, and then fine-tunes one LoRA module per inferred sub-domain (Lee et al., 3 Mar 2026).

1. Problem formulation and motivation

MixFT is motivated by a practical adaptation problem for TSFMs. When a practitioner cares about a new domain and has access to a set of related datasets, a standard response is either to fine-tune a single shared LoRA on all datasets or to fine-tune separate modules on each dataset. The paper argues that both strategies can be suboptimal when the available data are internally heterogeneous (Lee et al., 3 Mar 2026).

The “Shared” strategy forces one low-rank module to absorb all patterns present across all fine-tuning datasets. The paper frames this as making the learning problem harder and increasing the chance of destructive interference between heterogeneous patterns. Per-dataset modules are more specialized, but they still assume that dataset boundaries coincide with coherent statistical domains. MixFT is based on the opposite premise: datasets are administrative containers, not necessarily coherent statistical units. A single dataset may contain several sub-domains because of temporal distribution shift, changing regimes, or heterogeneous channels in multivariate series (Lee et al., 3 Mar 2026).

This concern is especially acute for time series. The paper explicitly points to sub-domains arising from seasonal changes, spikiness, changing volatility, and concept shift. A multivariate dataset may contain channels with very different dynamics, and a single time series may alternate between qualitatively different behaviors over time. This suggests that per-dataset fine-tuning can remain too coarse even when it is more modular than shared tuning (Lee et al., 3 Mar 2026).

MixFT therefore replaces dataset-level partitioning with sub-domain-level partitioning. Its central claim is that more homogeneous fine-tuning partitions produce LoRA modules that are more focused on specific sub-domains, and that this improves zero-shot forecasting more reliably than either shared fine-tuning or per-dataset modular tuning (Lee et al., 3 Mar 2026).

2. Statistical formulation and representation space

MixFT operates on context windows extracted from the available fine-tuning datasets. For time series $i$, with context length $L$, the paper forms context windows

$\bm{x}_{i, j}=[\bm{v}_{i, t}]_{t=j}^{j+L},$

and constructs a dataset

$D_i = [\bm{x}_{i, j} ]_{j=1}^{N_i},$

where $N_i = T_i - L$. In the paper’s notation, $M$ is the number of fine-tuning datasets, $L$ the context length, and $H$ the forecast horizon (Lee et al., 3 Mar 2026).

Each context window is then embedded by the frozen foundation model: $\bm{z}_{i,j} = \text{TSFM}_{embed}(\bm{x}_{i,j}).$ The appendix states that the feature vector used for the mixture model is formed by taking the mean of each TSFM token representation. The clustering stage therefore operates in learned representation space rather than on raw time-domain summary statistics (Lee et al., 3 Mar 2026).

The latent sub-domains are modeled with a Bayesian Gaussian mixture model with diagonal covariance: $$\bm{\mu}_k, \bm{\Sigma}_k \sim \text{NIDW}(\bm{m}, \kappa, \nu, \bm{W}) \;\;\text{for } k \in [1,\ldots,K],$$

$L$0

$L$1

$L$2

Here $L$3 is the number of mixture components, $L$4 is the latent component assignment, and $L$5 is a normal-inverse-diagonal-Wishart prior chosen so that each $L$6 is diagonal. The use of diagonal covariance is explicitly justified as reducing parameter count and overfitting risk (Lee et al., 3 Mar 2026).

The model is fitted with mean-field variational inference. The variational posterior is written as

$L$7

$L$8

The paper states that the latent variables $L$9 are treated with a mean-field factorization assumption independent of $\bm{x}_{i, j}=[\bm{v}_{i, t}]_{t=j}^{j+L},$0, $\bm{x}_{i, j}=[\bm{v}_{i, t}]_{t=j}^{j+L},$1, and $\bm{x}_{i, j}=[\bm{v}_{i, t}]_{t=j}^{j+L},$2 in the variational distribution. The stated reason for using variational Bayes rather than maximum-likelihood clustering is greater stability and less dependence on initialization (Lee et al., 3 Mar 2026).

3. Re-partitioning, LoRA specialization, and inference

After fitting the Bayesian GMM, MixFT assigns every context window to a learned sub-domain using the posterior predictive distribution. The assignments are hard: $\bm{x}_{i, j}=[\bm{v}_{i, t}]_{t=j}^{j+L},$3 where

$\bm{x}_{i, j}=[\bm{v}_{i, t}]_{t=j}^{j+L},$4

The new partitions are then defined as

$\bm{x}_{i, j}=[\bm{v}_{i, t}]_{t=j}^{j+L},$5

These $\bm{x}_{i, j}=[\bm{v}_{i, t}]_{t=j}^{j+L},$6 are the sub-domain partitions used for separate fine-tuning (Lee et al., 3 Mar 2026).

This re-division step is the central difference from per-dataset modular tuning. A single original dataset can contribute windows to multiple $\bm{x}_{i, j}=[\bm{v}_{i, t}]_{t=j}^{j+L},$7, and windows from different datasets can be assigned to the same sub-domain if their embeddings are similar. The paper presents this as the key mechanism by which expert modules become more statistically homogeneous than dataset-based experts (Lee et al., 3 Mar 2026).

One LoRA module is fine-tuned per sub-domain. The paper does not give an explicit LoRA formula, but it does specify that all methods use the same LoRA hyperparameters: rank $\bm{x}_{i, j}=[\bm{v}_{i, t}]_{t=j}^{j+L},$8, $\bm{x}_{i, j}=[\bm{v}_{i, t}]_{t=j}^{j+L},$9, dropout $D_i = [\bm{x}_{i, j} ]_{j=1}^{N_i},$0, OLoRA initialization/training trick, AdamW with learning rate $D_i = [\bm{x}_{i, j} ]_{j=1}^{N_i},$1, and batch size $D_i = [\bm{x}_{i, j} ]_{j=1}^{N_i},$2 (Lee et al., 3 Mar 2026).

At test time, MixFT embeds the new context,

$D_i = [\bm{x}_{i, j} ]_{j=1}^{N_i},$3

selects the most likely component,

$D_i = [\bm{x}_{i, j} ]_{j=1}^{N_i},$4

and forecasts with the corresponding LoRA-enhanced TSFM. The paper explicitly states that assignments are hard for training and also hard at inference. It further reports that contexts are usually assigned with high confidence, and that forecasts from unlikely components can be inaccurate because they are effectively out-of-distribution for those experts; this is the reason soft mixture weighting is not used as the default (Lee et al., 3 Mar 2026).

4. End-to-end pipeline and implementation choices

The paper gives a pipeline that can be summarized in four stages. First, each fine-tuning dataset is windowed using context length $D_i = [\bm{x}_{i, j} ]_{j=1}^{N_i},$5. In the reported experiments, $D_i = [\bm{x}_{i, j} ]_{j=1}^{N_i},$6 and forecast horizon $D_i = [\bm{x}_{i, j} ]_{j=1}^{N_i},$7. Second, all windows from all fine-tuning datasets are embedded by the frozen TSFM. Third, a Bayesian GMM is fitted jointly on the union of these embeddings. Fourth, windows are reassigned into learned sub-domain partitions, and one LoRA is fine-tuned on each partition (Lee et al., 3 Mar 2026).

The appendix specifies the priors used for the Bayesian GMM: $D_i = [\bm{x}_{i, j} ]_{j=1}^{N_i},$8

$D_i = [\bm{x}_{i, j} ]_{j=1}^{N_i},$9

$N_i = T_i - L$0

$N_i = T_i - L$1

$N_i = T_i - L$2

The paper uses $N_i = T_i - L$3 components in the main experiments, and notes that $N_i = T_i - L$4 is exactly Shared fine-tuning, so MixFT can be viewed as a strict generalization of Shared (Lee et al., 3 Mar 2026).

The full inference procedure mirrors the training decomposition. A new context is embedded, classified into a sub-domain, and then forecast with the corresponding LoRA-enhanced backbone. The paper reports that inference is slower than Shared because it requires embedding the context and classifying its sub-domain, but also reports that runtime remains below the sampling frequency of the datasets studied, making it operationally acceptable (Lee et al., 3 Mar 2026).

The additional computational cost relative to Shared comes from computing TSFM embeddings for all fine-tuning windows, fitting a Bayesian GMM, training $N_i = T_i - L$5 LoRAs instead of one, and performing lightweight sub-domain classification at inference. The paper states that the extra memory is one LoRA per sub-domain plus negligible GMM parameters, and that this is similar to Poly or MBC and usually less than per-dataset methods when the number of datasets exceeds $N_i = T_i - L$6 (Lee et al., 3 Mar 2026).

5. Empirical evaluation and comparative results

MixFT is evaluated in a zero-shot adaptation setting in which the TSFM backbones were not pretrained on the fine-tuning or evaluation datasets used in the experiments. The fine-tuning datasets are CloudD3, CloudD4, BizITObs-Service, BitBrains Fast Storage (hourly), M4-Hourly, and M4-Weekly. The evaluation datasets are CloudD1, CloudD2, BizITObs-L2C, BizITObs-App, US-Births, M4-Daily, M4-Monthly, M4-Quarterly, ETTh2, and ETTm2 (Lee et al., 3 Mar 2026).

The TSFM backbones are Chronos Bolt (small) and Moirai-1.1-R (small). The main baselines are Base, Shared, $N_i = T_i - L$7-Datasets, Arrow-Datasets, Poly, and MBC. For fairness, MixFT, MBC, and Poly all use $N_i = T_i - L$8 components in the main experiments (Lee et al., 3 Mar 2026).

Evaluation uses mean absolute scaled error (MASE): $N_i = T_i - L$9 where $M$0 is seasonality. Lower is better (Lee et al., 3 Mar 2026).

The paper reports that MixFT performs best overall across both TSFMs. On Chronos Bolt, MixFT achieves the best average rank, $M$1, ahead of Shared ($M$2), Poly ($M$3), $M$4-Datasets ($M$5), and Arrow-Datasets ($M$6). On Moirai-1.1-R, MixFT again has the best average rank, $M$7, ahead of Shared ($M$8) and the remaining baselines (Lee et al., 3 Mar 2026).

Several concrete results on Chronos Bolt illustrate the pattern:

Evaluation dataset	MixFT	Shared	Base
BizITObs-App	0.989	1.060	3.494
US-Births	0.942	0.967	1.014
M4-Quarterly	8.135	8.269	8.178

The gains are not universal. The paper explicitly notes that fine-tuning TSFMs for better zero-shot forecasting is hard, and many fine-tuning methods can perform worse than not fine-tuning at all. There are cases in which the pretrained base model remains strongest. On Chronos Bolt, for example, Base is best on M4-Daily ($M$9 versus MixFT $L$0), ETTh2 ($L$1 versus MixFT $L$2), and ETTm2 ($L$3 versus MixFT $L$4) (Lee et al., 3 Mar 2026).

A controlled experiment fixes the test-time selection mechanism to Arrow for all modular methods. The paper reports that Arrow-Datasets has average absolute degradation relative to Base of $L$5 MASE and relative improvement $L$6, MBC has $L$7 absolute improvement and $L$8, and Arrow-MixFT has $L$9 absolute improvement and $H$0. The stated interpretation is that MixFT’s gains are attributable not only to routing but also to better data partitioning and stronger experts (Lee et al., 3 Mar 2026).

6. Ablations, interpretation, and scope

The paper provides several analyses of the learned sub-domains. A key qualitative result is that the same dataset often contains both mixture components. Some datasets are mostly assigned to one component, whereas others alternate periodically between components. The authors interpret this as evidence that the learned mixtures capture meaningful sub-domains such as seasonal regimes or spike/non-spike behavior. In BizITObs-Service, for example, one identified sub-domain corresponds to contexts whose recent tail is flat, while another corresponds to contexts with recent spikes (Lee et al., 3 Mar 2026).

Assignment confidence is quantified through average entropy,

$H$1

The paper reports average entropies around $H$2 to $H$3 bits, with overall averages $H$4 on fine-tuning datasets and $H$5 on evaluation datasets. Since the maximum entropy for $H$6 is $H$7 bit, the authors interpret these values as showing that MixFT is usually very certain about the correct component; this is offered as support for hard routing (Lee et al., 3 Mar 2026).

The test-time selection ablation compares hard MixFT routing against $H$8-MixFT, Arrow-MixFT, Soft-MixFT, and Ensemble-MixFT. Hard MixFT achieves the best average rank, $H$9, ahead of $\bm{z}_{i,j} = \text{TSFM}_{embed}(\bm{x}_{i,j}).$0-MixFT ($\bm{z}_{i,j} = \text{TSFM}_{embed}(\bm{x}_{i,j}).$1), Ensemble-MixFT ($\bm{z}_{i,j} = \text{TSFM}_{embed}(\bm{x}_{i,j}).$2), Arrow-MixFT ($\bm{z}_{i,j} = \text{TSFM}_{embed}(\bm{x}_{i,j}).$3), and Soft-MixFT ($\bm{z}_{i,j} = \text{TSFM}_{embed}(\bm{x}_{i,j}).$4). The paper explains this by noting that soft combinations can be degraded by poor forecasts from experts for which the context is out-of-distribution (Lee et al., 3 Mar 2026).

For the probabilistic model, the paper compares K-means-based clustering, a topic-model variant, and the Bayesian GMM. Bayesian GMM achieves the best average rank, $\bm{z}_{i,j} = \text{TSFM}_{embed}(\bm{x}_{i,j}).$5, versus topic model $\bm{z}_{i,j} = \text{TSFM}_{embed}(\bm{x}_{i,j}).$6 and K-means $\bm{z}_{i,j} = \text{TSFM}_{embed}(\bm{x}_{i,j}).$7. The stated interpretation is that Bayesian GMM is stable and avoids some overfitting and local-minimum issues of K-means while being simpler and more robust than the topic-model alternative (Lee et al., 3 Mar 2026).

For the number of components, the paper evaluates $\bm{z}_{i,j} = \text{TSFM}_{embed}(\bm{x}_{i,j}).$8. On validation, $\bm{z}_{i,j} = \text{TSFM}_{embed}(\bm{x}_{i,j}).$9 gives the best average rank, $$\bm{\mu}_k, \bm{\Sigma}_k \sim \text{NIDW}(\bm{m}, \kappa, \nu, \bm{W}) \;\;\text{for } k \in [1,\ldots,K],$$0, and on evaluation it also gives the best average rank, $$\bm{\mu}_k, \bm{\Sigma}_k \sim \text{NIDW}(\bm{m}, \kappa, \nu, \bm{W}) \;\;\text{for } k \in [1,\ldots,K],$$1, though some individual datasets favor larger $$\bm{\mu}_k, \bm{\Sigma}_k \sim \text{NIDW}(\bm{m}, \kappa, \nu, \bm{W}) \;\;\text{for } k \in [1,\ldots,K],$$2. This motivates the use of $$\bm{\mu}_k, \bm{\Sigma}_k \sim \text{NIDW}(\bm{m}, \kappa, \nu, \bm{W}) \;\;\text{for } k \in [1,\ldots,K],$$3 in the main setup (Lee et al., 3 Mar 2026).

The paper also delimits the method’s applicability. It states that MixFT is especially appealing when dataset labels are poor proxies for actual time-series regimes. If datasets are already very homogeneous, the advantage over per-dataset experts may be smaller. More fundamentally, the method assumes that the target domain shares latent sub-domains with the fine-tuning data; the paper explicitly says that if this assumption fails, MixFT’s premise breaks down (Lee et al., 3 Mar 2026).

A terminological distinction is necessary because similarly named methods exist in other domains. “Feature Tuning Mixup” (FTM) addresses transferable targeted adversarial attacks and is not a time-series adaptation method (Liang et al., 2024). “Boosting Factorization Machines via Saliency-Guided Mixup” introduces MixFM and SMFM for recommender systems rather than TSFMs (Wu et al., 2022). “Federated Mixture of Experts” proposes FedMix for non-IID federated learning (Reisser et al., 2021). These works are conceptually adjacent only in the broad sense that they use mixtures, mixup, or modular specialization; they are not the same method as MixFT for TSFM adaptation (Lee et al., 3 Mar 2026).