Robust State-Conditional Feature-Weighted Jump Models for Temporal Clustering

Abstract: We propose a robust feature-weighted jump model for time-dependent clustering. A penalty is used to encourage smoothness of transitions over time, while robustness is achieved through the use of a Tukey's biweight loss function. An additional parameter controls the variability of feature weights across states, allowing the model to assign state-specific relevance to each feature. We illustrate in simulation how the method accurately recovers the true cluster sequence and reliably identifies relevant features, outperforming competing approaches, particularly in the presence of outliers. We conclude with two empirical applications, one on the number of conflict-related homicides in Kosovo in the period 1998-2000, and another on macroeconomic performance of twelve European countries in the period 1949-2024.

• The paper introduces FWJM, a model that integrates state-conditional feature weighting with temporal clustering to improve latent regime detection.
• It employs a modified Gower distance and Tukey’s biweight loss to robustly handle noise, while using dynamic programming for state sequence updates.
• Empirical and simulation studies show superior ARI and BAC scores, confirming the model’s effectiveness in noisy, high-dimensional data contexts.

Robust State-Conditional Feature-Weighted Jump Models for Temporal Clustering

Motivation and Model Formulation

The paper introduces a robust temporal clustering framework, the Feature-Weighted Jump Model (FWJM), for multivariate time series, aiming to identify latent states/regimes and state-dependent variable importance. It extends classical regime-switching and jump models by allowing feature weights to vary across states and incorporates robustness to noise and outliers. The model builds on dissimilarity-based clustering, specifically leveraging the COSA methodology, and adapts it for time-dependent structure by penalizing state transitions and using robust loss for dissimilarity computation.

Given a $T \times P$ multivariate time series $\mathbf{Y}$, FWJM assumes latent state assignments $\mathbf{s}$ and state-specific feature weight matrix $\mathbf{W}$. Dissimilarities between observations are calculated via a modified Gower distance, separately handling continuous, categorical, and ordinal features; continuous feature-wise dissimilarities are robustified using Tukey’s biweight loss to downweight outliers. The per-state feature relevance is encoded in $\mathbf{W}$ and is learned as part of the clustering process.

The objective function integrates three terms:

• A within-state dispersion measure weighted by $\mathbf{W}$, promoting cohesive clusters and enabling explicit feature weighting in state assignment.
• An entropy-based penalty regularizing the feature weight distribution within each state, with hyperparameter $\zeta$ controlling uniformity versus concentration.
• A temporal persistence penalty with hyperparameter $\lambda$, discouraging excessive state transitions and promoting temporal coherence.

Estimation Algorithm

FWJM estimation proceeds via blockwise alternating optimization:

1. Medoid Update: For each cluster, select observations minimizing weighted within-cluster dissimilarities as medoids.
2. State Sequence Update: Employ dynamic programming to minimize the sum of cluster assignment costs and transition penalties, leveraging recursion analogous to Viterbi-style updates.
3. Feature Weight Update: Feature weights per state are updated in closed form by minimizing the objective subject to simplex constraints, yielding normalized exponentials of weighted dispersion per feature.

Due to non-convexity of the objective, a multi-start strategy is adopted for initialization, and hyperparameters $(\lambda, \zeta)$ can be tuned via cross-validation, cluster validation indices (e.g., Silhouette index), or task-specific criteria.

Simulation Studies

The paper reports comprehensive simulation benchmarks across four scenarios, varying sample size ($T$), feature dimension ($\mathbf{Y}$0), and state separation:

• Scenario A: Large $\mathbf{Y}$1, small $\mathbf{Y}$2 (classical regime-switching).
• Scenario B: Large $\mathbf{Y}$3, small $\mathbf{Y}$4 (high-dimensional, small-sample).
• Scenario C: Large $\mathbf{Y}$5, large $\mathbf{Y}$6 (high-dimensional, long series).
• Scenario D: Small $\mathbf{Y}$7, small $\mathbf{Y}$8 (difficult, sparse setting).

Latent states are sampled from Student-$\mathbf{Y}$9 HMMs, with state-wise informative/non-informative features and additional uniform noise features. Robustness is evaluated with and without artificial outlier contamination.

FWJM consistently achieves near-perfect ARI and BAC scores in uncontaminated settings and outperforms non-robust alternatives and other methods under outlier contamination, especially in large-$\mathbf{s}$0 scenarios. Feature weights reliably recover true state-dependent relevance structures, confirming the model's ability to distinguish relevant and irrelevant features. Sensitivity analyses demonstrate that optimal $\mathbf{s}$1 scales with temporal length, and optimal $\mathbf{s}$2 depends on feature informativity and sparsity.

Empirical Applications

Two empirical case studies validate FWJM:

• Kosovo Conflict Homicides (1998-2000): Daily counts of homicides disaggregated by gender and role exhibit distinct temporal regimes of violence. FWJM sharply discriminates high-risk versus low-risk periods and reveals regime-specific feature weighting, e.g., periods dominated by civilian female killings. The robust loss is indispensable, as classical loss functions lead to pathological weighting and distorted centroids in the presence of extreme outliers.
• Eurozone Macroeconomic Indicators (1949-2024): Multivariate indicators across 12 countries cluster into regimes with heterogeneous country/indicator importance. FWJM produces highly variable state-wise weights and identifies macroeconomic periods characterized by specific country-indicator combinations, demonstrating scalability to moderate $\mathbf{s}$3, high $\mathbf{s}$4 settings and effective regularization.

Practical and Theoretical Implications

FWJM addresses critical limitations of classical temporal clustering approaches, notably:

• State-conditional variable weighting: Explicit estimation of state-wise feature importance enables interpretable and actionable cluster assignments in high-dimensional, temporally structured data.
• Robustness: Use of Tukey's biweight loss for dissimilarity calculation ensures resilience to heavy-tailed distributions and outlier contamination.
• Flexibility: Distribution-free formulation accommodates mixed-type data and does not impose parametric assumptions.

Practical implications include improved identification of temporal regimes in domains where feature relevance varies across states (e.g., finance, macroeconomics, conflict analysis), robustness to real-world data imperfections, and enhanced interpretability for downstream tasks.

Theoretical implications arise from distribution-free state estimation, explicit feature weighting, and the entropic regularization. FWJM constitutes an advance in unsupervised state-dependent variable importance estimation and suggests future directions for scalable algorithms (as current pairwise dissimilarity computation scales poorly in large $\mathbf{s}$5), deeper integration with variable selection, and generalization to fuzzy, soft clustering and spatio-temporal domains.

Conclusion

FWJM provides a robust, interpretable framework for temporally coherent, state-conditional clustering with feature weighting in multivariate time series. It demonstrates superior accuracy in state recovery and feature relevance identification even under outlier contamination. The model's flexibility and robustness are validated by simulation and empirical studies. Future developments should focus on algorithmic scalability, integration with variable selection, and further extension to high-dimensional temporal and spatio-temporal clustering tasks (2606.13146).