Joint Functional Registration & Clustering

• Joint functional registration and clustering is a unified approach that simultaneously aligns functional data and uncovers latent amplitude-based groups.
• It leverages methodologies such as Bayesian hierarchical models, variational optimization, and deep learning to jointly estimate warping functions and cluster templates.
• Empirical evaluations show that joint models enhance reliability and interpretability over sequential methods, especially under strong phase–amplitude confounding.

Joint functional registration and clustering is a class of statistical and computational frameworks that simultaneously address two canonical problems in functional data analysis: aligning (registering) a collection of observed curves or functions to account for phase variability, and uncovering their latent group (cluster) structure based on amplitude variation. This interleaving is motivated by the profound impact that temporal misalignment (phase variation) has on the efficacy of clustering, and conversely, the challenge that unknown cluster structure poses for effective alignment. Modern methods formalize joint registration and clustering as a unified estimation problem, typically via hierarchical Bayesian models, variational formulations, information-theoretic objectives, or, more recently, deep learning–based optimizations. Empirical and theoretical work has demonstrated that joint models can recover both discriminative sub-domains and cluster structure more reliably than two-stage approaches, especially in the presence of strong phase–amplitude confounding.

1. Statistical Problem Formulation and Motivations

Joint functional registration and clustering arises when the observed data consist of $n$ curves $f_i(t)$ defined on a common domain $D\subset\mathbb R$ (for functional data) or as multivariate images (for image clustering), which are subject to both phase distortion through unknown time-warping maps and heterogeneity in amplitude patterns indicative of underlying clusters. The analytic goal is to simultaneously estimate:

• A partition or cluster assignment of the curves (or images), representing commonality in the shape dynamics;
• A set of smooth, strictly increasing warping functions $h_i$ (e.g., affine or spline-based maps) or diffeomorphic flows $\gamma_i$ for each sample $i$;
• Cluster templates, mean shapes, or amplitude factors for each group;
• Additional structure such as sparse discriminative domains or domain weights, and, in some models, covariate effects.

The core challenge is that misalignment can obscure cluster-specific amplitude features, while the choice of cluster labels can influence the inferred alignment. Addressing both in a joint fashion avoids the propagation of errors inherent to sequentially aligning and clustering.

2. Methodological Frameworks

A diversity of modeling paradigms have been developed:

Bayesian Hierarchical Mixture Models with Warping

Dirichlet process mixture models with B-spline or RKHS representations for warping functions, as in Zhang & Telesca (Zhang et al., 2014), formalize the observation model as

$y_i(t) = s_{z_i}(\gamma_i(t)) + \varepsilon_i(t),$

with cluster label $z_i$, monotone warping $\gamma_i$, cluster-specific mean function $s_k$, and residuals. Priors over both amplitude and phase parameters yield a joint posterior, estimated via Metropolis-within-Gibbs MCMC. This approach directly links Bayesian nonparametric clustering to flexible phase variability modeling, and accommodates uncertainty propagation across both structural components.

Variational and Coordinate-Ascent Functionals

Frameworks such as Vitelli's sparse registration-and-clustering (Vitelli, 2019) pose a variational optimization that embeds cluster assignment, alignment, and domain selection:

$$J(C,\{h_i\},w) = \int_D w(t) \, g(\{f_j(h_j(t))\}; C)\, dt,$$

where $w$ is a data-adaptive domain weight, $C$ is a partition, and $g$ measures pointwise between-cluster separation. Hard constraints or sparsity penalties enforce that only domains discriminative for clustering contribute. The problem is solved by block coordinate ascent, alternating alignment, clustering, and weight update steps until convergence.

Factor Analytic Joint Models

Earls & Hooker (Earls et al., 2015) extend registration to a low-rank, joint factor analytic framework,

$$f_i(t) \approx z_{0i} + z_{1i} f_1(t) + z_{2i} f_2(t) + \varepsilon_i(t),$$

where after curve-specific warping, each registered function decomposes into a linear combination of functional factors. Inference proceeds via Adapted Variational Bayes with a post-hoc clustering stage on estimated factor weights.

Gaussian Process Regression Mixtures with Warping

Zeng, Shi, and Kim (Zeng et al., 2017) propose a two-level model: Gaussian Process regression on latent curves with cluster-shared and individual warping, and allocation models—such as multivariate logistic regression—on scalar covariates. Warping functions are parameterized with splines and monotonicity constraints. Maximum likelihood is computed via EM, alternating between soft assignment, warping, and amplitude parameter updates.

Deep Learning and Neural ODE Models

NeuralFLoC (Xiong et al., 3 Feb 2026) introduces an end-to-end neural methodology. Warping maps $\gamma_i$ are parameterized as flows of Neural ODEs, ensuring diffeomorphic mapping and efficiently representing complex phase variability. A SRVF-domain loss captures intra-cluster registration, and a Fourier/spectral clustering module—via DEC-style KL divergence objectives—drives cluster separation. The framework provides universal approximation and consistency guarantees in the joint model.

Information-Theoretic Universal Algorithms

Joint registration and clustering for discrete-valued images (and by extension, functional data) can be cast in a nonparametric, universal framework (Raman et al., 2017). Here, partition information, multiinformation, and mutual information are used to define optimality criteria for both registration and clustering, with multiway information maximization achieving consistency across both dimensions under minimal model assumptions.

3. Theoretical Properties and Guarantees

Methodologies for joint registration and clustering have established several critical theoretical properties:

• W-invariance: For certain warping classes (e.g., affine maps), the weighted $L^2$ metric is invariant under common reparameterizations, preventing artifacts from alignment steps (Vitelli, 2019).
• Well-posedness: Variational problems under mild regularity conditions have unique solutions in each subspace (alignment, clustering, weight selection) and exhibit monotonic improvement under block-coordinate ascent (Vitelli, 2019).
• Consistency: Bayesian models recover true partition and warp assignments in the large-sample limit, provided clusters are well-separated and regularity conditions are met (Zhang et al., 2014, Xiong et al., 3 Feb 2026).
• Universal Approximation: Diffeomorphic warping by Neural ODEs approximates any Lipschitz time warp arbitrarily well given sufficient network capacity (Xiong et al., 3 Feb 2026).
• Information-theoretic optimality: Universal plug-in algorithms using partition/multiinformation achieve error exponents matching the maximum-likelihood Bayesian optimum, and scale logarithmically in the bit cost per sample with the number of images (Raman et al., 2017).

These results collectively ensure that joint models not only perform well empirically, but also possess strong identifiability and optimality guarantees under general conditions.

4. Algorithmic Implementations and Computational Aspects

Algorithmic solutions reflect the interdisciplinary nature of the problem:

• Block coordinate ascent is the mainstay in functional variational formulations, efficiently alternating between warping (phase), clustering (amplitude), and domain selection (sparsity) steps (Vitelli, 2019).
• Gibbs and Metropolis–Hastings MCMC algorithms are employed for full posterior inference in hierarchical Bayesian settings (Zhang et al., 2014), accommodating infinite cluster models via Dirichlet process mixtures.
• Adaptive Variational Bayes with MCMC refinement provides scalable and rapid fitting in hybrid analytic models (Earls et al., 2015).
• Expectation-Maximization (EM) is used for mixture models with latent allocation variables and high-dimensional warping parameters, harnessing low-rank structure and sparse GP covariance updates (Zeng et al., 2017).
• Gradient-based optimization and ODE solvers drive deep learning approaches, with the adjoint method enabling backpropagation through Neural ODE warping modules (Xiong et al., 3 Feb 2026).
• Information-maximizing combinatorial search underlies universal model-free algorithms, though computational complexity scales factorially with data size for fully-exhaustive clustering and registration (Raman et al., 2017); blockwise and hierarchical variants mitigate scaling concerns.

Empirically, modern implementations offer practical performance for sample sizes up to thousands, with vectorized code, mini-batching, and approximate inference often used for larger-scale problems.

5. Empirical Evaluation and Applications

Robust empirical studies demonstrate the effectiveness of joint functional registration and clustering:

• On simulated datasets with artificially induced phase distortion, joint approaches recover true cluster structure with lower misclassification and more accurate region selection than two-stage procedures (Vitelli, 2019, Xiong et al., 3 Feb 2026).
• In growth-curve data (e.g., Berkeley Growth Study), methods such as sparse K-means alignment and DP mixture hierarchies recover biologically meaningful clusters and align phase features to reveal interpretable group differences (Zhang et al., 2014, Vitelli, 2019).
• In multi-dimensional settings (e.g., 2D hyoid bone trajectories), simultaneous registration and clustering using both functional and scalar covariates achieves higher adjusted Rand index and accuracy than function-only or covariate-only approaches (Zeng et al., 2017).
• Deep neural approaches demonstrate scalability, robustness to missing data, irregular sampling, and additive noise, outperforming two-stage and alternative baselines across a variety of publicly available functional datasets (Xiong et al., 3 Feb 2026).
• Information-theoretic algorithms achieve exponentially consistent clustering and registration in universal scenarios with limited or unknown distributional structure, and clarify the sample complexity required for reliable partition recovery (Raman et al., 2017).

A summary of settings and empirical findings is given below:

Methodology	Setting/Dataset	Empirical Result / Metric
Sparse reg.+cluster (Vitelli, 2019)	Simulated, Berkeley Growth	$\leq$1.6% misclassification; correct feature-region selection
DP mixture (Zhang et al., 2014)	Berkeley Growth, gene expression	Highest LPML, lowest MSE, biologically interpretable clusters
Factor-analysis reg.+cluster (Earls et al., 2015)	Juggling cycle data	Tight alignment; clusters via factor weights; lower SLS than alternatives
SRC (GP/EM) (Zeng et al., 2017)	2D trajectories + scalar info	ARI = 0.71 (real), ARI ≥ 0.96 (simulated)
Neural ODE (Xiong et al., 3 Feb 2026)	UCR, large-scale (70K curves)	ACC 0.93+, NMI 0.65+, robust to missing/noisy data
Universal info-theoretic (Raman et al., 2017)	Noisy, permuted images	Exponential consistency, optimal scaling

6. Extensions and Limitations

Joint registration and clustering frameworks extend to various contexts:

• Models for multi-dimensional and multi-modal functional data, with conditional independence or common warping across dimensions (Zeng et al., 2017).
• Incorporation of domain selection or domain weighting for interpretability and adaptation to localized discriminative regions (Vitelli, 2019).
• Integration of scalar or auxiliary covariates via multinomial/logistic allocation models (Zeng et al., 2017).
• Deep neural generalizations permit learning of highly complex, nonlinear warps and clusters, supporting deployment at scale (Xiong et al., 3 Feb 2026).
• Universal nonparametric models provide performance under minimal distributional knowledge, but exhibit computational scaling limitations for large numbers of objects (Raman et al., 2017).

Typical limitations reflect computational demands in high-dimensional/large-scale settings, requirement for sophisticated model selection (e.g., number of clusters, rank), and phase-amplitude identifiability—mitigated by careful model regularization and empirical validation.

A plausible implication is that future work will further integrate uncertainty quantification, richer dependence structures, and scalable optimization, and may address computational complexity in universal settings by leveraging structure or approximate search techniques.