Adaptive Mean Shift Algorithm

• Adaptive mean shift is a variant of classical mean shift that adjusts the bandwidth dynamically using sample-point or estimate-point strategies to better capture local density.
• It employs adaptive mechanisms such as full covariance matrices, feature weighting, and generalized distances to accommodate heterogeneous data and complex geometries.
• The algorithm finds practical use in clustering, image segmentation, and tracking by balancing enhanced accuracy with improved computational efficiency.

Searching arXiv for recent and foundational papers on adaptive mean shift. Adaptive mean shift algorithm denotes a family of mean-shift methods in which the quantity governing the shift is no longer fixed globally. In the literature, this includes estimate-point adaptive bandwidth and sample-point adaptive bandwidth, bidirectional combinations of the two, full anisotropic bandwidth matrices learned online, feature-weighted norms for high-dimensional data, generalized distance-based updates, subspace-constrained ridge seeking, and application-specific adaptations of observation models, scales, seeds, or stopping rules (Meng et al., 2017, Sawhney et al., 2014, Chakraborty et al., 2020, Razakarivony et al., 2020, Zhang et al., 2021, Topkaya et al., 2011, You et al., 2021). The unifying idea is to preserve the mode-seeking character of classical mean shift while reducing the sensitivity of a single global isotropic bandwidth to heterogeneous density, anisotropy, irrelevant features, non-Euclidean geometry, or computational cost.

1. Classical formulation and the source of adaptivity

In the standard fixed-bandwidth setting, mean shift is a non-parametric iterative method for locating modes of a kernel density estimate. In one-dimensional notation, the kernel density estimate is

$$\hat{f}_{h,k}(x) = \frac{c_{k,D}}{nh} \sum_{i=1}^{n} k\!\left(\left(\frac{x-x_i}{h}\right)^2\right),$$

with profile $k$, bandwidth $h$, and $g(x)=-k'(x)$. The induced mean-shift update is

$$y_{j+1} = \frac{\sum_{i=1}^n x_i\, g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)} {\sum_{i=1}^n g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)},$$

or equivalently $y_{j+1}=y_j+m_{h,g}(y_j)$ (Ghassabeh, 2014).

The central parameter is the bandwidth. In adaptive-bandwidth literature, the sample-point kernel density estimate is written as

$$\hat f(x) = \frac{1}{n}\sum_{i=1}^n \frac{k\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)}{h_i^d},$$

and the associated mean shift vector is

$$m_{h_i}(x)= \frac{ \sum_{i=1}^{n} h_i^{-(d+2)}\, g\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)x_i } { \sum_{i=1}^{n} h_i^{-(d+2)}\, g\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right) } - x.$$

This formulation makes explicit why adaptation matters: if the bandwidth is too small, the estimate is noisy and the algorithm is easily trapped by spurious local modes; if too large, fine structure is smoothed out and clusters merge (Meng et al., 2017).

The same issue appears in tracking. A traditional color-based mean-shift tracker is described as simple and efficient, but “the lack of dynamism in its target model makes it unsuitable for tracking objects which have changes in their sizes and shapes,” which motivates adaptive model and bandwidth variants (Mohammadi et al., 2012). Adaptive mean shift is therefore best understood as a response to the bandwidth, geometry, and representation assumptions built into the classical isotropic update.

2. Adaptive bandwidth and neighborhood design

Several non-equivalent adaptive mechanisms modify the neighborhood over which mean shift averages are computed.

Variant	Adaptive quantity	Representative source
EAMS / SAMS / BAMS	Bandwidth attached to estimate point, sample point, or both	(Meng et al., 2017)
AAAMS	Full local bandwidth matrix learned online from trajectory sets	(Sawhney et al., 2014)
Local-cluster-cardinality method	Local cluster size, bandwidth, and kernel radius threshold	(Pepin, 17 Aug 2025)

In the estimate-point adaptive bandwidth strategy (EAMS), the bandwidth is attached to the current iterate $x$, and with $k$-nearest-neighbor bandwidths and the Epanechnikov kernel the mean shift vector reduces to

$k$0

In the sample-point adaptive bandwidth strategy (SAMS), each sample contributes according to its own bandwidth: $k$1 The bidirectional adaptive bandwidth mean shift strategy (BAMS) combines these viewpoints through a signed contrast,

$k$2

and in practice stabilizes it with

$k$3

The stated purpose is to combine the stability of EAMS with the bias-reduction intuition of SAMS while improving the ability to escape from local maximum density regions (Meng et al., 2017).

Anisotropic Agglomerative Adaptive Mean-Shift (AAAMS) replaces scalar bandwidths by a full local covariance matrix $k$4 for each evolving cluster $k$5. Each point starts with $k$6, and once sufficient evidence has accumulated, $k$7 is estimated from the cluster’s trajectory set $k$8 rather than from raw member locations: $k$9 AAAMS couples this online bandwidth estimation with agglomerative merging of trajectories that appear to be heading to the same mode, so cluster growth and local geometry estimation reinforce one another (Sawhney et al., 2014).

A more recent bandwidth-and-neighborhood design estimates the local cluster cardinality from the ordered distances from a point to all others. For a sorted distance prefix $h$0, it defines

$h$1

with $h$2 and $h$3 the prefix mean and variance, and takes the minimizing rank as a local cluster-size estimate. During mean shift, this estimate is used to adaptively set both the bandwidth and a hard kernel radius threshold,

$h$4

with a truncated Gaussian kernel

$h$5

The method is explicitly designed for datasets with varying local scale and cluster cardinality (Pepin, 17 Aug 2025).

3. Learned geometry, feature weighting, and generalized distances

Adaptive mean shift is not restricted to bandwidth selection. A distinct line of work modifies the geometry in which neighborhood structure is measured.

Weighted Blurring Mean Shift (WBMS) addresses the case where only a subset of features is informative in high dimension. It replaces the ordinary Euclidean norm by a feature-weighted norm

$h$6

and updates the blurred points by

$h$7

The feature weights are learned jointly with the shifts: $h$8 This is a global, iteration-wise feature reweighting rather than a per-point bandwidth adaptation (Chakraborty et al., 2020).

Generalized mean shift replaces the squared Euclidean argument of the kernel by an arbitrary distance-like function $h$9: $g(x)=-k'(x)$0 For the triangular profile,

$g(x)=-k'(x)$1

the update becomes an active-set minimization

$g(x)=-k'(x)$2

This makes the method metric-adapted rather than bandwidth-adapted in the classical sense. In the same framework, Wasserstein Median Shift reported ARI $g(x)=-k'(x)$3 versus Mean Shift $g(x)=-k'(x)$4 on a synthetic histogram dataset and ARI $g(x)=-k'(x)$5 versus Mean Shift $g(x)=-k'(x)$6 on a real aeronautical histogram dataset, illustrating the practical importance of adapting the notion of locality to the data geometry (Razakarivony et al., 2020).

A recurrent misconception is that adaptive mean shift always means variable bandwidth. The literature above shows a broader taxonomy: the adaptive quantity may be a full covariance, a feature-weight vector, or an underlying distance.

4. Subspace, functional, and regression generalizations

Another major branch of the literature changes the target of mean shift itself. Subspace Constrained Mean Shift (SCMS) does not seek modes but density ridges. For a $g(x)=-k'(x)$7-dimensional ridge in $g(x)=-k'(x)$8,

$g(x)=-k'(x)$9

where $$y_{j+1} = \frac{\sum_{i=1}^n x_i\, g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)} {\sum_{i=1}^n g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)},$$0 spans the Hessian-normal eigenspace. The SCMS update is

$$y_{j+1} = \frac{\sum_{i=1}^n x_i\, g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)} {\sum_{i=1}^n g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)},$$1

with adaptive step size

$$y_{j+1} = \frac{\sum_{i=1}^n x_i\, g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)} {\sum_{i=1}^n g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)},$$2

The paper’s central interpretation is that SCMS is a subspace constrained gradient ascent algorithm with an adaptive step size. The same framework is extended from Euclidean to directional data on the sphere $$y_{j+1} = \frac{\sum_{i=1}^n x_i\, g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)} {\sum_{i=1}^n g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)},$$3, where gradients and Hessians become Riemannian objects and the iterate is renormalized back to the sphere after each projected step (Zhang et al., 2021).

Functional mean shift extends the algorithm from vector-valued observations to Hilbert-space-valued data, chiefly $$y_{j+1} = \frac{\sum_{i=1}^n x_i\, g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)} {\sum_{i=1}^n g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)},$$4. The functional mean shift operator is

$$y_{j+1} = \frac{\sum_{i=1}^n x_i\, g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)} {\sum_{i=1}^n g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)},$$5

and the blurring functional mean shift update is

$$y_{j+1} = \frac{\sum_{i=1}^n x_i\, g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)} {\sum_{i=1}^n g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)},$$6

A stochastic scalable variant replaces the full dataset at each step by random partitions, reducing the per-iteration cost from $$y_{j+1} = \frac{\sum_{i=1}^n x_i\, g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)} {\sum_{i=1}^n g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)},$$7 to $$y_{j+1} = \frac{\sum_{i=1}^n x_i\, g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)} {\sum_{i=1}^n g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)},$$8 when $$y_{j+1} = \frac{\sum_{i=1}^n x_i\, g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)} {\sum_{i=1}^n g\!\left(\left(\frac{y_j-x_i}{h}\right)^2\right)},$$9 subsets are used (Chen et al., 19 Jul 2025).

Regression-mode-seeking methods are mean-shift-inspired rather than standard mean shift. Instead of density modes, they estimate local modes of a regression surface $y_{j+1}=y_j+m_{h,g}(y_j)$0. After transforming the response so that $y_{j+1}=y_j+m_{h,g}(y_j)$1, the regression mean shift vector is

$y_{j+1}=y_j+m_{h,g}(y_j)$2

with

$y_{j+1}=y_j+m_{h,g}(y_j)$3

This generates basins of attraction in the input space, but the target is a transformed regression function rather than a density (Qiao et al., 2021).

5. Convergence, stability, and theoretical tensions

The convergence theory of adaptive mean shift is fragmented because different papers study different objects. In one dimension, standard fixed-bandwidth mean shift with a convex, differentiable, strictly decreasing profile $y_{j+1}=y_j+m_{h,g}(y_j)$4 and strictly decreasing $y_{j+1}=y_j+m_{h,g}(y_j)$5 produces a bounded sequence that is eventually monotone and therefore convergent (Ghassabeh, 2014). In the generalized triangular-kernel setting, the active-set structure is finite, and the resulting sequence is stationary after a finite number of steps regardless of the distance used (Razakarivony et al., 2020).

By contrast, a PDE-based analysis of mean-shift-type dynamic clustering models the velocity field as

$y_{j+1}=y_j+m_{h,g}(y_j)$6

which turns the conservation law into the anti-diffusion equation

$y_{j+1}=y_j+m_{h,g}(y_j)$7

Under this analysis, unsupervised mean shift is intrinsically anti-diffusive, entropy decreases,

$y_{j+1}=y_j+m_{h,g}(y_j)$8

and correct convergence can occur only when the evolving density is a mixture of normal distributions with equal variances and no dependence structure. The proposed remedy is to introduce a supervision function $y_{j+1}=y_j+m_{h,g}(y_j)$9 through

$$\hat f(x) = \frac{1}{n}\sum_{i=1}^n \frac{k\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)}{h_i^d},$$0

(Wang et al., 2012).

These results are not interchangeable. They concern, respectively, one-dimensional fixed-bandwidth iterates, combinatorial active-set updates under triangular kernels, and continuum density evolution. A plausible implication is that “convergence of adaptive mean shift” is model-specific rather than uniform across the literature.

Later work provides stronger guarantees for specific adaptive variants. SCMS attains local linear convergence because it can be written as subspace constrained gradient ascent with an adaptive step size; in Euclidean space the iterates satisfy a geometric bound of the form

$$\hat f(x) = \frac{1}{n}\sum_{i=1}^n \frac{k\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)}{h_i^d},$$1

under eigengap and regularity conditions (Zhang et al., 2021). Functional blurring mean shift proves monotonic increase of the average surrogate density, convergence of each trajectory in $$\hat f(x) = \frac{1}{n}\sum_{i=1}^n \frac{k\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)}{h_i^d},$$2, and asymptotic local-mode character of each limit point (Chen et al., 19 Jul 2025). WBMS establishes finite-iteration satisfaction of its stopping criterion and, under a Gaussian population model, a convergence rate of at least cubic order for the coordinatewise spread parameters (Chakraborty et al., 2020).

6. Applications, acceleration, and practical interpretation

In visual tracking, adaptation often concerns the observation model rather than the optimizer itself. A confidence-based dynamic classifier combination tracker generates the mean-shift weight image by fusing two classifiers, one coming from a background modeling method, and computes their contributions dynamically from correlations between weight-image histograms and an ideal weight image from the previous frame (Topkaya et al., 2011). A later color tracker explicitly advertises a three-phase algorithm with adaptive model and bandwidth to handle changes in target size and shape (Mohammadi et al., 2012).

In systems papers, “adaptive mean shift” frequently refers to adaptive computation rather than adaptive bandwidth. MeanShift++ replaces the exact neighbor search by a density-weighted mean over adjacent grid cells,

$$\hat f(x) = \frac{1}{n}\sum_{i=1}^n \frac{k\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)}{h_i^d},$$3

reducing per-iteration runtime from $$\hat f(x) = \frac{1}{n}\sum_{i=1}^n \frac{k\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)}{h_i^d},$$4 to $$\hat f(x) = \frac{1}{n}\sum_{i=1}^n \frac{k\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)}{h_i^d},$$5 and reporting speedups of more than $$\hat f(x) = \frac{1}{n}\sum_{i=1}^n \frac{k\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)}{h_i^d},$$6 for image segmentation with nearly identical segmentations to standard mean shift (Jang et al., 2021). A GPU-oriented Euclidean variant adapts the execution strategy by selecting a subset of seeds, stopping early when a fraction of seeds has converged, and changing the seed count according to observed mode count; at $$\hat f(x) = \frac{1}{n}\sum_{i=1}^n \frac{k\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)}{h_i^d},$$7K points it reports $$\hat f(x) = \frac{1}{n}\sum_{i=1}^n \frac{k\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)}{h_i^d},$$8 s versus $$\hat f(x) = \frac{1}{n}\sum_{i=1}^n \frac{k\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)}{h_i^d},$$9 s for a strong GPU baseline and $$m_{h_i}(x)= \frac{ \sum_{i=1}^{n} h_i^{-(d+2)}\, g\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)x_i } { \sum_{i=1}^{n} h_i^{-(d+2)}\, g\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right) } - x.$$0 MB versus $$m_{h_i}(x)= \frac{ \sum_{i=1}^{n} h_i^{-(d+2)}\, g\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right)x_i } { \sum_{i=1}^{n} h_i^{-(d+2)}\, g\!\left(\left\| \frac{x-x_i}{h_i}\right\|^2\right) } - x.$$1 MB of GPU memory (You et al., 2021).

The practical interpretation of adaptive mean shift therefore depends on which component is allowed to vary. In clustering papers, the adaptive quantity is usually bandwidth, covariance, feature weight, or metric. In ridge and regression formulations, it is the projected subspace or the response-weighted ascent field. In tracking and large-scale systems, it may be the target model, the search region, the active seed set, or the stopping policy. The term names a design principle rather than a single canonical algorithm.