---
title: Weighted Minkowski Distance
url: https://www.emergentmind.com/topics/weighted-minkowski-distance
type: topic
---

# Weighted Minkowski Distance

The weighted Minkowski distance is a generalization of the standard Minkowski (or $L_p$) norms that introduces nonnegative, typically normalized, weight parameters—one per feature dimension or coordinate—to modulate the contribution of each feature when computing inter-point distances. It is central to modern clustering algorithms where features vary significantly in relevance or noise characteristics and is especially prominent in adaptive extensions of $k$-means, both in input and (explicit) kernel-induced feature spaces [2012.02805][2603.25958]. This distance, parametrized by weights $w = (w_1, \ldots, w_d)$ and exponent $p>0$, interpolates between selective and uniform feature usage, enables robust clustering, and can alleviate the “curse of dimensionality” via fractional norms and power-mean aggregation mechanisms.

## 1. Mathematical Definition and Norm Properties

Let $x, y \in \mathbb{R}^d$ and $w=(w_1,\dots,w_d)$ be a vector of nonnegative feature weights summing to one. The weighted Minkowski distance of order $p>0$ is
\[
d_{p,w}(x,y)
= \left( \sum_{j=1}^d w_j\,|x_j-y_j|^p \right)^{1/p}.
\]
In feature spaces induced by explicit (approximate) kernel mappings $\phi: \mathbb{R}^D \to \mathbb{R}^{D'}$, the distance is
\[
D_{w,p}\bigl(\phi(x),\phi(y)\bigr)
= \biggl(\sum_{j=1}^{D'}(w_j)^{p} |\phi_j(x) - \phi_j(y)|^p\biggr)^{1/p}.
\]
This formulation covers the Euclidean ($p=2$, $w_j=1/d$), Manhattan ($p=1$), and Chebyshev ($p\to\infty$) distances as special cases, but allows arbitrary exponents—including fractional $p<1$—and arbitrary nonnegative weights.

Weighted Minkowski distances are not metrics for all $p$; for $p<1$ they are quasinorms (sometimes termed “fractional norms”), lacking convexity but providing useful discriminatory power in high dimension [2012.02805].

## 2. Weight and Exponent Learning Mechanisms

Feature relevance is encoded through the $w_j$ weights. In clustering formulations such as Minkowski-weighted $k$-means (mwk-means) and its kernel variants, each cluster $k$ may be endowed with its own weight vector $w^{(k)} = (w_{k1}, \dots, w_{kD'})$. These weights are adapted in closed form in each iteration:
\[
w_{kj} = \frac{1}{\displaystyle\sum_{u=1}^{D'} (V_{kj}/V_{ku})^{1/(p-1)}}
\]
where $V_{kj} = \sum_{i:\,\delta_{ik}=1}|\phi_j(x_i) - m_{kj}|^p$ is within-cluster dispersion of feature $j$ [2012.02805]. Thus, features with lower dispersion receive more weight, effectively downweighting noisy or uninformative dimensions.

The Minkowski exponent $p$ governs the selectivity versus uniformity of feature usage. Empirical or data-driven selection is achieved by maximizing the contrast $E[\max_i \|x_i\|_p - \min_i \|x_i\|_p]$ or maximizing relative variance $RV_{F,p} = \sqrt{\mathrm{Var}(\|x\|_p)}/E[\|x\|_p]$, both of which often peak for $p<2$—sometimes even $p<1$ [2012.02805].

## 3. The Power-Mean Aggregation and Feature Selectivity

The weighted Minkowski clustering objective can be reformulated as a sum of power means of within-cluster dispersions:
\[
J_p(w, C) = d^{-(p-1)} \sum_{i=1}^{k} M_r(D_{i1}, \ldots, D_{id})
\]
where $D_{ij}$ is the $p$-dispersion of feature $j$ in cluster $i$, $M_r$ is the power mean of order $r = -1/(p-1)$, and $r<0$ [2603.25958].

This structure leads to:
- Low $p$ ($p \to 1^+$): $M_r$ approaches $\min_j D_{ij}$, making the objective focus on compact (informative) features, favoring sparse, selective usage.
- Large $p$ ($p \to \infty$): $M_r$ approaches the geometric or uniform mean, yielding nearly equal weighting of all features.

Suppression of noisy or high-dispersion features is explicit: for large $D_{ij}$ relative to other $D_{iu}$, $w_{ij}$ tends to zero as a power law $w_{ij}/w_{iu} = (D_{iu}/D_{ij})^{1/(p-1)}$ [2603.25958].

## 4. Alternating Optimization in $k$-Means Frameworks

Clustering algorithms based on weighted Minkowski distances typically alternate three steps:
1. **Assignment:** Points assigned to nearest cluster using $d_{p,w}$.
2. **Center Update:** Cluster centers recomputed as Minkowski centers, minimizing $\sum_{x \in C_i} |x_j - z|^p$ for each $j$.
3. **Weight Update:** Weights updated via $w_{ij} \propto D_{ij}^{-1/(p-1)}$, normalized to sum to one.

Each step strictly decreases the objective, guaranteeing convergence in a finite number of steps due to the finiteness of possible partitions [2012.02805][2603.25958].

When implemented in explicit kernel feature spaces, the algorithm bypasses $O(N^2)$ kernel matrix computation by employing finite-dimensional mappings, thus scaling linearly in both $N$ and $D'$ for distance and center computations, and enabling easy feature-weight learning [2012.02805].

## 5. Fractional Norms and Distance Concentration Effects

Distance concentration—the phenomenon wherein distances between points in high dimensions become nearly identical—can severely degrade clustering reliability. For $p \geq 2$, the expected contrast $E[\max_i \|x_i\|_p - \min_i \|x_i\|_p]$ decays as dimension increases, leading to clustering ambiguity.

By contrast, setting $p < 2$ (especially $p < 1$) in the weighted Minkowski norm reverses this trend: the contrast grows as $D^{1/p - 1/2}$, enhancing discrimination between clusters in high-dimensional regimes [2012.02805]. This adjustable discrimination is a principal reason for the revival of non-Euclidean, and fractional, norms in modern unsupervised learning.

## 6. Theoretical Guarantees and Suppression of Noisy Features

The power-mean structure allows precise theoretical bounds:
\[
\min_j D_{ij} \leq M_r(D_{i1},\ldots,D_{id}) \leq \left(\prod_{j=1}^d D_{ij}\right)^{1/d}
\]
and for the weighted clustering objective,
\[
\frac{1}{d^{p-1}} \sum_{i=1}^{k} \min_j D_{ij}
\leq J_p(w,C)
\leq \frac{1}{d^{p-1}} \sum_{i=1}^k \left( \prod_j D_{ij} \right)^{1/d}
\]
[2603.25958]. As $p \to 1^+$, noisy features are assigned vanishingly small weights; as $p\to\infty$, weights approach equality, sacrificing selective suppression. Therefore, weighted Minkowski clustering provides explicit, parameter-controlled guarantees on noise and outlier feature suppression.

## 7. Computational Benefits via Explicit Feature Mapping

In kernelized $k$-means, direct computation of the Gram matrix for $N$ samples incurs $O(N^2)$ memory and time per iteration. Replacing implicit kernels with explicit, approximate finite-dimensional feature maps (e.g., via Fourier series sampling for homogeneous additive kernels) yields embeddings $\phi(x) \in \mathbb{R}^{D'}$ that satisfy $\phi(x)^\top \phi(y) \approx K(x,y)$, reducing memory to $O(N D')$ and per-distance computation to $O(D')$ [2012.02805]. This enables scalability to large datasets while allowing for explicit feature weighting and integration of advanced $k$-means extensions in the transformed space.

| Algorithm           | Memory Complexity | Distance Computation | Weight Learning |
|---------------------|------------------|---------------------|-----------------|
| Kernel $k$-means    | $O(N^2)$         | $O(N)$              | Indirect        |
| Explicit KMWK-means | $O(N D')$        | $O(D')$             | Direct (vector) |

This choice yields efficient clustering, robust selectivity through the weight/exponent mechanism, and practical applicability to nonlinearly separable or high-dimensional data [2012.02805].

---

Weighted Minkowski distances, through adaptable exponents and learned weights, offer a unified and theoretically grounded extension to classical distance measures—supporting robust, interpretable, and scalable unsupervised learning both in the original and kernel-embedded feature domains.

Source: https://www.emergentmind.com/topics/weighted-minkowski-distance