Weiszfeld-Type Algorithm

Updated 6 October 2025

Weiszfeld-type algorithms are iterative fixed-point schemes designed to minimize weighted sums of distances, incorporating constraints and non-smooth objectives.
They generalize the classical Weiszfeld algorithm by integrating projection steps and modifications to handle singularities and ensure monotonic cost descent.
Their robust convergence properties make them applicable in location science, robust statistical estimation, and distributed optimization scenarios.

A Weiszfeld-type algorithm is an iterative fixed-point scheme, conceptually descended from the classical Weiszfeld algorithm for the unconstrained Fermat–Weber location problem, but generalized to handle additional constraints, nonsmoothness, or other modifications to the objective that preserve the core weighted average structure. Such algorithms arise in continuous location theory, nonconvex and nonsmooth optimization, robust statistical estimation, and several applied fields where one seeks a point minimizing a (possibly weighted) sum of distances to given points, sets, or under certain model constraints. Weiszfeld-type algorithms maintain the essential property of monotonic cost descent and, under suitable conditions, convergence to exact or stationary solutions.

1. Classical Weiszfeld Algorithm and Generalization

The prototypical Weiszfeld algorithm solves the unconstrained Weber problem: for noncollinear points $a_1, \ldots, a_m \in \mathbb{R}^n$ and positive weights $w_1, \ldots, w_m$ , find $x^* \in \mathbb{R}^n$ minimizing

$f(x) = \sum_{i=1}^m w_i \|x - a_i\|$

The fixed-point equation is

$x = \frac{\sum_{i=1}^m \frac{w_i}{\|x - a_i\|} a_i}{\sum_{i=1}^m \frac{w_i}{\|x - a_i\|}}$

Classical Weiszfeld iterates are defined by applying this map until convergence, typically initialized away from the anchor points. Global convergence of the sequence to the unique minimizer is guaranteed, except for an exceptional set of initial points of Lebesgue measure zero.

Weiszfeld-type algorithms extend this framework in several directions. They are defined for more general cost functions—allowing constraints, alternative geometries, and projections—while retaining descent and fixed-point properties. In particular, the studies by Vardi and Zhang (2001) introduced refined treatments of non-differentiability issues when an iterate coincides with a vertex, which is central to most robust Weiszfeld-type methods (Torres, 2012).

2. Constrained Weiszfeld-Type Algorithms: Methodological Framework

Constrained Weiszfeld-type algorithms target optimization problems of the form

$\min_{x \in \mathcal{C}} \;\; f(x) = \sum_{i=1}^m w_i \|x - a_i\|$

where $\mathcal{C} \subset \mathbb{R}^n$ is a nonempty closed convex set. The essential adaptation is the incorporation of a projection onto $\mathcal{C}$ at each iteration:

$x_{k+1} = P_{\mathcal{C}} \left( \frac{\sum_{i=1}^m \frac{w_i}{\|x_k - a_i\|} a_i}{\sum_{i=1}^m \frac{w_i}{\|x_k - a_i\|}} \right), \quad x_0 \in \mathcal{C}, \; x_0 \notin \{a_1, \ldots, a_m\}$

where $P_{\mathcal{C}}$ denotes the metric projection operator. In the case of infinite-dimensional real Hilbert spaces, the same structure applies, and the projection ensures iterates remain feasible (Nguyen, 2018).

An alternative structural approach, relevant when the objective admits nonsmooth or nonconvex components—e.g., the sum of set distances or a difference-of-convex structure (d.c.)—is to combine the Weiszfeld mapping with higher-level decomposition strategies such as DCA (Pham Dinh–Le Thi algorithm) (An et al., 2014). Here, at each DCA iteration, one solves a convex subproblem with a generalized Weiszfeld update (involving projections and possibly quadratic or regularization terms), ensuring a sequence of feasible, descent-producing iterates.

3. Desingularization and Robustness at Non-Differentiable Points

A recurring difficulty is the presence of singularities when an iterate lands precisely on a datum (e.g., $x_k = a_j$ ). The denominator in the classical update vanishes and the gradient is not classically defined. Two principal strategies address this:

Modified Iteration: The mapping is locally redefined to “skip” over the singular point (as in Vardi and Zhang, or via one-sided derivatives and subdifferential calculus).
Desingularization Subgradient Method: For generalized cost functions with $1 \le q < 2$ ,

$C_q(y) = \sum_i \xi_i \|y - x_i\|^q,$

the “singular” term is omitted at $y = x_k$ and a subgradient from the nonsingular part provides a valid descent direction:

$y_{p+1} = x_k - \lambda \nabla D_q(x_k)$

with $D_q$ the sum excluding the singular term (Lai et al., 11 May 2024). The step size $\lambda$ is selected so that $C_q$ decreases. This approach ensures that iterates can escape singularities and that convergence is preserved.

4. Theoretical Properties: Descent, Fixed Points, and Convergence

Weiszfeld-type algorithms, under non-collinearity of the anchors and strict convexity of the objective, generate monotonic, cost-decreasing sequences. The following are typical properties established via convex analysis and fixed-point theorems:

Descent Property: Each non-fixed-point iteration results in

$f(x_{k+1}) < f(x_k)$

Feasibility: Projection steps guarantee that $x_k \in \mathcal{C}$ for all $k$ .
Accumulation and Optimality: Every accumulation point is a fixed point of the update mapping and, under regularity (strict convexity, compactness), is the unique minimizer.
Handling Singularities: The algorithm is well defined even when iterates coincide with data points, due to modifications described above.
Convergence Rate: For $1 < q < 2$, desingularized Weiszfeld-type schemes can exhibit superlinear convergence when the minimizer is a singular point (Lai et al., 11 May 2024). For standard $q=1$ , the convergence is linear.

Numerical experiments confirm these properties, demonstrating robust performance and convergence under perturbations, and favoring intermediate values $1 < q < 2$ for certain robust estimation tasks.

5. Algorithmic Realizations and Practical Implementation

Generic structure of a projection-based Weiszfeld-type iteration:

Initialize $x_0 \in \mathcal{C}$ , $x_0 \neq a_i$ .
For $k=0,1,2,\ldots$ : a. Compute the Weiszfeld update (possibly generalized or regularized). b. If iterate lands on a singularity, invoke desingularized subgradient or modified update. c. Project onto convex feasible set: $x_{k+1} = P_{\mathcal{C}}( \text{update} )$ . d. Terminate if descent is below a set tolerance or fixed-point criterion is met.

For DCA/Weiszfeld hybrids (An et al., 2014), at each meta-iteration, a convexified (possibly regularized) subproblem is solved by a projected Weiszfeld iteration with respect to a fixed linearization of the concave component.

6. Applications and Extensions

Weiszfeld-type algorithms are employed for a wide class of problems beyond classical facility location:

Location Science: Classical and extended Weber/Fermat–Weber problems, including forbidden regions, barriers, and more general constraints.
Robust Statistics: Computation of multivariate medians, geometric and $L_1$ means (e.g., on manifolds such as SO(3)), and M-estimation for PCA in elliptical models (Virta et al., 3 Oct 2025).
Optimization with Sets: Minimization of weighted sums of distances to convex sets, including problems with nonconvex objectives expressible as d.c. functions.
Federated and Distributed Learning: Aggregation of model updates via robust geometric medians for Byzantine resilience, leveraging the additive structure of Weiszfeld updates for over-the-air computation (Huang et al., 2021).

Behavior in each context depends crucially on objective structure, choice of $q$ in power-type objectives, constraints, and possible regularizations.

7. Literature and Historical Context

Foundational work is attributed to E. Weiszfeld (1937), who introduced the fixed-point approach for unconstrained weighted Euclidean medians. Subsequent advances (e.g., Vardi and Zhang, 2001) resolved delicate issues at nondifferentiability. Extensions have addressed convergence guarantees [Canovas et al.], inclusion of constraints (Nguyen, 2018, Torres, 2012), difference-of-convex settings (An et al., 2014), and robustness issues.

Works such as (Torres, 2012) synthesize these developments, providing algorithms precisely defined at vertices, ensuring that each iterate is feasible, and proving convergence—often via monotonic descent and KKT condition satisfaction. These methodologies have inspired generalizations to infinite-dimensional Hilbert spaces, robust statistical functionals, and algorithmic primitives for modern distributed and adversarial optimization.

In summary, Weiszfeld-type algorithms represent a fundamental framework for iterative minimization of sums of distances (possibly weighted and constrained), characterized by descend-based, projection-augmented fixed-point iteration, careful handling of nondifferentiability, and broad applicability in mathematical programming, statistical estimation, and robust optimization contexts.