Constrained Weiszfeld-Type Algorithms

• Constrained Weiszfeld-type algorithms are iterative fixed-point methods that extend the classical Weiszfeld algorithm by incorporating explicit projections onto convex or manifold constraints.
• They utilize one-sided derivative and subdifferential techniques to handle non-differentiable points, ensuring global convergence and feasibility in applications like facility location and robust PCA.
• Empirical results demonstrate efficient convergence with low iteration counts, often outperforming standard interior-point methods in structured minimization tasks.

Constrained Weiszfeld-type algorithms are iterative fixed-point methods for solving structured minimization problems involving the sum of Euclidean distances under constraints. They generalize the classical Weiszfeld algorithm for the unconstrained Fermat–Weber problem, extending it to settings with closed convex constraints, manifold constraints (such as spheres), and problems with nondifferentiability, including robust principal component analysis (PCA) tasks. This class of algorithms is characterized by their handling of non-differentiable energy functionals, integration of explicit constraint projections, and, where necessary, one-sided derivative and subdifferential techniques to ensure global convergence and feasibility.

1. The Classical Weiszfeld Algorithm and Its Constrained Extensions

The cornerstone is the unconstrained Weber (Fermat–Weber) problem: find $x^* \in \mathbb{R}^n$ minimizing $f(x) = \sum_{i=1}^m w_i \|x - a_i\|$ for given points $a_i$ and positive weights $w_i$. Weiszfeld's 1937 algorithm employs a fixed-point iteration based on the mapping

$$T(x) = \frac{\sum_{i=1}^m \frac{w_i a_i}{\|x - a_i\|}}{\sum_{i=1}^m \frac{w_i}{\|x - a_i\|}}$$

for $x \neq a_i$. This method yields the unique global minimizer under non-collinearity and positivity assumptions on the $a_i$ and $w_i$.

Constrained Weiszfeld-type algorithms extend this framework by seeking the minimizer over a closed convex set $C \subset \mathbb{R}^n$ or a manifold such as the sphere $S^{d-1}$. After each Weiszfeld step, a projection onto the feasible set ensures that all iterates satisfy the imposed constraints. For general constraint sets $C$, the algorithm becomes

$x^{k+1} = P_C(T(x^k))$

where $P_C$ denotes metric projection onto $C$. This strategy guarantees feasibility at every iteration and exploits the strict convexity of $f$ to ensure uniqueness and strong convergence to the global minimizer (Nguyen, 2018, Torres, 2012).

2. Algorithmic Structures and Special Cases

The general structure of a constrained Weiszfeld-type algorithm involves three key components:

• Weiszfeld mapping: Construction of $T(x^k)$ as above. Special treatment is necessary if $x^k$ coincides with an anchor (vertex) $a_i$, where divisions by zero may occur. Modified mappings, such as the Vardi–Zhang variant, interpolate between the anchor and the usual step.
• Constraint projection: Explicit projection to $C$ after the Weiszfeld step, guaranteeing feasibility. For simple sets such as boxes or balls, projections are analytic; for polyhedra, they reduce to quadratic programs or half-space projections; for general convex sets, a proximal map or nested solver is required.
• Anchor handling and non-differentiability: When an iterate lands exactly on a vertex, the algorithm switches to a specially constructed step (e.g., moving along the segment between $T(a_k)$ and $a_k$ until re-entering $C$ as in (Torres, 2012); for manifold settings, such as on the sphere, one-sided directional derivatives and subgradient techniques are essential (Neumayer et al., 2019)).

For robust PCA in $\mathbb{R}^d$, the energy functional becomes

$E(a) = \sum_{i=1}^N \|P_a y_i\|$

where $P_a = I_d - a a^T$ projects onto $a^\perp$ and $a \in S^{d-1}$. Off anchor points (where $a \parallel y_i$), the fixed-point step updates $a$ via normalized power iteration of a weighted sum, followed by projection onto the sphere. On anchors, local minimality is assessed by one-sided derivatives, and if violated, a subgradient step is taken along a stabilized direction, before renormalization (Neumayer et al., 2019).

3. Convergence Analysis and Optimality Properties

Constrained Weiszfeld-type algorithms possess strong convergence properties under standard assumptions:

• Descent property: At each iteration, the objective strictly decreases unless a fixed point is reached. If $x \neq Q(x)$, then $f(Q(x)) < f(x)$, where $Q$ is the (possibly composite) constrained update map (Torres, 2012).
• Fejér monotonicity and optimality: The distance to the unique minimizer is nonincreasing. All iterates remain feasible, and for compact sets, the sequence admits cluster points.
• KKT equivalence and fixed-point characterization: Under classical constraint regularity (convex inequality and affine equality constraints), a point $x^* \notin \{a_i\}$ is a fixed point of $Q$ if and only if it is a Karush–Kuhn–Tucker (KKT) point and the global minimizer. For anchor points, sufficiency still holds (Torres, 2012).
• Non-differentiability and convergence on manifolds: For objectives defined on manifolds (e.g., the energy functional $E(a)$ on $S^{d-1}$ for robust PCA), non-differentiability at anchor points demands the use of one-sided derivatives. Global convergence to a critical point (either stationary or satisfying a one-sided minimality condition at anchors) is established via KL property arguments and the Attouch–Bolte–Svaiter theorem, assuming local Lipschitz and semi-algebraic structure (Neumayer et al., 2019).

4. Computational Behavior and Numerical Observations

Empirical evaluations in facility location and robust PCA settings have demonstrated that constrained Weiszfeld-type algorithms are efficient and robust:

• In two-dimensional constrained Weber problems with $m=50$ demand points and feasibility regions defined by convex inequalities, the Weiszfeld-type algorithm converged within 20–50 iterations for a tolerance of $10^{-5}$. Across 1000 replications, the algorithm always found feasible solutions and its objective value was never exceeded by that of the interior-point solver fmincon, with a maximum observed gap of approximately $0.12$ (Torres, 2012).
• For robust PCA, the algorithm maintains feasibility (unit norm), and converges globally for any initialization on the sphere, up to satisfaction of local criticality (which may only be local due to the nonconvex nature of the problem) (Neumayer et al., 2019).

5. Extensions, Special Cases, and Related Methods

Constrained Weiszfeld-type methodology admits several extensions and generalizations:

• Hilbert space generalization: The method extends to real Hilbert spaces $(H, (\cdot, \cdot))$ with the only requirement that the set $C$ is closed and convex, and the anchor points are not collinear. Existence, uniqueness, and stability of the minimizer carry through (Nguyen, 2018).
• Alternative norms and Bregman projections: The framework can be extended to objective functions involving other norms (e.g., $\|\cdot\|_p$), in which case the gradient surrogates and projections must be replaced by appropriate subgradients and generalized projections such as Bregman or proximal operators. Convergence proofs remain valid under strict convexity (Torres, 2012).
• Block-coordinate updates and multi-facility problems: Block-coordinate Weiszfeld-type maps enable treatment of multi-facility location problems, with each sub-problem solved via a constrained Weiszfeld update for its own variable (Torres, 2012).

A comparative summary of variations is presented below:

Problem type	Constraint type	Algorithmic Step
Classical Weber	None	Weiszfeld iteration
Convex-constrained	Convex set $C$	Weiszfeld + $P_C$
Robust PCA, $S^{d-1}$	Sphere (manifold)	Fixed-point + norm
Anchors, non-diff.	Any	One-sided steps

6. Implementation Considerations and Practical Issues

Implementation requires only routines for the Weiszfeld mapping and projections:

• Computational cost: Feasibility projections are trivial for boxes/balls but may involve quadratic programs for polyhedra. For general convex sets, a nested solver is needed.
• Handling non-differentiability: For points where the iterate equals an anchor or is a non-differentiable configuration (as in robust PCA), a special step along the anchor–Weiszfeld direction or a stabilized subgradient must be executed.
• Convergence speed: Sublinear convergence rates are guaranteed in the general setting. Under additional local error bounds or strong convexity, local linear rates can be achieved (Nguyen, 2018). There is no need for step-size tuning; the method is “parameter-free” in the classical and robust PCA frameworks (Neumayer et al., 2019).
• Multiple initializations: For nonconvex objectives (e.g., robust PCA on the sphere), only convergence to a local critical point can be ensured; multiple restarts may be employed to approach a global minimum (Neumayer et al., 2019).

7. Applications and Connections to Broader Optimization

Constrained Weiszfeld-type algorithms occupy a central role in both geometric location theory and robust subspace estimation:

• Facility location: The classical application is optimal placement with respect to demand or anchor points, subject to feasibility constraints representing operational, geographic, or regulatory considerations (Torres, 2012, Nguyen, 2018).
• Robust PCA: The constrained minimization of sum-of-distances energies provides a robust alternative to standard variance-maximization PCA, accommodating outliers by focusing on direct Euclidean residuals rather than squared deviations (Neumayer et al., 2019).
• Signal processing and statistics: Variants are applicable to trimmed or L1-norm principal components, convex relaxation schemes (REAPER), or block-coordinate settings arising in distributed optimization (Neumayer et al., 2019, Torres, 2012).

A plausible implication is that the explicit handling of constraints and non-differentiable configurations makes these algorithms particularly suitable whenever direct distance objectives interact with general feasibility requirements. The techniques of one-sided subdifferential analysis and manifold projection, pioneered in this context, have broader relevance to optimization over non-Euclidean geometry and composite objectives.