---
title: 'cNKZ Projection: Nonlinear & Constrained Kaczmarz'
url: https://www.emergentmind.com/topics/constrained-nonlinear-kaczmarz-cnkz-projection
type: topic
---

# cNKZ Projection: Nonlinear & Constrained Kaczmarz

The constrained nonlinear Kaczmarz (cNKZ) projection encompasses a class of iterative projection methods for solving high-dimensional systems of nonlinear equations $F(x) = 0$ with side constraints $x \in C$, where $C$ is a (possibly composite) closed convex set. These methods generalize the classical Kaczmarz projection from linear systems to nonlinear and constrained contexts, employing row-action updates derived from residuals and one-row Jacobians, and enforcing constraint feasibility via (metric or Bregman) projections. Modern cNKZ variants admit stochastic sampling, greedy selection, Bregman geometry, and manifold-based formulations, and are supported by sharp convergence theory under local tangential cone, convexity, or full-rank Jacobian assumptions [2209.04195][2303.08549][2012.07555][2410.21630][2011.05870][2406.16112].

## 1. Problem Setting and General Framework

The prototypical cNKZ problem is to compute $x^*\in C$ such that $F(x^*) = 0$ for a differentiable $F: D(F) \subset \mathbb{R}^n \to \mathbb{R}^m$ and a nonempty closed convex $C\subset\mathbb{R}^n$. $C$ can be a Cartesian product of sets, an $\ell_1$-ball, the probability simplex, or more complex polyhedral/semialgebraic constraints. The solution set $S=\{x\in C:F(x)=0\}$ is assumed nonempty.

Standard cNKZ methods operate via sequential projections onto the tangent hyperplanes of $F_i(x)=0$ (possibly linearized at the current iterate), interleaved with (metric or Bregman) projections onto $C$ or its components. Variants include cyclic, random, greedy, and sampling-based selection of the equation index; projection types depend on the geometry of $C$ and the chosen divergence.

### Main update equation (Euclidean metric):
At iteration $k$,
$$
x_{k+1} = P_C\left( x_k - \frac{F_{i_k}(x_k)}{\|\nabla F_{i_k}(x_k)\|^2} \nabla F_{i_k}(x_k) \right)
$$
where $i_k$ is chosen by a sampling, residual maximization, or cyclic rule. For general Bregman divergences $D_\phi$, the step is
$$
x_{k+1} = \operatorname{argmin}_{z\in C} D_\phi(x_k, z) \quad \text{s.t.} \; \langle \nabla F_{i_k}(x_k), z - x_k \rangle = -F_{i_k}(x_k)
$$
which is equivalently solved by a dual update involving the convex conjugate $\phi^*$ [2303.08549][2406.16112].

## 2. Algorithmic Variants and Bregman Extensions

### Metric vs. Bregman Projections

- **Metric (Euclidean) projection:** Suitable when $C$ is simple (e.g., box, ball, simplex). In this case, projections onto $C$ are $O(n)$ and projection onto linearized equation hyperplanes admits closed form; standard Kaczmarz steps result.
- **Bregman projection:** Used for structured $C$ (e.g., $\ell_1$ ball, sparsity, simplex, entropic constraints) or non-Euclidean geometry. Each update projects onto a tangent affine hyperplane wrt. the Bregman distance induced by a strictly convex distance-generating function $\phi$. The resulting problem is a one-dimensional convex program in the dual space and is solved by a globalized Newton step per iteration [2303.08549][2406.16112].

### Algorithmic sampling/selection rules

- **Cyclic:** Step through $i=1,\ldots,m$ in order.
- **Randomized:** Sample $i_k$ uniformly or with nonuniform weights.
- **Greedy-residual:** Choose $i_k = \operatorname{argmax}_i |F_i(x_k)|/\|\nabla F_i(x_k)\|$ to maximize expected decrease per step [2012.07555][2406.16112].
- **Greedy-distance:** Maximize the step length to the projected constraint.

### Accelerated and Motzkin Variants

- **Projected Sampling Kaczmarz–Motzkin (PSKM):** At each iteration, sample a batch of row indices, select the one with maximal residual, perform one-row Kaczmarz update, and project onto a randomly chosen component constraint. The accelerated variant (APSKM) uses sequential projections plus extrapolation to further reduce distance to the feasible set [2209.04195].

## 3. Analytical Assumptions and Convergence Theory

Key assumptions enabling convergence include:
- **Local tangential cone condition (TCC):** Controls the local nonlinearity of each $F_i$ [2209.04195][2011.05870][2303.08549][2406.16112].
- **Row boundedness below:** Bounds $\|\nabla F_i(x)\| \geq \epsilon > 0$ [2209.04195].
- **Full-rank Jacobian / convexity:** Ensures global linear convergence; else, monotonic decrease [2209.04195][2303.08549][2012.07555].

### Central convergence statements:

- **Linear convergence in expectation:** Under TCC with constant $\eta<1/2$ and full-rank Jacobian, for both metric and Bregman cNKZ,
  $$
  \mathbb{E}[\|x_k-x^*\|^2] \leq (1-\rho) \mathbb{E}[\|x_{k-1}-x^*\|^2]
  $$
  for explicit rate $\rho$ dependent on $\sigma_{\min}(F'(x))$, $\eta$, $m$, and norm structure [2209.04195][2303.08549][2406.16112].
- **Monotonic decrease:** If each $F_i$ is convex nonnegative, the error norm strictly decreases per update until convergence [2209.04195].
- **Bregman descent property:** Each Bregman projection step strictly reduces $D_\phi(x_k,\hat x)$ up to a term proportional to squared residual [2303.08549][2406.16112].
- **Regularization and noise tolerance:** Landweber-Kaczmarz/cNKZ admits robust regularization in inverse problems with noisy data and converges weakly/strongly to the solution set, halting when residual falls below noise floor [2011.05870].

## 4. Computational Structure and Practical Complexity

Each cNKZ iteration for a single constraint has per-step cost determined by:
- Computing $F_i(x)$ and $\nabla F_i(x)$: $O(n)$ or per-component cost $G$.
- (Bregman) projection step: $O(n)$ plus a one-dimensional Newton solve for Bregman variants.
- Projection onto $C$ (if nontrivial): $O(n)$ or a prox operator.
- APSKM/accelerated steps increase per-iteration cost marginally due to additional projections or extrapolation steps.

The overall complexity per iteration is $O(G + n)$ for single-row methods, and $O(\beta G + n)$ for batch sampling of size $\beta$ [2209.04195][2303.08549]. Empirical studies show that batch sampling ($10\lesssim\beta\lesssim 100$) balances convergence speed and work per update.

## 5. Applications and Comparative Empirical Performance

Recent cNKZ implementations have proven especially effective for:
- **Large-scale nonlinear equation systems** (e.g. $m \sim 10^4$, $n \sim 10^3$), including equations with strong nonlinearity or sparsity constraints [2209.04195][2303.08549][2406.16112].
- **Inverse problems under noisy data:** Demonstrated robust performance and fewer forward/adjoint solves compared to Landweber–Kaczmarz with line search [2011.05870].
- **Constrained motion planning and coordinated multi-robot manipulation:** Enabling rapid, robust projection onto the intersection of up to 80 nonlinear manifolds, with order-of-magnitude speed-ups in feasibility projection and success rates exceeding 90% in complex settings [2410.21630].
- **Graph realization and quadratic systems:** Locally linear convergence established for systems of quadratic (or sphere) constraints [2012.07555].

Empirically, cNKZ and its variants (especially those using greedy-residual sampling or Bregman geometry) outperform classical randomized Kaczmarz, SGD, and projected gradient methods by factors of 2–10$\times$ in iterations and 10–50$\times$ in wall-clock time for large systems under convex or nearly convex constraints [2209.04195][2406.16112][2303.08549].

## 6. Parameter and Implementation Guidelines

Tuning critical parameters:
- **Step size $\alpha_k$:** Basic Kaczmarz step uses $\alpha_k=1$; $\alpha_k\in(0,2)$ can enforce descent; for Bregman steps, the scalar $t_k$ is adaptively solved per-iteration [2209.04195][2303.08549][2406.16112].
- **Batch/sample size $\beta$:** Empirically, intermediate batch sizes ($10\lesssim\beta\lesssim 100$) yield favorable trade-offs [2209.04195].
- **Bregman generator $\phi$:** Must be strictly convex with $C$ as its domain; entropy, $\ell_1+\ell_2$, and block-simplex penalties are common [2303.08549][2406.16112].
- **Projection thresholds/tolerances:** Set according to application specifics, e.g., $10^{-8}$ for equality, or application-specific physical tolerances [2410.21630].
- **Randomization vs. greedy selection:** Greedy-residual or distance selection provides significant iteration savings in practice; uniform random selection is simple and effective for homogeneous systems [2012.07555][2406.16112].

## 7. Extensions, Limitations, and Current Research Directions

cNKZ methods can be further generalized to:
- Handle general nonlinear manifold intersection via manifold-based projection formulation, as in complex robotic systems [2410.21630].
- Incorporate acceleration (e.g., Motzkin or APSKM extrapolation steps) to improve convergence in ill-conditioned or nearly inconsistent systems [2209.04195].
- Integrate relaxed/adaptive projection step sizes (mirror-SPS, adaptive Bregman steps) for further empirical gains [2303.08549][2406.16112].
- Support additional side constraints via Bregman-generated domains (sparsity, simplex, entropy).
- Employ scalable parallel implementations and composite constraint handling for high-dimensional applications.

Observed limitations include sensitivity of convergence to initializations in nonconvex settings, slowdown if projections onto $C$ are expensive, and the need for empirical tuning of per-constraint tolerances and step sizes in complex applications (e.g., multi-robot systems). Extensions to force/torque constraints, adaptive projection rules, and coupling with trajectory optimization are active research areas [2410.21630].

---

**Key references:**  
- [2209.04195]: Introduction of sampling Kaczmarz–Motzkin methods and their constrained variants (PSKM/APSKM)  
- [2303.08549]: Bregman–Kaczmarz cNKZ with adaptive steps and strong convergence guarantees  
- [2012.07555]: Successive projection and greedy/greedy-residual cNKZ for nonlinear/inequality systems  
- [2410.21630]: Manifold-based cNKZ for multi-robot manipulation  
- [2011.05870]: Projective Landweber–Kaczmarz with regularization for ill-posed constrained systems  
- [2406.16112]: Greedy randomized Bregman–Kaczmarz with sharp convergence for large-scale, structured, constraint-rich nonlinear systems

Source: https://www.emergentmind.com/topics/constrained-nonlinear-kaczmarz-cnkz-projection