---
title: Rescaling Perceptron Algorithm
url: https://www.emergentmind.com/topics/rescaling-perceptron-algorithm
type: topic
---

# Rescaling Perceptron Algorithm

Searching arXiv for the cited and closely related papers on rescaled perceptron / projection-and-rescaling.
The rescaling perceptron algorithm is a family of feasibility methods that couples a perceptron-type basic procedure with intermittent geometric rescaling. In linear programming form, the method addresses the open cone
\[
P=\{x\in\mathbb{R}^n\mid A x>0\},
\]
while in the projection-and-rescaling framework it addresses
\[
\text{find }x\in L\cap\Omega,
\]
where \(L\) is a linear subspace and \(\Omega\) is the interior of a symmetric cone in a finite-dimensional Euclidean Jordan algebra [1612.04782], [1512.06154]. The central idea is invariant across these settings: run a low-cost perceptron-like subroutine until it either finds a feasible point or certifies poor scaling, then apply an invertible transformation that improves a geometric condition measure by a constant factor. Repeating this two-phase cycle yields polynomial-time algorithms in settings where the classical perceptron alone is not polynomial-time [1612.04782], [1512.06154].

## 1. Historical development and conceptual scope

The classical perceptron algorithm for linear programming has been known since the 1950s and is valued for simplicity and robustness, but it is not polynomial-time in the worst case because its convergence depends on a margin parameter that can be exponentially small in the input bit size [1612.04782]. For the feasibility region
\[
P=\{x\in\mathbb{R}^n\mid A x>0\},
\]
if \(\rho>0\) denotes the radius of the largest Euclidean ball contained in \(P\cap B(0,1)\), then the classical perceptron terminates in at most \(1/\rho^2\) updates [1612.04782]. This dependence on \(\rho\) motivates rescaling.

A key milestone was the observation that rescaling can turn perceptron-style feasibility search into a polynomial-time algorithm. Dunagan and Vempala showed that a randomized rescaling turns the perceptron method into a polynomial-time algorithm, and Peña and Soheili later gave a deterministic rescaling [1612.04782]. The 2016 deterministic improvement made it possible to rescale much earlier by raising the rescaling threshold from \(\Theta(1/(m\sqrt n))\) to \(\Theta(1/n)\), thereby accelerating the overall running time [1612.04782].

In parallel, Peña and Soheili developed a projection-and-rescaling algorithm over symmetric cones, explicitly describing the method as inspired by previous work on rescaled versions of the perceptron algorithm and by Chubanov’s projection-based method for linear feasibility problems [1512.06154]. This generalized the paradigm from polyhedral cones to Euclidean Jordan algebras and symmetric cones, preserving the same two-step architecture: a basic procedure and a rescaling step [1512.06154].

The phrase “rescaling perceptron” also appears in a different but related sense in large-margin learning. Panagiotakopoulos and Tsampouka introduced perceptron updates with multiplicative shrinking of the current weight vector before each correction step, obtaining finite-step approximation guarantees for the maximal margin hyperplane [1205.4698]. This is algorithmically distinct from geometric projection-and-rescaling, but it shares the same structural theme: modify the geometry or effective scale of the iterate to improve convergence.

## 2. Core feasibility formulations and condition measures

In linear programming feasibility, the target set is the open polyhedral cone
\[
P=\{x\in\mathbb{R}^n\mid A x>0\},
\]
with rows \(A_i\neq 0\), typically normalized so that \(\|A_i\|_2=1\) [1612.04782]. The associated condition parameter is
\[
\rho=\text{radius of the largest Euclidean ball contained in }P\cap B(0,1),
\]
which governs the classical perceptron complexity \(1/\rho^2\) [1612.04782].

In the symmetric-cone formulation, the fundamental problem is
\[
\text{find }x\in L\cap\Omega,
\]
where \(V\) is a finite-dimensional real vector space with inner product \(\langle\cdot,\cdot\rangle\), \((V,\circ,e)\) is an Euclidean Jordan algebra, \(Q=\{y^2:y\in V\}\) is its cone of squares, and \(\Omega=\operatorname{int}Q\) [1512.06154]. An equivalent formulation is
\[
\text{find }z\in V\text{ such that }P_L z\in\Omega,
\]
where \(P_L:V\to L\) is the orthogonal projection onto \(L\) [1512.06154].

The geometric depth of feasibility is quantified by
\[
\delta(L\cap\Omega):=\max\{\det(x):x\in L\cap\Omega,\ \|x\|_{\mathrm{op}}=1\}\in(0,1],
\]
where \(\|x\|_{\mathrm{op}}\) is the operator norm and \(\det(x)\) is the Jordan-algebra determinant [1512.06154]. When \(L\cap\Omega\neq\emptyset\), one has \(0<\delta(L\cap\Omega)\le 1\), and \(\delta(L\cap\Omega)=1\) if and only if the unit \(e\in\Omega\) belongs to \(L\), meaning that \(L\) passes through the center of the cone [1512.06154].

Two special cases are explicitly identified. If \(\Omega=\mathbb{R}^n_+\) and \(V=\mathbb{R}^n\) with the coordinate-wise algebra, then
\[
\delta(L\cap\mathbb{R}^n_+)=\max\left\{\prod_i x_i:x\in L\cap\mathbb{R}^n_+,\ \|x\|_\infty=1\right\}.
\]
If \(\Omega=S^n_{++}\) and \(V=S^n\), then \(\det(x)\) is the usual determinant and \(\|x\|_{\mathrm{op}}\) is the spectral norm [1512.06154].

The polyhedral implementation studied computationally by Peña and Soheili also uses a centrality measure
\[
\delta\bigl(L\cap\mathbb{R}^n_{++}\bigr)=\max\Bigl\{\prod_{j=1}^n x_j:\ x\in L\cap\mathbb{R}^n_{++},\ \|x\|_\infty=1\Bigr\},
\]
which lies in \((0,1]\) if the primal is strictly feasible [1803.07107]. In that setting the algorithm is used not only to find feasible points but also to recover the Goldman–Tucker partition for the complementary pair \(L\cap\mathbb{R}^n_+\) and \(L^\perp\cap\mathbb{R}^n_+\) [1803.07107].

## 3. Two-phase architecture: basic procedure and rescaling

The canonical rescaling perceptron template alternates between a perceptron-type search phase and a geometric transformation phase. In the linear-programming version, the basic deterministic-rescaling framework is:

1. run a Perceptron-type subroutine to either find a feasible \(x\in P\) or return a convex combination \(\lambda\in\mathbb{R}^m_{\ge 0}\), \(\|\lambda\|_1=1\), satisfying
   \[
   \|\lambda A\|_2\le \Delta;
   \]
2. apply an invertible linear map \(F\) so that
   \[
   \operatorname{vol}(F(P)\cap B(0,1))
   \]
   increases by a constant factor, then replace \(P\) by \(F(P)\) and repeat [1612.04782].

The symmetric-cone version has the same structure. The basic procedure takes as input a projection \(P\) onto the current subspace \(L\), and returns either:

1. a certificate of feasibility \(Pz\in\Omega\), so \(x=Pz\in L\cap\Omega\); or
2. a witness \(z\in\Omega\setminus\{0\}\) such that
   \[
   \|(Pz)_+\|_F\le \epsilon\|z\|,
   \]
   where \((Pz)_+\) is the projection of \(Pz\) onto \(Q\) and \(\|\cdot\|_F\) is the Frobenius norm [1512.06154].

The rescaling step then identifies a “most-violated” primitive idempotent \(c\in V\) and constructs a quadratic rescaling map from \(v=e+ac\), where \(a>0\) is fixed and in practice \(a=1/\sqrt2-1\) [1512.06154]. The outer loop replaces \(L\) by its rescaled image and recomputes the projection. Because each successful rescaling multiplies \(\delta(L\cap\Omega)\) by a constant larger than \(1\), the number of rescaling rounds is logarithmic in \(1/\delta\) [1512.06154].

The same architecture is visible in the computational implementation over \(\mathbb{R}^n_+\), where an “Enhanced Projection-and-Rescaling Algorithm” runs the basic procedure on both \(L\) and \(L^\perp\), then updates diagonal scaling matrices \(D\) and \(\hat D\) and recomputes the projectors \(P_{D(L)}\) and \(P_{\hat D(L^\perp)}\) [1803.07107]. There the rescaling step is diagonal rather than Jordan-quadratic, but the logical role is identical: detect poor scaling, transform the subspace, and continue.

## 4. Perceptron-type basic procedures

Several basic procedures are explicitly described in the literature.

For symmetric cones, Peña and Soheili work in the “spectraplex”
\[
\Delta(\Omega):=\{z\in\Omega:\langle z,e\rangle=1\}
\]
and maintain iterates \(z_t\in\Delta(\Omega)\) [1512.06154]. Given a separation oracle for \(\Omega\), the perceptron scheme initializes \(z_0\in\Delta(\Omega)\) and repeats: if \(Pz_t\in\Omega\), return \(x=Pz_t\); if \(\|(Pz_t)_+\|_F\le \epsilon\|z_t\|\), return witness \(z=z_t\); otherwise choose \(u_t\in\Delta(\Omega)\) with \(\langle u_t,Pz_t\rangle\le 0\) and update
\[
z_{t+1}=z_t+\frac{1}{t+1}(u_t-z_t).
\]
The analysis shows by induction that
\[
\|Pz_t\|_F\le \frac{1}{\sqrt t},
\]
so after \(T\le 1/\epsilon^2=O(r^4)\) iterations either feasibility is certified or the witness condition holds [1512.06154].

For polyhedral feasibility, the 2016 deterministic algorithm uses a smooth perceptron phase. With \(\Delta=\Theta(1/n)\), one phase runs smooth perceptron for \(\tilde O(mn/\Delta)\) steps; either it finds \(x\in P\) or returns convex \(\lambda\) with \(\|\lambda A\|_2\le \Delta\) [1612.04782]. Lemma 2.1 states that this is achieved in time \(\tilde O(mn/\Delta)\) [1612.04782].

The 2018 enhancement paper replaces the vanilla perceptron by a Limited-Support Perceptron (LSP), using an efficient algorithmic implementation of Carathéodory’s theorem through the modified Incremental Representation Reduction procedure, mIRR [1807.05982]. LSP maintains \(x_t\in\Delta_{n-1}\) with support size at most \(m+1\), picks
\[
j_t=\arg\min_{1\le i\le n}\langle q_i,z_t\rangle,
\qquad
\theta_t=\frac{1}{t+1},
\]
and updates
\[
x'_{t+1}=x_t+\theta_t(e_{j_t}-x_t),\qquad
z_{t+1}=Q^T x'_{t+1}=(1-\theta_t)z_t+\theta_t q_{j_t}.
\]
If a new support index appears, mIRR prunes the support back to size at most \(m+1\) in \(O(m^2)\) time per call without changing the point \(x'_{t+1}\) [1807.05982]. The stopping rule is either strict feasibility \(Px_t>0\) or
\[
\|(Px_t)^+\|\le \frac{\|x_t\|_\infty}{3\sqrt n}.
\]

In computational experiments on \(\mathbb{R}^n_+\), four basic procedures were compared: perceptron, von Neumann, von Neumann+away, and smooth perceptron. The smooth perceptron scheme consistently required 1–2 orders of magnitude fewer iterations and CPU time than the other three schemes [1803.07107]. This supports the view that the practical efficiency of a rescaling perceptron method depends as much on the inner routine as on the outer rescaling logic.

## 5. Rescaling operators and geometric improvement

The defining feature of the rescaling perceptron algorithm is that rescaling is not merely a heuristic restart; it is a geometric transformation with a provable multiplicative improvement in a condition measure.

For linear programming, two deterministic rescalings are described in detail [1612.04782]:

| Rescaling | Definition | Proven effect |
|---|---|---|
| Rank-one “thin-direction” stretch | \(F(c)=2c,\ F(x)=x\) for \(x\perp c\), equivalently \(F=I+cc^\top\) | \(\operatorname{vol}(F(P)\cap B)\ge \tfrac32\,\operatorname{vol}(P\cap B)\) |
| Deterministic multi-rank rescaling | \(M=\sum_{i=1}^m \lambda_i A_iA_i^\top,\ F=(I+\alpha M)^{1/2}\) | \(\operatorname{vol}(F(P)\cap B)\ge e^{\alpha/5}\,\operatorname{vol}(P\cap B)\) |

The first applies when a unit vector \(c\) satisfies
\[
\operatorname{width}(P,c)=\max_{x\in P\cap B(0,1)}|\langle c,x\rangle|
\le \frac{1}{3\sqrt n},
\]
and in matrix terms updates the constraints to
\[
A'=A\Bigl(I-\frac12 cc^\top\Bigr)
\]
[1612.04782]. The second uses the PSD matrix
\[
M=\sum_{i=1}^m \lambda_iA_iA_i^\top
\]
with \(\operatorname{tr}M=1\) and \(\delta_{\max}=\|M\|_{\mathrm{op}}\), then takes \(0\le \alpha\le 1/\delta_{\max}\) [1612.04782]. Taking \(\alpha=1\) increases volume by a constant factor [1612.04782].

For symmetric cones, the rescaling operator is quadratic. If \(z\in\Omega\setminus\{0\}\) satisfies
\[
\|(P_L z)_+\|_F<\epsilon\|z\|,
\]
choose a primitive idempotent \(c\) with \(z\circ c=\lambda_{\max}(z)c\), \(\|c\|_F=1\), set \(v=e+ac\), and define
\[
D_v(x)=2\,v\circ(v\circ x)-(v^2)\circ x.
\]
Then
\[
\det(D_v(x))=(\det v)^2\det x=(1+a)^2\det x,
\]
and
\[
\|D_v(x)\|_F\le (1+(2a+a^2)\epsilon)\|x\|_F.
\]
It follows that
\[
\delta(D_v(L)\cap\Omega)\ge
\frac{(1+a)^2}{(1+(2a+a^2)\epsilon)^r}\,\delta(L\cap\Omega),
\]
and for \(\epsilon=1/\sqrt{2r}\) and \(a=1/\sqrt2-1\) this multiplier exceeds \(1.5\) [1512.06154].

In the orthant-specific computational algorithm, the rescaling is diagonal. If the basic procedure returns \(z\) with
\[
\|(Pz)^+\|_1\le \epsilon\|z\|_\infty,
\]
the algorithm forms
\[
e=\Bigl(z/\|(Pz)^+\|_1-1\Bigr)^+
\]
and updates
\[
D\leftarrow \min\bigl((I+\operatorname{diag}(e))D,\ U\bigr),
\]
with an analogous update on the dual side [1803.07107]. The theory yields
\[
\delta(D(L)\cap\mathbb{R}^n_{++})\ge 2\,\delta(L\cap\mathbb{R}^n_{++}),
\]
so each round doubles the condition measure [1803.07107].

These constructions share a common pattern: the witness returned by the perceptron phase identifies a direction or coordinate system in which the feasible region is thin, and the rescaling thickens that region enough to guarantee a constant-factor improvement.

## 6. Complexity, refinements, and adjacent large-margin variants

The complexity theory of rescaling perceptron algorithms is organized around two quantities: the number of basic-procedure updates per phase and the number of rescaling phases.

For symmetric cones, each of the \(O(\log(1/\delta(L\cap\Omega)))\) outer rounds invokes a basic procedure. In perceptron form this takes \(O(r^4)\) updates; a smooth perceptron variant drives the same witness test in only \(O(r^2)\) steps, at the price of one full Jordan-eigenvalue decomposition per step [1512.06154]. Thus the total number of perceptron updates is
\[
O(r^4\log(1/\delta))
\quad\text{or}\quad
O(r^2\log(1/\delta)),
\]
depending on the basic-procedure implementation [1512.06154].

For linear programming, Theorem 2.3 of the improved deterministic algorithm states that with \(\Delta=\Theta(1/n)\), one phase takes
\[
\tilde O(mn/\Delta+mn^2)=\tilde O(mn^2),
\]
and since there are \(O(n\log(1/\rho))\) phases, the total time is
\[
\tilde O\bigl(mn^3\log(1/\rho)\bigr)
\]
[1612.04782]. The paper compares this with previous bounds:
\[
\tilde O\bigl(mn^4\log(1/\rho)\bigr)
\]
for Dunagan–Vempala 2004 and
\[
\tilde O\bigl(m^2n^{2.5}\log(1/\rho)\bigr)
\]
for Peña–Soheili 2016 [1612.04782]. The improvement comes from earlier rescaling, since lowering the amount of progress demanded from the inner phase reduces the cost of each phase while preserving a constant-factor gain in volume [1612.04782].

The 2018 limited-support enhancement improves the basic procedure in the projection-and-rescaling framework by keeping support size at most \(m+1\). The resulting iteration bound is \(O(m^2n)\), the per-iteration cost is \(O(nm)\), and the total cost of the basic procedure becomes
\[
O(n^2m^3),
\]
improving on the original \(O(n^4m)\) basic-procedure complexity when \(m\ll n\) [1807.05982].

The computational study of the orthant case reports that with the smooth perceptron basic procedure and \(\epsilon=\tfrac12\), one has
\[
T_{\mathrm{basic}}(\epsilon)=O(1/\epsilon)\quad\Longrightarrow\quad O(n^2/\epsilon)\text{ floating-point ops},
\]
and because each rescaling doubles \(\delta\), the total work is
\[
O\Bigl(n^2\log\frac1\delta\Bigr)
\]
for the primal side, with an analogous bound for the dual side [1803.07107]. The reported numerical behavior is also specific: on controlled-condition instances with \(L\cap\mathbb{R}^n_{++}\neq\emptyset\), across \((m,n)\) up to \((1000,2000)\), the average number of rescaling rounds was 9–15, total basic-procedure iterations were in the low thousands, and CPU times were under a minute even at \(n=2000\); on mixed-feasible instances the partition was correctly identified in over \(99\%\) of trials; and on naive random instances EPRA was 5–10× faster than CPLEX, whereas on controlled-\(\delta\) instances CPLEX outperformed EPRA by about 10× [1803.07107].

An adjacent but distinct line of work uses “rescaling” internally in the perceptron update rather than externally in the geometry of the feasible set. In the weight-shrinking perceptron of Panagiotakopoulos and Tsampouka, if a pattern \(y_k\) violates the scaled margin
\[
\bar c_t(w_t\cdot y_k)\le b,
\]
the update is
\[
w_{t+1}=c_t w_t+\eta y_k,
\]
with either constant shrinking \(c_t=1-\eta\lambda\) or variable shrinking
\[
c_t=\bar c_t=\Bigl(\frac{t}{t+1}\Bigr)^n
\]
[1205.4698]. Standard perceptron with margin corresponds to \(\lambda=0\) or \(n=0\), for which the guaranteed margin-fraction is \(1/2\) [1205.4698]. By contrast, the shrinking variants can provably attain in a finite number of steps any desirable approximation of the maximal margin hyperplane, with update complexity summarized as \(O(R^2/(\gamma_{\rm d}^2\delta))\) in the small-\(\delta\) regime described in the paper [1205.4698]. This suggests that “rescaling perceptron algorithm” has at least two technical meanings in the literature: outer geometric rescaling for feasibility, and inner shrink-then-update rescaling for large-margin learning.

## 7. Interpretation and relation to neighboring methods

The rescaling perceptron algorithm occupies a junction between classical online learning updates, projection methods for conic feasibility, and interior-geometric conditioning. In the LP setting, it is connected to multiplicative weights update methods: the same deterministic rescaling can be combined with the potential
\[
\Phi(x)=\sum_{i=1}^m e^{-\langle A_i,x\rangle}
\]
to yield a polynomial-time rescaled-MWU algorithm with overall time
\[
\tilde O\bigl(mn^{\omega+1}\log(1/\rho)\bigr)
\]
[1612.04782]. This establishes an explicit connection between perceptron-style feasibility search and a broader class of first-order methods.

In the projection-and-rescaling setting, the relation to Chubanov’s method is direct: the algorithm is projection-based, works with feasibility certificates and witnesses of poor scaling, and uses rescaling to improve a centrality measure until feasibility becomes easy to certify [1512.06154]. The orthant specialization further links the method to complementarity structure through the Goldman–Tucker partition, allowing simultaneous recovery of relative-interior primal and dual solutions [1803.07107].

A common misconception is to treat the perceptron phase as the main algorithm and the rescaling phase as an implementation detail. The cited works indicate the opposite. The perceptron subroutine alone inherits the adverse dependence on \(\rho\) or on the depth of feasibility, whereas the rescaling phase is what converts this behavior into logarithmic dependence on a condition measure such as \(\delta\) or a linear factor in \(n\log(1/\rho)\) [1612.04782], [1512.06154]. Another plausible implication is that improvements in support control, separation-oracle design, or eigenvalue computations are best understood as improvements to the basic procedure inside a larger conditioning framework rather than as stand-alone replacements for rescaling.

Across its variants, the rescaling perceptron algorithm is therefore best understood as a meta-algorithmic pattern: use a perceptron-type routine to expose poor geometry, transform the geometry by a provably beneficial map, and iterate until a feasible or most-interior solution is obtained. In polyhedral, orthant, and symmetric-cone settings alike, the theoretical and computational results center on that pattern [1612.04782], [1512.06154], [1803.07107], [1807.05982].

Source: https://www.emergentmind.com/topics/rescaling-perceptron-algorithm