Rescaling Perceptron Algorithm

• The paper introduces a rescaling perceptron that couples a perceptron subroutine with periodic geometric rescaling to achieve polynomial-time feasibility certification.
• It details a two-phase architecture where a low-cost perceptron search alternates with an invertible transformation that improves the geometric condition measure.
• The method extends classical perceptron techniques to symmetric cones and orthant-specific implementations, demonstrating faster convergence in structured feasibility problems.

Searching arXiv for the cited and closely related papers on rescaled perceptron / projection-and-rescaling. The rescaling perceptron algorithm is a family of feasibility methods that couples a perceptron-type basic procedure with intermittent geometric rescaling. In linear programming form, the method addresses the open cone

$P=\{x\in\mathbb{R}^n\mid A x>0\},$

while in the projection-and-rescaling framework it addresses

$\text{find }x\in L\cap\Omega,$

where $L$ is a linear subspace and $\Omega$ is the interior of a symmetric cone in a finite-dimensional Euclidean Jordan algebra (Hoberg et al., 2016, Pena et al., 2015). The central idea is invariant across these settings: run a low-cost perceptron-like subroutine until it either finds a feasible point or certifies poor scaling, then apply an invertible transformation that improves a geometric condition measure by a constant factor. Repeating this two-phase cycle yields polynomial-time algorithms in settings where the classical perceptron alone is not polynomial-time (Hoberg et al., 2016, Pena et al., 2015).

1. Historical development and conceptual scope

The classical perceptron algorithm for linear programming has been known since the 1950s and is valued for simplicity and robustness, but it is not polynomial-time in the worst case because its convergence depends on a margin parameter that can be exponentially small in the input bit size (Hoberg et al., 2016). For the feasibility region

$P=\{x\in\mathbb{R}^n\mid A x>0\},$

if $\rho>0$ denotes the radius of the largest Euclidean ball contained in $P\cap B(0,1)$, then the classical perceptron terminates in at most $1/\rho^2$ updates (Hoberg et al., 2016). This dependence on $\rho$ motivates rescaling.

A key milestone was the observation that rescaling can turn perceptron-style feasibility search into a polynomial-time algorithm. Dunagan and Vempala showed that a randomized rescaling turns the perceptron method into a polynomial-time algorithm, and Peña and Soheili later gave a deterministic rescaling (Hoberg et al., 2016). The 2016 deterministic improvement made it possible to rescale much earlier by raising the rescaling threshold from $\Theta(1/(m\sqrt n))$ to $\text{find }x\in L\cap\Omega,$0, thereby accelerating the overall running time (Hoberg et al., 2016).

In parallel, Peña and Soheili developed a projection-and-rescaling algorithm over symmetric cones, explicitly describing the method as inspired by previous work on rescaled versions of the perceptron algorithm and by Chubanov’s projection-based method for linear feasibility problems (Pena et al., 2015). This generalized the paradigm from polyhedral cones to Euclidean Jordan algebras and symmetric cones, preserving the same two-step architecture: a basic procedure and a rescaling step (Pena et al., 2015).

The phrase “rescaling perceptron” also appears in a different but related sense in large-margin learning. Panagiotakopoulos and Tsampouka introduced perceptron updates with multiplicative shrinking of the current weight vector before each correction step, obtaining finite-step approximation guarantees for the maximal margin hyperplane (Panagiotakopoulos et al., 2012). This is algorithmically distinct from geometric projection-and-rescaling, but it shares the same structural theme: modify the geometry or effective scale of the iterate to improve convergence.

2. Core feasibility formulations and condition measures

In linear programming feasibility, the target set is the open polyhedral cone

$\text{find }x\in L\cap\Omega,$1

with rows $\text{find }x\in L\cap\Omega,$2, typically normalized so that $\text{find }x\in L\cap\Omega,$3 (Hoberg et al., 2016). The associated condition parameter is

$\text{find }x\in L\cap\Omega,$4

which governs the classical perceptron complexity $\text{find }x\in L\cap\Omega,$5 (Hoberg et al., 2016).

In the symmetric-cone formulation, the fundamental problem is

$\text{find }x\in L\cap\Omega,$6

where $\text{find }x\in L\cap\Omega,$7 is a finite-dimensional real vector space with inner product $\text{find }x\in L\cap\Omega,$8, $\text{find }x\in L\cap\Omega,$9 is an Euclidean Jordan algebra, $L$0 is its cone of squares, and $L$1 (Pena et al., 2015). An equivalent formulation is

$L$2

where $L$3 is the orthogonal projection onto $L$4 (Pena et al., 2015).

The geometric depth of feasibility is quantified by

$L$5

where $L$6 is the operator norm and $L$7 is the Jordan-algebra determinant (Pena et al., 2015). When $L$8, one has $L$9, and $\Omega$0 if and only if the unit $\Omega$1 belongs to $\Omega$2, meaning that $\Omega$3 passes through the center of the cone (Pena et al., 2015).

Two special cases are explicitly identified. If $\Omega$4 and $\Omega$5 with the coordinate-wise algebra, then

$\Omega$6

If $\Omega$7 and $\Omega$8, then $\Omega$9 is the usual determinant and $P=\{x\in\mathbb{R}^n\mid A x>0\},$0 is the spectral norm (Pena et al., 2015).

The polyhedral implementation studied computationally by Peña and Soheili also uses a centrality measure

$P=\{x\in\mathbb{R}^n\mid A x>0\},$1

which lies in $P=\{x\in\mathbb{R}^n\mid A x>0\},$2 if the primal is strictly feasible (Pena et al., 2018). In that setting the algorithm is used not only to find feasible points but also to recover the Goldman–Tucker partition for the complementary pair $P=\{x\in\mathbb{R}^n\mid A x>0\},$3 and $P=\{x\in\mathbb{R}^n\mid A x>0\},$4 (Pena et al., 2018).

3. Two-phase architecture: basic procedure and rescaling

The canonical rescaling perceptron template alternates between a perceptron-type search phase and a geometric transformation phase. In the linear-programming version, the basic deterministic-rescaling framework is:

1. run a Perceptron-type subroutine to either find a feasible $P=\{x\in\mathbb{R}^n\mid A x>0\},$5 or return a convex combination $P=\{x\in\mathbb{R}^n\mid A x>0\},$6, $P=\{x\in\mathbb{R}^n\mid A x>0\},$7, satisfying

$P=\{x\in\mathbb{R}^n\mid A x>0\},$8

2. apply an invertible linear map $P=\{x\in\mathbb{R}^n\mid A x>0\},$9 so that

$\rho>0$0

increases by a constant factor, then replace $\rho>0$1 by $\rho>0$2 and repeat (Hoberg et al., 2016).

The symmetric-cone version has the same structure. The basic procedure takes as input a projection $\rho>0$3 onto the current subspace $\rho>0$4, and returns either:

1. a certificate of feasibility $\rho>0$5, so $\rho>0$6; or
2. a witness $\rho>0$7 such that

$\rho>0$8

where $\rho>0$9 is the projection of $P\cap B(0,1)$0 onto $P\cap B(0,1)$1 and $P\cap B(0,1)$2 is the Frobenius norm (Pena et al., 2015).

The rescaling step then identifies a “most-violated” primitive idempotent $P\cap B(0,1)$3 and constructs a quadratic rescaling map from $P\cap B(0,1)$4, where $P\cap B(0,1)$5 is fixed and in practice $P\cap B(0,1)$6 (Pena et al., 2015). The outer loop replaces $P\cap B(0,1)$7 by its rescaled image and recomputes the projection. Because each successful rescaling multiplies $P\cap B(0,1)$8 by a constant larger than $P\cap B(0,1)$9, the number of rescaling rounds is logarithmic in $1/\rho^2$0 (Pena et al., 2015).

The same architecture is visible in the computational implementation over $1/\rho^2$1, where an “Enhanced Projection-and-Rescaling Algorithm” runs the basic procedure on both $1/\rho^2$2 and $1/\rho^2$3, then updates diagonal scaling matrices $1/\rho^2$4 and $1/\rho^2$5 and recomputes the projectors $1/\rho^2$6 and $1/\rho^2$7 (Pena et al., 2018). There the rescaling step is diagonal rather than Jordan-quadratic, but the logical role is identical: detect poor scaling, transform the subspace, and continue.

4. Perceptron-type basic procedures

Several basic procedures are explicitly described in the literature.

For symmetric cones, Peña and Soheili work in the “spectraplex”

$1/\rho^2$8

and maintain iterates $1/\rho^2$9 (Pena et al., 2015). Given a separation oracle for $\rho$0, the perceptron scheme initializes $\rho$1 and repeats: if $\rho$2, return $\rho$3; if $\rho$4, return witness $\rho$5; otherwise choose $\rho$6 with $\rho$7 and update

$\rho$8

The analysis shows by induction that

$\rho$9

so after $\Theta(1/(m\sqrt n))$0 iterations either feasibility is certified or the witness condition holds (Pena et al., 2015).

For polyhedral feasibility, the 2016 deterministic algorithm uses a smooth perceptron phase. With $\Theta(1/(m\sqrt n))$1, one phase runs smooth perceptron for $\Theta(1/(m\sqrt n))$2 steps; either it finds $\Theta(1/(m\sqrt n))$3 or returns convex $\Theta(1/(m\sqrt n))$4 with $\Theta(1/(m\sqrt n))$5 (Hoberg et al., 2016). Lemma 2.1 states that this is achieved in time $\Theta(1/(m\sqrt n))$6 (Hoberg et al., 2016).

The 2018 enhancement paper replaces the vanilla perceptron by a Limited-Support Perceptron (LSP), using an efficient algorithmic implementation of Carathéodory’s theorem through the modified Incremental Representation Reduction procedure, mIRR (Gutman, 2018). LSP maintains $\Theta(1/(m\sqrt n))$7 with support size at most $\Theta(1/(m\sqrt n))$8, picks

$\Theta(1/(m\sqrt n))$9

and updates

$\text{find }x\in L\cap\Omega,$00

If a new support index appears, mIRR prunes the support back to size at most $\text{find }x\in L\cap\Omega,$01 in $\text{find }x\in L\cap\Omega,$02 time per call without changing the point $\text{find }x\in L\cap\Omega,$03 (Gutman, 2018). The stopping rule is either strict feasibility $\text{find }x\in L\cap\Omega,$04 or

$\text{find }x\in L\cap\Omega,$05

In computational experiments on $\text{find }x\in L\cap\Omega,$06, four basic procedures were compared: perceptron, von Neumann, von Neumann+away, and smooth perceptron. The smooth perceptron scheme consistently required 1–2 orders of magnitude fewer iterations and CPU time than the other three schemes (Pena et al., 2018). This supports the view that the practical efficiency of a rescaling perceptron method depends as much on the inner routine as on the outer rescaling logic.

5. Rescaling operators and geometric improvement

The defining feature of the rescaling perceptron algorithm is that rescaling is not merely a heuristic restart; it is a geometric transformation with a provable multiplicative improvement in a condition measure.

For linear programming, two deterministic rescalings are described in detail (Hoberg et al., 2016):

Rescaling	Definition	Proven effect
Rank-one “thin-direction” stretch	$\text{find }x\in L\cap\Omega,$07 for $\text{find }x\in L\cap\Omega,$08, equivalently $\text{find }x\in L\cap\Omega,$09	$\text{find }x\in L\cap\Omega,$10
Deterministic multi-rank rescaling	$\text{find }x\in L\cap\Omega,$11	$\text{find }x\in L\cap\Omega,$12

The first applies when a unit vector $\text{find }x\in L\cap\Omega,$13 satisfies

$\text{find }x\in L\cap\Omega,$14

and in matrix terms updates the constraints to

$\text{find }x\in L\cap\Omega,$15

(Hoberg et al., 2016). The second uses the PSD matrix

$\text{find }x\in L\cap\Omega,$16

with $\text{find }x\in L\cap\Omega,$17 and $\text{find }x\in L\cap\Omega,$18, then takes $\text{find }x\in L\cap\Omega,$19 (Hoberg et al., 2016). Taking $\text{find }x\in L\cap\Omega,$20 increases volume by a constant factor (Hoberg et al., 2016).

For symmetric cones, the rescaling operator is quadratic. If $\text{find }x\in L\cap\Omega,$21 satisfies

$\text{find }x\in L\cap\Omega,$22

choose a primitive idempotent $\text{find }x\in L\cap\Omega,$23 with $\text{find }x\in L\cap\Omega,$24, $\text{find }x\in L\cap\Omega,$25, set $\text{find }x\in L\cap\Omega,$26, and define

$\text{find }x\in L\cap\Omega,$27

Then

$\text{find }x\in L\cap\Omega,$28

and

$\text{find }x\in L\cap\Omega,$29

It follows that

$\text{find }x\in L\cap\Omega,$30

and for $\text{find }x\in L\cap\Omega,$31 and $\text{find }x\in L\cap\Omega,$32 this multiplier exceeds $\text{find }x\in L\cap\Omega,$33 (Pena et al., 2015).

In the orthant-specific computational algorithm, the rescaling is diagonal. If the basic procedure returns $\text{find }x\in L\cap\Omega,$34 with

$\text{find }x\in L\cap\Omega,$35

the algorithm forms

$\text{find }x\in L\cap\Omega,$36

and updates

$\text{find }x\in L\cap\Omega,$37

with an analogous update on the dual side (Pena et al., 2018). The theory yields

$\text{find }x\in L\cap\Omega,$38

so each round doubles the condition measure (Pena et al., 2018).

These constructions share a common pattern: the witness returned by the perceptron phase identifies a direction or coordinate system in which the feasible region is thin, and the rescaling thickens that region enough to guarantee a constant-factor improvement.

6. Complexity, refinements, and adjacent large-margin variants

The complexity theory of rescaling perceptron algorithms is organized around two quantities: the number of basic-procedure updates per phase and the number of rescaling phases.

For symmetric cones, each of the $\text{find }x\in L\cap\Omega,$39 outer rounds invokes a basic procedure. In perceptron form this takes $\text{find }x\in L\cap\Omega,$40 updates; a smooth perceptron variant drives the same witness test in only $\text{find }x\in L\cap\Omega,$41 steps, at the price of one full Jordan-eigenvalue decomposition per step (Pena et al., 2015). Thus the total number of perceptron updates is

$\text{find }x\in L\cap\Omega,$42

depending on the basic-procedure implementation (Pena et al., 2015).

For linear programming, Theorem 2.3 of the improved deterministic algorithm states that with $\text{find }x\in L\cap\Omega,$43, one phase takes

$\text{find }x\in L\cap\Omega,$44

and since there are $\text{find }x\in L\cap\Omega,$45 phases, the total time is

$\text{find }x\in L\cap\Omega,$46

(Hoberg et al., 2016). The paper compares this with previous bounds: $\text{find }x\in L\cap\Omega,$47 for Dunagan–Vempala 2004 and

$\text{find }x\in L\cap\Omega,$48

for Peña–Soheili 2016 (Hoberg et al., 2016). The improvement comes from earlier rescaling, since lowering the amount of progress demanded from the inner phase reduces the cost of each phase while preserving a constant-factor gain in volume (Hoberg et al., 2016).

The 2018 limited-support enhancement improves the basic procedure in the projection-and-rescaling framework by keeping support size at most $\text{find }x\in L\cap\Omega,$49. The resulting iteration bound is $\text{find }x\in L\cap\Omega,$50, the per-iteration cost is $\text{find }x\in L\cap\Omega,$51, and the total cost of the basic procedure becomes

$\text{find }x\in L\cap\Omega,$52

improving on the original $\text{find }x\in L\cap\Omega,$53 basic-procedure complexity when $\text{find }x\in L\cap\Omega,$54 (Gutman, 2018).

The computational study of the orthant case reports that with the smooth perceptron basic procedure and $\text{find }x\in L\cap\Omega,$55, one has

$\text{find }x\in L\cap\Omega,$56

and because each rescaling doubles $\text{find }x\in L\cap\Omega,$57, the total work is

$\text{find }x\in L\cap\Omega,$58

for the primal side, with an analogous bound for the dual side (Pena et al., 2018). The reported numerical behavior is also specific: on controlled-condition instances with $\text{find }x\in L\cap\Omega,$59, across $\text{find }x\in L\cap\Omega,$60 up to $\text{find }x\in L\cap\Omega,$61, the average number of rescaling rounds was 9–15, total basic-procedure iterations were in the low thousands, and CPU times were under a minute even at $\text{find }x\in L\cap\Omega,$62; on mixed-feasible instances the partition was correctly identified in over $\text{find }x\in L\cap\Omega,$63 of trials; and on naive random instances EPRA was 5–10× faster than CPLEX, whereas on controlled-$\text{find }x\in L\cap\Omega,$64 instances CPLEX outperformed EPRA by about 10× (Pena et al., 2018).

An adjacent but distinct line of work uses “rescaling” internally in the perceptron update rather than externally in the geometry of the feasible set. In the weight-shrinking perceptron of Panagiotakopoulos and Tsampouka, if a pattern $\text{find }x\in L\cap\Omega,$65 violates the scaled margin

$\text{find }x\in L\cap\Omega,$66

the update is

$\text{find }x\in L\cap\Omega,$67

with either constant shrinking $\text{find }x\in L\cap\Omega,$68 or variable shrinking

$\text{find }x\in L\cap\Omega,$69

(Panagiotakopoulos et al., 2012). Standard perceptron with margin corresponds to $\text{find }x\in L\cap\Omega,$70 or $\text{find }x\in L\cap\Omega,$71, for which the guaranteed margin-fraction is $\text{find }x\in L\cap\Omega,$72 (Panagiotakopoulos et al., 2012). By contrast, the shrinking variants can provably attain in a finite number of steps any desirable approximation of the maximal margin hyperplane, with update complexity summarized as $\text{find }x\in L\cap\Omega,$73 in the small-$\text{find }x\in L\cap\Omega,$74 regime described in the paper (Panagiotakopoulos et al., 2012). This suggests that “rescaling perceptron algorithm” has at least two technical meanings in the literature: outer geometric rescaling for feasibility, and inner shrink-then-update rescaling for large-margin learning.

7. Interpretation and relation to neighboring methods

The rescaling perceptron algorithm occupies a junction between classical online learning updates, projection methods for conic feasibility, and interior-geometric conditioning. In the LP setting, it is connected to multiplicative weights update methods: the same deterministic rescaling can be combined with the potential

$\text{find }x\in L\cap\Omega,$75

to yield a polynomial-time rescaled-MWU algorithm with overall time

$\text{find }x\in L\cap\Omega,$76

(Hoberg et al., 2016). This establishes an explicit connection between perceptron-style feasibility search and a broader class of first-order methods.

In the projection-and-rescaling setting, the relation to Chubanov’s method is direct: the algorithm is projection-based, works with feasibility certificates and witnesses of poor scaling, and uses rescaling to improve a centrality measure until feasibility becomes easy to certify (Pena et al., 2015). The orthant specialization further links the method to complementarity structure through the Goldman–Tucker partition, allowing simultaneous recovery of relative-interior primal and dual solutions (Pena et al., 2018).

A common misconception is to treat the perceptron phase as the main algorithm and the rescaling phase as an implementation detail. The cited works indicate the opposite. The perceptron subroutine alone inherits the adverse dependence on $\text{find }x\in L\cap\Omega,$77 or on the depth of feasibility, whereas the rescaling phase is what converts this behavior into logarithmic dependence on a condition measure such as $\text{find }x\in L\cap\Omega,$78 or a linear factor in $\text{find }x\in L\cap\Omega,$79 (Hoberg et al., 2016, Pena et al., 2015). Another plausible implication is that improvements in support control, separation-oracle design, or eigenvalue computations are best understood as improvements to the basic procedure inside a larger conditioning framework rather than as stand-alone replacements for rescaling.

Across its variants, the rescaling perceptron algorithm is therefore best understood as a meta-algorithmic pattern: use a perceptron-type routine to expose poor geometry, transform the geometry by a provably beneficial map, and iterate until a feasible or most-interior solution is obtained. In polyhedral, orthant, and symmetric-cone settings alike, the theoretical and computational results center on that pattern (Hoberg et al., 2016, Pena et al., 2015, Pena et al., 2018, Gutman, 2018).