Piecewise-Affine Regularization (PARQ)

• Piecewise-Affine Regularization (PARQ) is a family of methods that enforce affine structures on unknown regions, promoting model interpretability and sparsity.
• It leverages formulations like coordinatewise penalties, CPWL regression, and ℓ₀ regularization, proven effective in regression, system identification, and motion estimation.
• Optimization techniques such as proximal gradient methods, ADMM, and dynamic programming efficiently solve PARQ objectives with strong convergence guarantees.

Piecewise-Affine Regularization (PARQ) denotes a family of regularization strategies and variational frameworks that enforce or exploit piecewise-affine structure in the learned mappings, model parameters, or signal representations. PARQ has emerged across regression, system identification, dense motion estimation, and quantization-aware neural network training. The defining characteristic is the introduction of priors or penalties that induce the function or parameter vector to be exactly (or nearly) piecewise-affine—i.e., affine on unknown regions, with sharp transitions determined via optimization. This approach provides model interpretability, structural parsimony, and computational tractability in several high-dimensional learning contexts (Pourya et al., 2022, Breschi et al., 2020, Fortun et al., 2018, Jin et al., 19 Mar 2025, Ma et al., 14 Aug 2025).

1. Mathematical Formulations of Piecewise-Affine Regularizers

PARQ imposes piecewise-affinity through explicit parameterizations, convex composite penalties, or combinatorial regularizers. Several formulations instantiate the concept:

• Coordinatewise Piecewise-Affine Penalties: For quantization or parameter clustering, the regularizer operates per coordinate:

$\Psi(x) = \sum_{i=1}^d \Psi(x_i)$

where $\Psi(x_i)$ is a continuous, piecewise-affine function symmetric about zero. A general parameterization,

$$\Psi(x) = a_k(|x|-q_k) + b_k \quad \text{for } |x| \in [q_k, q_{k+1}]$$

with slopes $a_0 < a_1 < \cdots < a_m = +\infty$ and quantization targets $\mathcal{Q} = \{0, \pm q_1, ..., \pm q_m\}$, enforces clustering around $\mathcal{Q}$ (Ma et al., 14 Aug 2025, Jin et al., 19 Mar 2025).

• CPWL Regression via Triangulation: Given a Delaunay triangulation $T$ of $\Omega \subset \mathbb{R}^d$ with vertices $V = \{v_1,...,v_N\}$, any CPWL function $f$ is determined by its values $\Psi(x_i)$0 at the vertices. The regularization is constructed via the Hessian Total Variation,

$\Psi(x_i)$1

driving gradient jumps $\Psi(x_i)$2 across simplex facets towards zero, reducing the number of affine pieces (Pourya et al., 2022).

• $\Psi(x_i)$3 Regularization over Parameter Fields: In dense labeling problems (e.g., motion estimation), an $\Psi(x_i)$4 norm counts discontinuities of an affine parameter field $\Psi(x_i)$5, penalizing the number of boundaries and establishing piecewise-affinity on unknown regions (Fortun et al., 2018).

These formulations enable the translation of discrete, combinatorial model-selection or quantization tasks to continuous optimization with rigorous analytic properties.

2. Optimization Algorithms and Proximal Mapping Properties

Solving PARQ objectives generally involves non-differentiable, possibly nonconvex regularizers, but their special structure frequently allows efficient algorithms with closed-form or efficiently solvable proximal operators.

• Proximal Gradient and Variants: For objectives $\Psi(x_i)$6 (with smooth $\Psi(x_i)$7), PARQ admits coordinatewise closed-form proximal mappings due to the piecewise-affine structure. For convex $\Psi(x_i)$8, the proximal operator produces "quantization intervals" that widen with stronger regularization, yielding hard quantization as a limit (Ma et al., 14 Aug 2025).
• ADMM and Alternating Minimization: In CPWL regression and motion estimation, Alternating Direction Method of Multipliers (ADMM) exploits problem sparsity and structure. Updates alternate between fitting affinities on regions or scanlines and enforcing consensus, with per-iteration cost scaling linearly with the number of nonzeros in sparse matrices defining the model (Pourya et al., 2022, Fortun et al., 2018).
• Dynamic Programming for 1D Partitioning: In image/sequence settings, 1D partitions admitting affine fits can be computed efficiently via dynamic programming recurrences for all scanlines, reducing runtime virtually to linear in input size when parallelized (Fortun et al., 2018).
• Aggregate Proximal (AProx) for Stochastic Training: For quantization-aware neural networks, the AProx algorithm maintains latent and proximalized sequences, performing aggregate soft-quantization steps that transition to hard quantization as the aggregate step weight grows (Jin et al., 19 Mar 2025).
• Coordinate-Descent with Structure Selection: For piecewise-affine ARX models, coordinate-descent alternates parameter fits, polyhedral boundary estimation, and mode-label assignment, with heuristic shrinkage (via $\Psi(x_i)$9, $$\Psi(x) = a_k(|x|-q_k) + b_k \quad \text{for } |x| \in [q_k, q_{k+1}]$$0, or convex combinations) driving modes and regressors to sparsity and thus automatically pruning the model (Breschi et al., 2020).

These algorithms crucially exploit the piecewise-affine structure for efficiently solvable updates, often with strong convergence guarantees in convex or overparameterized regimes.

3. Applications Across Learning and Signal Processing

PARQ methodologies have been developed and validated in multiple learning and estimation domains:

• Regression and Function Learning: CPWL functions regularized with Hessian total variation on triangulations yield interpretable, stable, and competitive regression models against neural nets, particularly in low- and moderate-dimensional data. The generalized LASSO framework enables efficient learning and active control over model complexity through a single hyperparameter (Pourya et al., 2022).
• System Identification and Model Selection: For Piecewise-Affine ARX (PWARX) systems, regularized formulations coupled with automated heuristics reliably recover both the number of affine regions and regressor order, outperforming classical combinatorial search approaches. Shrinkage strategies identify redundant modes and select parsimonious models (Breschi et al., 2020).
• Dense Motion Estimation: By penalizing the number of discontinuities in parameter fields, piecewise-affine motion is estimated without explicit segmentation. This yields sharper motion boundaries, lower computational cost independent of the number of segments, and consistently competitive or superior accuracy to TV/TGV-based methods (Fortun et al., 2018).
• Quantization-Aware Neural Network Training: Coordinatewise convex PARs induce clustering of weights to discrete quantization levels, enabling quantization-aware training (QAT) with provable convergence. The PARQ approach matches or surpasses standard STE and Moreau-smoothing baselines on state-of-the-art convolutional and transformer architectures, especially at ultra-low bitwidths (Jin et al., 19 Mar 2025, Ma et al., 14 Aug 2025).
• General Supervised Learning: In the overparameterized regime ($$\Psi(x) = a_k(|x|-q_k) + b_k \quad \text{for } |x| \in [q_k, q_{k+1}]$$1), PAR-regularized empirical risk minimization ensures that most coordinates are exactly quantized, with quantization rate at least $$\Psi(x) = a_k(|x|-q_k) + b_k \quad \text{for } |x| \in [q_k, q_{k+1}]$$2, holding under broad model classes and sample assumptions (Ma et al., 14 Aug 2025).

4. Theoretical and Statistical Guarantees

PARQ frameworks come equipped with rigorous mathematical guarantees linking optimization and statistical accuracy:

• Quantization Rates in High Dimensions: For problems with more parameters than samples, every critical point of the PARQ-regularized loss exhibits at least $$\Psi(x) = a_k(|x|-q_k) + b_k \quad \text{for } |x| \in [q_k, q_{k+1}]$$3 of its coordinates exactly at quantization levels, with this property formalized by Clarke criticality and genericity conditions on the data (Ma et al., 14 Aug 2025).
• Statistical Error Bounds: Given suitable construction of the piecewise-affine penalty, PARQ can approximate $$\Psi(x) = a_k(|x|-q_k) + b_k \quad \text{for } |x| \in [q_k, q_{k+1}]$$4 (ridge), $$\Psi(x) = a_k(|x|-q_k) + b_k \quad \text{for } |x| \in [q_k, q_{k+1}]$$5 (lasso), and certain nonconvex penalties (e.g., SCAD, MCP), with matching minimax error rates for regression; e.g., $$\Psi(x) = a_k(|x|-q_k) + b_k \quad \text{for } |x| \in [q_k, q_{k+1}]$$6 for sparsity, or $$\Psi(x) = a_k(|x|-q_k) + b_k \quad \text{for } |x| \in [q_k, q_{k+1}]$$7 in ridge-like settings. Risk bounds transfer from classical regularization to their PARQ approximations with $$\Psi(x) = a_k(|x|-q_k) + b_k \quad \text{for } |x| \in [q_k, q_{k+1}]$$8 accuracy for quantization grid gap $$\Psi(x) = a_k(|x|-q_k) + b_k \quad \text{for } |x| \in [q_k, q_{k+1}]$$9 (Ma et al., 14 Aug 2025).
• Optimization Convergence: For convex or $a_0 < a_1 < \cdots < a_m = +\infty$0-smooth losses and convex PAR, standard proximal-gradient iterations decrease the objective monotonically, and limit points correspond to critical points. In stochastic settings with diminishing step sizes, last-iterate convergence in expectation is established at $a_0 < a_1 < \cdots < a_m = +\infty$1 rates (Jin et al., 19 Mar 2025).
• Combinatorial to Convex Relaxation: PARQ provides a principled relaxation of combinatorial quantization or model selection, translating intractable discrete optimization problems into continuous ones amenable to scalable gradient-based methods, while retaining the essential property of "snapping" to discrete/affine patterns in the global or local minimizers.

5. Model Selection, Shrinkage, and Hyperparameter Strategies

PARQ enables effective model structure selection through the interaction of regularization strength, norms, and coordinatewise shrinkage:

• Automatic Mode and Structure Pruning: Mixed-norm penalties (e.g., $a_0 < a_1 < \cdots < a_m = +\infty$2–$a_0 < a_1 < \cdots < a_m = +\infty$3) applied per affine mode in ARX/piecewise-linear models drive negligible modes to zero, supporting iterative pruning strategies that reduce mode count without grid search (Breschi et al., 2020).
• Regressor Order Selection: Elastic net–style or $a_0 < a_1 < \cdots < a_m = +\infty$4 penalties on AR/X lags across all modes allow detection and removal of inactive lag coefficients, recovering true regressor order in most trials without combinatorial enumeration (Breschi et al., 2020).
• Quantizer Parameter Selection: For quantization-aware training, quantizer levels $a_0 < a_1 < \cdots < a_m = +\infty$5 are periodically updated via least-squares binary quantization (LSBQ) on the current "latent" parameter distribution, adapting to the evolving parameter landscape (Jin et al., 19 Mar 2025).
• Regularization Schedule: Schedules that anneal the "inverse-slope" or regularization parameter allow a transition from soft to hard clustering/quantization, ensuring initial stability and final sharp partitioning (Jin et al., 19 Mar 2025).
• Efficiency and Practicality: The combination of coordinatewise shrinkage and blockwise updates permits rapid convergence and avoids exhaustive search over model or quantization parameters, resulting in up to $a_0 < a_1 < \cdots < a_m = +\infty$6 speedups over classical combinatorial/penalized MIQP schemes for structure selection (Breschi et al., 2020).

6. Benchmarks, Computational Characteristics, and Practical Considerations

Empirical studies and benchmarks demonstrate the practical efficacy and computational properties of PARQ approaches:

• Convexity and Sparsity: Many PARQ-based problems are convex (or blockwise convex) in parameters, enabling fast solvers based on ADMM, FISTA, or proximal gradient, with per-iteration cost scaling with the sparsity of underlying data matrices (Pourya et al., 2022, Breschi et al., 2020, Fortun et al., 2018).
• Scalability: CPWL regression and piecewise-affine motion algorithms routinely handle thousands of vertices, facets, or scanlines. Memory and computation remain manageable due to the local support of basis functions and discontinuity penalties.
• Accuracy and Parsimony: In multiple domains—terrain regression, power-plant modeling, motion estimation—PARQ-based models achieve accuracy comparable to, or better than, deep networks or other state-of-the-art baselines, using far fewer degrees of freedom and sparser solutions (Pourya et al., 2022, Fortun et al., 2018).
• No Need for Segmentation or Initialization: Piecewise-affine motion estimation proceeds without pre-segmentation or explicit region labeling, avoiding sensitivity to initialization and reducing susceptibility to local minima characteristic of alternating segmentation–estimation schemes (Fortun et al., 2018).
• Sharpening and Blurring Properties: Compared to TSV/TGV regularizers, PARQ produces solutions with sharper edges and less tendency to staircasing, due to the explicit affine-piece prior (Fortun et al., 2018).
• Open-Source Implementations: PARQ for neural quantization has been released in the PyTorch ecosystem, providing ready access to all major algorithmic components for reproduction and extension (Jin et al., 19 Mar 2025).

---

PARQ unifies a broad class of regularization and modeling techniques characterized by their enforcement of affine structure within unknown regions, their tractable algorithmic implementations, and their rigorous analytical and statistical underpinnings. Research demonstrates their efficacy in regression, system identification, motion estimation, and quantization-aware deep learning, with wide applicability wherever piecewise-affinity is a natural prior or modeling desideratum (Pourya et al., 2022, Breschi et al., 2020, Fortun et al., 2018, Jin et al., 19 Mar 2025, Ma et al., 14 Aug 2025).