Polyphase Decomposition: Fundamentals & Applications

• Polyphase Decomposition is a digital signal processing technique that expresses signals and filters as interleaved subcomponents to optimize analysis and synthesis.
• In multirate filter banks, this method reduces redundant computations by splitting filters into sub-filters, enabling efficient downsampling and upsampling with lower algorithmic complexity.
• Applied in deep learning, polyphase techniques enhance CNN pooling by enabling adaptive phase selection and learned mixtures to improve shift equivariance and robustness.

Polyphase decomposition is a foundational concept in digital signal processing and multirate systems, allowing efficient restructuring of linear and nonlinear signal operations such as filtering, downsampling, and upsampling. At its core, polyphase decomposition expresses a signal or filter in terms of multiple interleaved subcomponents ("phases"), facilitating both algorithmic efficiency and analytical clarity. The framework supports diverse applications including filter-bank design, efficient digital decimation, and shift-equivariant operations in convolutional neural networks (CNNs).

1. Mathematical Formulation of Polyphase Decomposition

Let $x[n]$ denote a discrete-time sequence and $M\geq 2$ an integer stride (decimation or interpolation factor). The $M$ polyphase components $x_i[n]$ for $i=0,\ldots,M-1$ are defined as

$x_i[n] = x[M n + i]$

and, in the $z$-domain,

$X(z) = \sum_{i=0}^{M-1} z^{-i} X_i(z^M)$

where $X_i(z) = \sum_n x_i[n] z^{-n}$. In two dimensions, with stride $s$ in each spatial direction, the $M\geq 2$0 polyphase tiles are

$M\geq 2$1

This decomposition enables blockwise signal manipulation and is key for both linear (filter banks) and nonlinear (neural network pooling) architectures (Chaman et al., 2021, Torres et al., 2015, Saha et al., 2024, 0707.2436).

2. Polyphase Decomposition in Multirate Filter Banks

In the context of multirate filtering, polyphase decomposition is used to express any finite impulse response (FIR) filter of length $M\geq 2$2 as $M\geq 2$3 sub-filters, each processing a different phase of the input, thereby reducing redundant computations in decimation or interpolation. The FIR system

$M\geq 2$4

can be reformulated as

$M\geq 2$5

with $M\geq 2$6 an FIR polynomial constructed from filter taps $M\geq 2$7. In multirate notation, multiple filters $M\geq 2$8 can be organized into an $M\geq 2$9 polyphase matrix $M$0, and perfect reconstruction requires synthesis polyphase matrix $M$1 satisfying $M$2. This approach forms the backbone of efficient FIR decimators and filter-bank implementations (Torres et al., 2015, 0707.2436).

In generalized comb filter (GCF) architectures, partial polyphase decomposition allows the computationally intensive portion of the filter to operate at a much lower output rate after decimation, greatly enhancing hardware efficiency and enabling multiplier-less realizations without loss of quantization noise rejection or filter selectivity (0707.2436).

3. Polyphase Decomposition in CNNs: Shift Equivariance and Pooling

Downsampling operators in CNNs, such as strided convolutions and max-pooling, typically break shift equivariance. Techniques based on polyphase decomposition have been proposed to restore equivariance or invariance:

• Adaptive Polyphase Downsampling (APS-D): Given the two stride-2 phases $M$3, $M$4, APS-D selects the phase with highest $M$5 norm (the “stronger” phase). For any pixel shift $M$6, the output is appropriately shifted, so distinct shifts are never collapsed (“$M$7-equivariance”). The formal property is
  • For $M$8 even: $M$9
  • For $x_i[n]$0 odd: $x_i[n]$1, where $x_i[n]$2 is the selected phase index (Chaman et al., 2021).
• Adaptive Polyphase Upsampling (APS-U): To invert APS-D, APS-U upsamples by zero-insertion and realigns the grid by a circular shift equal to the encoder’s phase index. This recovers lost offsets and yields

$x_i[n]$3

establishing true shift equivariance in symmetric encoder-decoder networks.

• Translation Invariant Polyphase Sampling (TIPS): Instead of hard selection or uniform averaging, TIPS learns a shift-invariant linear mixture of $x_i[n]$4 two-dimensional polyphase components. A fully convolutional sub-network computes mixture weights $x_i[n]$5 per channel and location, resulting in output

$x_i[n]$6

The network is regularized to avoid degenerate “one-hot” or uniform mixing. TIPS achieves the lowest maximum-sampling bias (MSB) and the highest empirical shift invariance among pooling schemes (Saha et al., 2024).

4. Algorithmic and Hardware Efficiency

Polyphase decomposition fundamentally alters the computational requirements of filtering and resampling operations. Rather than filtering at full input rate and discarding $x_i[n]$7 out of every $x_i[n]$8 outputs, the polyphase approach allows all major operations to be performed after downsampling, leading to an algorithmic complexity reduction from $x_i[n]$9 to $i=0,\ldots,M-1$0 per output, where $i=0,\ldots,M-1$1 is the sub-filter order (Torres et al., 2015, 0707.2436).

A summary of computational benefits in digital decimation is shown below:

Approach	Per-output complexity	Notable property
Direct filtering + decimate	$i=0,\ldots,M-1$2	All computations at full rate
Polyphase decomposition	$i=0,\ldots,M-1$3	Major computations after decimation

In generalized comb filter design, a partial polyphase decomposition pushes multipliers to the lowest possible post-decimation rates, enabling multiplier-less implementation at high decimation factors (0707.2436). In SAR POLYPHASE resampling, this approach also provides resilience to nonuniform sampling and allows real-time, in-place processing (Torres et al., 2015).

5. Applications Across Signal Processing and Deep Learning

Polyphase decomposition is integral in:

• Digital Downsampling/Resampling: Essential in SAR image formation, where variable pulse repetition frequencies (PRFs) are unified and efficiently resampled using polyphase FIR structures. These methods handle arbitrary, aperiodic gating and missing data, with normalization compensating for variable branch contributions (Torres et al., 2015).
• Filter-Bank Design: Underlies perfect reconstruction filter banks, wavelet transforms, and efficient implementation of multirate systems.
• Generalized Comb Filters: Enables the realization of high-order anti-aliasing decimation filters with strong selectivity and quantization noise suppression, supporting fully multiplier-less architectures suitable for ASIC/FPGA (0707.2436).
• CNN Downsampling/Pooling: Recent work adapts polyphase decomposition for translation-invariant pooling and adaptive downsampling, addressing the breakage of shift equivariance inherent in conventional strided layers and max-pooling. Adaptive selection (APS-D/U) and learned mixing (TIPS) provide architectures with improved shift stability and robustness across a range of computer vision tasks (Chaman et al., 2021, Saha et al., 2024).

6. Theoretical Guarantees and Empirical Performance

For nonlinear polyphase operations in CNNs, several key results are established:

• Shift-Equivariance of APS-D/U: The combination of nonlinear phase selection and explicit grid realignment yields perfect shift-equivariance in symmetric encoder-decoder networks:

$i=0,\ldots,M-1$4

with corresponding results at the network level for U-Nets whose down-/up-sampling is entirely built from these operations (Chaman et al., 2021).

• Maximum-Sampling Bias and Shift Invariance: High MSB correlates negatively with shift invariance in CNNs. TIPS achieves a substantially lower MSB and outperforms both hard-selection and average methods with respect to shift consistency, robustness to adversarial perturbation, and overall accuracy in extensive empirical evaluations (Saha et al., 2024).
• Effect of Coefficient Quantization: In GCF architectures, quantizing polyphase branch coefficients to powers-of-two, with error less than $i=0,\ldots,M-1$5, leads to no measurable performance loss in quantization-noise rejection, establishing their practical viability for resource-constrained implementation (0707.2436).

7. Practical Considerations and Design Tradeoffs

Polyphase decomposition provides significant flexibility:

• Linear versus Nonlinear Analysis/Synthesis: Classic multirate systems perform linear analysis (e.g., FIR filtering of each phase) and synthesis (e.g., recombination), whereas modern CNN approaches employ nonlinear selection (APS-D) or learned mixtures (TIPS) instead of weighting or summing phases (Chaman et al., 2021, Saha et al., 2024).
• Memory and Throughput: In signal-processing hardware, the polyphase architecture minimizes memory requirements and maximizes throughput by distributing sub-filters and exploiting parallelism (Torres et al., 2015, 0707.2436).
• Pooling Strategy Selection in CNNs: Maximum-sampling–based pooling (e.g., max-pool, APS-D) is prone to phase instability and high MSB, while linear mixing (TIPS) achieves better invariance at a modest computational overhead (Saha et al., 2024).
• Nonuniform and Sparse Sampling: Polyphase methods naturally extend to nonuniformly sampled inputs (e.g., SAR with variable PRF), with per-output normalization accommodating variable available samples (Torres et al., 2015).

A plausible implication is that future architectures in both digital signal processing and deep learning will leverage polyphase decomposition principles—possibly in hybrid, nonlinear, or hardware-aware forms—to optimally balance efficiency, invariance, and implementation complexity.