Distortion-Aware Hybrid Beamforming
- Distortion-aware hybrid beamforming is a technique that integrates hardware impairments, such as nonlinear PA distortions and quantization errors, into the beamforming design.
- It employs advanced optimization frameworks and combined-feedback digital predistortion to mitigate beamformed distortion and improve spectral and energy efficiency.
- The approach adapts to various architectures—including fully/partially connected arrays and RIS-aided systems—to maintain robust performance in realistic mmWave environments.
Distortion-aware hybrid beamforming denotes a class of hybrid analog/digital beamforming methods in which nonideal array, RF-front-end, or propagation effects are embedded in the beamforming model, optimization criterion, or linearization target instead of being treated as architecture-independent white noise. In the mmWave MIMO literature, the dominant interpretation is power-amplifier-aware hybrid beamforming, where the precoder shapes not only the desired signal but also the spatial covariance of nonlinear inband distortion (Moghadam et al., 2018). Closely related work extends the idea to subarray-level digital predistortion in hybrid transmitters (Abdelaziz et al., 2018), architecture-aware spatio-frequency Bussgang models for fully connected and partially connected arrays (Salman et al., 2020), integrated sensing and communication (ISAC) under nonlinear amplification (Zhang et al., 18 Jul 2025), coarse or quantized analog hardware (Wu, 2018, Zhang et al., 3 Jan 2025), electromagnetic radiation-pattern distortion and mutual coupling (Ji et al., 2024, Deshpande et al., 25 Feb 2025), and mobility-induced Doppler/ICI effects in active/passive beamforming for RIS-aided vehicular links (Li et al., 28 Feb 2026). The common principle is that the analog network, connectivity, and hardware operating point alter the effective channel, distortion field, or both, so beamforming and impairment modeling cannot be cleanly separated.
1. Scope and meanings of “distortion-aware”
The literature uses the term across several non-identical distortion mechanisms. In all cases, the beamformer is designed or analyzed against a distortion source that is structured by the hybrid architecture rather than appended afterward as a scalar penalty.
| Distortion notion | Representative model | Representative paper |
|---|---|---|
| PA-induced inband distortion | Distortion covariance as a function of pre-PA beamformed covariance | (Moghadam et al., 2018) |
| Beamformed aggregate distortion for DPD | One DPD per subarray learned from combined PA observation | (Abdelaziz et al., 2018) |
| Effective-channel / beam-pattern distortion | Post-analog channel with one-bit phase shifters | (Wu, 2018) |
| Quantization-aware hybrid precoding | DAC attenuation and discrete phase-shifter codebooks | (Zhang et al., 3 Jan 2025) |
| Electromagnetic pattern distortion | Coupling matrix , radiation patterns, and current-vector analog control | (Ji et al., 2024) |
| Mobility-induced distortion | Doppler-driven ICI and training-overhead/channel-aging tradeoff | (Li et al., 28 Feb 2026) |
In standard mmWave transmit architectures, hybrid precoding is typically expressed as a digital/baseband stage followed by an analog/RF network and then per-antenna PAs. A decisive consequence is that the PA inputs are already spatially correlated before amplification. In partially connected ISAC arrays, the analog matrix is block diagonal, whereas in fully connected architectures each PA input is a mixture of multiple RF-chain signals; these distinctions directly change distortion structure (Moghadam et al., 2018, Zhang et al., 18 Jul 2025, Salman et al., 2020).
A broader but adjacent line of work treats mismatch-aware or robustness-aware receive HAD beamforming, especially under DOA mismatch and phase-only combining constraints. That literature is relevant to distortion-aware hybrid beamforming only indirectly, because its “distortion” is primarily steering-vector/model mismatch rather than RF-front-end impairment (Almagboul et al., 2018).
2. Nonlinear PA distortion as a beamformed field
In the PA-dominant formulation for single-user mmWave hybrid MIMO, the beamformed signal before the PA bank is
with . Each PA is modeled as a memoryless odd-order polynomial,
and the transmit vector is decomposed as
where is the average linear gain and is zero-mean distortion uncorrelated with . The central result is that the distortion covariance 0 is a deterministic function of 1, built from Hadamard powers of 2 and per-antenna gain factors. Consequently, the distortion is not spatially white and not independent across antennas in general; it is partially beamformed by 3 and 4 (Moghadam et al., 2018).
This changes the usual large-array intuition. Narrow desired beams do not imply that nonlinear distortion becomes negligible. The same spatial correlation that sharpens the useful beam can also shape inband distortion toward the intended receiver. In the received model
5
the effective noise-plus-interference covariance is
6
so spectral efficiency depends on a colored, beamformer-dependent distortion term rather than a scalar noise floor (Moghadam et al., 2018).
The ISAC extension adopts a third-order Bussgang model. With overall beamformer 7, the nonlinear PA output is
8
with
9
Thus both the effective linear gain and the additive distortion covariance depend on 0. User-1 SINR includes 2, and sensing MI includes 3, so the precoder shapes distortion in both communication and sensing directions (Zhang et al., 18 Jul 2025).
A plausible implication is that “distortion-aware” hybrid beamforming is fundamentally covariance-aware: once the hybrid precoder determines per-antenna powers and cross-antenna correlations, it also determines where nonlinear distortion is radiated and how strongly it enters the relevant directions.
3. Linearization and beam-domain mitigation
In hybrid subarray transmitters, one digital stream drives a whole analog-beamformed subarray. Because only one DPD can be placed before the analog network of that subarray, perfect per-PA linearization is generally impossible when the PAs differ. The subarray-level solution is to learn a single DPD from the combined observation of all PA outputs after anti-beamforming or co-phasing, so that the feedback waveform emulates the signal that adds coherently in the intended beam direction. For branch weights 4 and PA outputs 5, the effective main-beam signal is
6
and the observation signal can be formed as
7
DPD learning then minimizes the correlation between the residual error 8 and orthogonalized nonlinear basis functions, using the block-adaptive decorrelation update
9
This directly targets the beamformed distortion seen in the intended direction (Abdelaziz et al., 2018).
That viewpoint is architecture-specific rather than per-device-specific. The linearization target is the aggregate radiated distortion in the main beam, not the individual distortion of each PA. Under realistic measured PA models from the Lund University massive MIMO testbed, the combined-feedback DPD reduced main-beam EVM from 0 without DPD to 1, and improved ACLR L/R from 2 dBc to 3 dBc. The reference method that learned from one PA output only achieved 4 EVM and 5 dBc ACLR L/R, so the combined-feedback design delivered more than 6 dB ACLR gain relative to that single-PA-learning baseline (Abdelaziz et al., 2018).
A second architecture-aware linearization framework models PA distortion via spatio-frequency Bussgang decomposition for hybrid OFDM arrays. In fully connected architectures, the frequency-domain relation is
7
where 8 is a matrix-valued nonlinearized effective beamformer and 9 is a full spatial covariance. In partially connected arrays, the decomposition becomes subarray-wise and the distortion covariance is block diagonal. Anti-beamforming feedback is then used to reduce the 0-dimensional PA output to 1 observed streams, enabling one DPD per RF chain. The analysis shows that classical per-RF-chain DPD is effective for partially connected arrays and best for fully digital arrays, but inadequate for fully connected arrays because each PA input is a superposition of all RF-chain signals and the nonlinear cross-terms cannot be inverted by branchwise DPD alone (Salman et al., 2020).
A recurring conclusion across these linearization papers is that hybrid beamforming changes the correct observation model. Distortion suppression is most effective when feedback reconstructs the spatially combined signal that the array actually radiates in the dominant direction.
4. Architecture-dependent distortion beyond PA nonlinearity
Not all distortion-aware hybrid beamforming papers treat distortion as additive nonlinear RF noise. In wireless fronthaul with one-bit analog phase shifters, the central distortion mechanism is the effective or distorted channel after analog precoding: 2 The digital precoder is computed from an SLNR generalized eigenproblem using only 3, not the inaccessible true 4, and a genetic algorithm searches the discrete one-bit analog precoder space. This formulation is distortion-aware in the sense of being phase-quantization-aware and effective-channel-aware. Its main practical warning is that hybrid SLNR with one-bit phase shifters can create strong undesired beams and hence intercell interference in multicell wireless fronthaul, even when single-cell sum rate remains close to digital beamforming (Wu, 2018).
A different hardware-aware formulation models DAC quantization and finite-resolution phase shifters in single-user narrowband mmWave. The analog precoder is factorized as
5
and DAC quantization is introduced through
6
with diagonal attenuation 7. Phase shifters are restricted to a discrete alphabet
8
The resulting energy-efficiency optimization jointly tunes digital precoding, switch connectivity, analog phases, and DAC resolution. The paper explicitly introduces additive quantization noise but then neglects it in later derivations because its covariance is argued to be very small for the DAC resolutions of interest; accordingly, its strongest distortion-aware content is the joint modeling of multiplicative DAC attenuation and discrete phase constraints, rather than full quantization-noise covariance propagation (Zhang et al., 3 Jan 2025).
Electromagnetic hybrid beamforming shifts the locus of distortion from RF blocks to radiation physics. For a 3D holographic array, the emitted field is
9
where 0 models mutual coupling and 1 is the analog current vector. Coupling-aware directivity maximization yields
2
Here the distortion-aware step is not linearization of a nonlinear PA but compensation of pattern distortion induced by sector non-uniformity and coupling. The effective electromagnetic channel already contains the array pattern and coupling effects, so digital precoding is optimized on that channel rather than on an RF-only abstraction (Ji et al., 2024).
Hybrid reconfigurable parasitic arrays move further toward a circuit-theoretic interpretation. With one active antenna and multiple parasitic elements, the induced parasitic currents satisfy
3
and the beam pattern is
4
The analog “weights” are therefore not free complex coefficients but functions of mutual impedances and realizable reactances. A shift-of-origin transformation maps the resulting coupled magnitude-phase constraint into a unit-modulus variable, enabling a closed-form reactance design. This suggests that, in parasitic hybrid arrays, distortion-awareness is essentially realizability-awareness (Deshpande et al., 25 Feb 2025).
5. Optimization objectives and algorithmic frameworks
Once the distortion source is specified, the optimization criterion changes materially. In distortion-aware ISAC with partially connected hybrid beamforming, the objective is the weighted sum
5
subject to a power constraint imposed after nonlinear amplification: 6 The solution strategy first optimizes a fully digital beamformer 7 via alternating optimization over 8, 9, and 0; the 1-subproblem is solved on a constant-Frobenius-norm manifold with Riemannian conjugate-gradient updates and Armijo line search, while 2 and 3 admit closed-form updates. The optimized 4 is then decomposed into block-diagonal unit-modulus 5 and digital 6 through a hybrid decomposition stage (Zhang et al., 18 Jul 2025).
In effective-channel-aware fronthaul, the analog search variable is discrete and the digital stage is simple. For fixed one-bit analog precoder 7, the digital beamformer is the dominant generalized eigenvector of
8
and a genetic algorithm evolves the analog precoder population to maximize a low-SNR approximation of sum rate,
9
The architecture is thus distortion-aware through the measured post-analog channel rather than through explicit RF impairment penalties (Wu, 2018).
Electromagnetic hybrid beamforming for 3D holographic arrays uses alternating optimization with semidefinite relaxation. The single-user closed-form updates
0
lead to multi-user formulations lifted into semidefinite variables and solved alternately. The analog stage is initialized from the coupling-aware directivity solution 1, making environmental and electromagnetic adaptation explicit (Ji et al., 2024).
In RIS-aided vehicular communication, the dominant impairment is mobility. The system first performs two-timescale grouped channel estimation, then optimizes active BS beamforming and grouped RIS coefficients. In the single-VUE narrowband case, active beamforming uses water-filling over the grouped equivalent channel, while grouped RIS phases are updated either in closed form for single-element groups or by gradient ascent for larger groups. In the broadband multi-VUE case, subcarrier assignment, power allocation, active beamforming, and passive RIS beamforming are jointly optimized under explicit Doppler-ICI terms, relaxed binary assignment, and difference-of-concave approximation. The beamforming problem is therefore distortion-aware because the resource-allocation variables and beamformers are optimized against ICI that depends on velocity and subcarrier spacing (Li et al., 28 Feb 2026).
A plausible synthesis is that distortion-aware hybrid beamforming has no single canonical solver. Its algorithmic structure is inherited from the distortion mechanism: manifold optimization for nonlinear post-PA objectives, decorrelation learning for combined-radiation DPD, discrete search for quantized analog networks, SDR for coupling-aware electromagnetic synthesis, and two-timescale or D.C.-programming tools for Doppler-aware active/passive designs.
6. Empirical regularities, misconceptions, and scope limits
Several empirical patterns recur across the literature. First, larger arrays and narrower beams do not eliminate nonlinear distortion. In mmWave hybrid MIMO with nonlinear PAs, increasing transmit power as the number of transmit antennas grows large can be counter-effective in terms of energy efficiency, and at high transmit power analog beamforming can achieve higher spectral and energy efficiency than digital and hybrid beamforming because the architecture changes both PA operating conditions and total power consumption (Moghadam et al., 2018). Second, linearization targeted at the beamformed aggregate signal can markedly outperform per-device surrogates: the combined-feedback subarray DPD reached 2 EVM and 3 dBc ACLR L/R, versus 4 and 5 dBc for single-PA-learning DPD (Abdelaziz et al., 2018). Third, in distortion-aware ISAC, reducing nonlinear radiation toward important directions can be more valuable than maximizing linear beam power alone: the reported design produced about 6 dB reduction in nonlinear power toward the user direction while having about 7 dB weaker linear power than MRT because of the nonzero sensing weight (Zhang et al., 18 Jul 2025).
The electromagnetic and hardware-aware literature reveals analogous tradeoffs. The 3D holographic EHB scheme was reported to achieve only around 8 average gain in single-user transmission at 9 dB SNR, but in multi-user settings it achieved an average sum-rate gain of over 0 relative to conventional beamforming algorithms because pattern programmability and coupling-aware design significantly reduced inter-user interference (Ji et al., 2024). The dual-switch mmWave architecture with DAC-bit optimization reported spectral-efficiency loss below 1 relative to the fully connected architecture while improving energy efficiency; under imperfect CSI with 2 at 3 dB SNR, its energy efficiency was about 4 bits/Hz/J versus about 5 bits/Hz/J with perfect CSI, which the paper interpreted as around 6 loss (Zhang et al., 3 Jan 2025). In effective-channel-aware fronthaul, hybrid SLNR with one-bit or two-bit phase shifters was approximately 7 to 8 worse than digital SLNR from 9 dB to 0 dB, with the gap reducing to about 1 in the moderate-SNR regime, yet the same coarse analog network could create undesired beams that matter in multicell operation (Wu, 2018).
A common misconception is that distortion-aware hybrid beamforming refers only to PA nonlinearity. The surveyed work shows otherwise: in some papers it denotes post-analog effective-channel distortion and beam-pattern distortion from finite phase resolution (Wu, 2018); in others it denotes radiation-pattern distortion and mutual coupling (Ji et al., 2024), realizability constraints in parasitic arrays (Deshpande et al., 25 Feb 2025), or Doppler-induced ICI and training-overhead/channel-aging tradeoffs (Li et al., 28 Feb 2026). A second misconception is that more degrees of freedom necessarily improve distortion-limited performance. Coarse evidence runs in both directions: more RF chains or more flexible digital excitation can worsen PA operating conditions at high power (Moghadam et al., 2018), more subcarriers can worsen OFDM performance in high mobility because they reduce 2 and intensify ICI (Li et al., 28 Feb 2026), and more parasitic elements do not necessarily keep improving spectral efficiency beyond 3 in the reported examples (Deshpande et al., 25 Feb 2025).
The scope limits are equally consistent. Much of the PA-nonlinearity literature is narrowband or single-carrier, assumes Gaussian signaling, and focuses on identical memoryless polynomial or PH PA models (Moghadam et al., 2018, Abdelaziz et al., 2018, Zhang et al., 18 Jul 2025). Some optimization stages are indirect: the ISAC design optimizes a fully digital target before hybrid decomposition (Zhang et al., 18 Jul 2025), and the energy-efficient mmWave architecture introduces additive DAC quantization noise but later neglects it and approximates 4 in the beamformer block (Zhang et al., 3 Jan 2025). Electromagnetic formulations abstract several implementation details into real-time programmability of currents or circuit parameters (Ji et al., 2024), while the vehicular RIS work is “hybrid” in the active/passive sense and not in the conventional RF-chain/phase-shifter sense (Li et al., 28 Feb 2026). These limitations suggest that a fully unified distortion-aware hybrid beamforming theory—simultaneously covering PA memory, quantization, mutual coupling, wideband frequency selectivity, architectural connectivity, and imperfect CSI—remains only partially developed.
A plausible overall conclusion is that distortion-aware hybrid beamforming is best understood not as a single algorithmic doctrine but as a modeling stance: the hybrid architecture determines the structure of the relevant distortion, and any credible beamforming or linearization method must therefore be architecture-aware, direction-aware, and operating-point-aware.