---
title: Distortion-Aware Hybrid Beamforming
url: https://www.emergentmind.com/topics/distortion-aware-hybrid-beamforming
type: topic
---

# Distortion-Aware Hybrid Beamforming

Distortion-aware hybrid beamforming denotes a class of hybrid analog/digital beamforming methods in which nonideal array, RF-front-end, or propagation effects are embedded in the beamforming model, optimization criterion, or linearization target instead of being treated as architecture-independent white noise. In the mmWave MIMO literature, the dominant interpretation is power-amplifier-aware hybrid beamforming, where the precoder shapes not only the desired signal but also the spatial covariance of nonlinear inband distortion [1806.01602]. Closely related work extends the idea to subarray-level digital predistortion in hybrid transmitters [1804.02178], architecture-aware spatio-frequency Bussgang models for fully connected and partially connected arrays [2006.15930], integrated sensing and communication (ISAC) under nonlinear amplification [2507.14018], coarse or quantized analog hardware [1803.02741, 2501.01684], electromagnetic radiation-pattern distortion and mutual coupling [2403.05970, 2502.17864], and mobility-induced Doppler/ICI effects in active/passive beamforming for RIS-aided vehicular links [2603.00407]. The common principle is that the analog network, connectivity, and hardware operating point alter the effective channel, distortion field, or both, so beamforming and impairment modeling cannot be cleanly separated.

## 1. Scope and meanings of “distortion-aware”

The literature uses the term across several non-identical distortion mechanisms. In all cases, the beamformer is designed or analyzed against a distortion source that is structured by the hybrid architecture rather than appended afterward as a scalar penalty.

| Distortion notion | Representative model | Representative paper |
|---|---|---|
| PA-induced inband distortion | Distortion covariance as a function of pre-PA beamformed covariance | [1806.01602] |
| Beamformed aggregate distortion for DPD | One DPD per subarray learned from combined PA observation | [1804.02178] |
| Effective-channel / beam-pattern distortion | Post-analog channel \(\mathbf H^{\mathrm E}=\mathbf H\mathbf A\) with one-bit phase shifters | [1803.02741] |
| Quantization-aware hybrid precoding | DAC attenuation \(\Delta\) and discrete phase-shifter codebooks | [2501.01684] |
| Electromagnetic pattern distortion | Coupling matrix \(\mathbf C\), radiation patterns, and current-vector analog control | [2403.05970] |
| Mobility-induced distortion | Doppler-driven ICI and training-overhead/channel-aging tradeoff | [2603.00407] |

In standard mmWave transmit architectures, hybrid precoding is typically expressed as a digital/baseband stage followed by an analog/RF network and then per-antenna PAs. A decisive consequence is that the PA inputs are already spatially correlated before amplification. In partially connected ISAC arrays, the analog matrix is block diagonal, whereas in fully connected architectures each PA input is a mixture of multiple RF-chain signals; these distinctions directly change distortion structure [1806.01602, 2507.14018, 2006.15930].

A broader but adjacent line of work treats mismatch-aware or robustness-aware receive HAD beamforming, especially under DOA mismatch and phase-only combining constraints. That literature is relevant to distortion-aware hybrid beamforming only indirectly, because its “distortion” is primarily steering-vector/model mismatch rather than RF-front-end impairment [1811.01176].

## 2. Nonlinear PA distortion as a beamformed field

In the PA-dominant formulation for single-user mmWave hybrid MIMO, the beamformed signal before the PA bank is
\[
\mathbf u=\mathbf F_{\mathrm{RF}}\mathbf F_{\mathrm{BB}}\mathbf s,\qquad
\mathbf C_{\mathbf u}=\mathbf F_{\mathrm{RF}}\mathbf F_{\mathrm{BB}}\mathbf F_{\mathrm{BB}}^H\mathbf F_{\mathrm{RF}}^H,
\]
with \(\mathbf s\sim\mathcal{CN}(\mathbf 0,\mathbf I)\). Each PA is modeled as a memoryless odd-order polynomial,
\[
x_n=f(u_n)=\sum_{m=0}^{M}\beta_{2m+1}|u_n|^{2m}u_n,
\]
and the transmit vector is decomposed as
\[
\mathbf x=\overline{\mathbf G}\mathbf u+\mathbf d,
\]
where \(\overline{\mathbf G}\) is the average linear gain and \(\mathbf d\) is zero-mean distortion uncorrelated with \(\mathbf u\). The central result is that the distortion covariance \(\mathbf C_{\mathbf d}\) is a deterministic function of \(\mathbf C_{\mathbf u}\), built from Hadamard powers of \(\mathbf C_{\mathbf u}\) and per-antenna gain factors. Consequently, the distortion is not spatially white and not independent across antennas in general; it is partially beamformed by \(\mathbf F_{\mathrm{RF}}\) and \(\mathbf F_{\mathrm{BB}}\) [1806.01602].

This changes the usual large-array intuition. Narrow desired beams do not imply that nonlinear distortion becomes negligible. The same spatial correlation that sharpens the useful beam can also shape inband distortion toward the intended receiver. In the received model
\[
\mathbf y=\mathbf H\overline{\mathbf G}\mathbf u+\mathbf H\mathbf d+\mathbf n,
\]
the effective noise-plus-interference covariance is
\[
\mathbf R_\eta=\mathbf H\mathbf C_{\mathbf d}\mathbf H^H+\sigma_n^2\mathbf I,
\]
so spectral efficiency depends on a colored, beamformer-dependent distortion term rather than a scalar noise floor [1806.01602].

The ISAC extension adopts a third-order Bussgang model. With overall beamformer \(\mathbf F=\mathbf F_{\mathrm A}\mathbf F_{\mathrm D}\), the nonlinear PA output is
\[
\phi(\mathbf x)=\mathbf B\mathbf x+\mathbf e,
\]
with
\[
\mathbf B=\beta_1\mathbf I_{N_t}+2\beta_3\mathbf F\mathbf F^H\odot\mathbf I_{N_t},\qquad
\mathbf C_{\mathbf e}=2|\beta_3|^2\mathbf F\mathbf F^H\odot|\mathbf F\mathbf F^H|^2.
\]
Thus both the effective linear gain and the additive distortion covariance depend on \(\mathbf F\). User-\(k\) SINR includes \(\mathbf h_k^H\mathbf C_{\mathbf e}\mathbf h_k\), and sensing MI includes \(|\alpha_s|^2\mathbf a_s^H\mathbf C_{\mathbf e}\mathbf a_s\), so the precoder shapes distortion in both communication and sensing directions [2507.14018].

A plausible implication is that “distortion-aware” hybrid beamforming is fundamentally covariance-aware: once the hybrid precoder determines per-antenna powers and cross-antenna correlations, it also determines where nonlinear distortion is radiated and how strongly it enters the relevant directions.

## 3. Linearization and beam-domain mitigation

In hybrid subarray transmitters, one digital stream drives a whole analog-beamformed subarray. Because only one DPD can be placed before the analog network of that subarray, perfect per-PA linearization is generally impossible when the PAs differ. The subarray-level solution is to learn a single DPD from the combined observation of all PA outputs after anti-beamforming or co-phasing, so that the feedback waveform emulates the signal that adds coherently in the intended beam direction. For branch weights \(w_m\) and PA outputs \(y_m(n)\), the effective main-beam signal is
\[
y(n)=\sum_{m=1}^{M} w_m^* y_m(n),
\]
and the observation signal can be formed as
\[
z(n)=\sum_{m=1}^{M} w_m^* g_c y_m(n)=Gx(n)+d(n).
\]
DPD learning then minimizes the correlation between the residual error \(e(n)=z(n)-\hat Gx(n)\) and orthogonalized nonlinear basis functions, using the block-adaptive decorrelation update
\[
\bar{\boldsymbol{\alpha}}(i+1)=\bar{\boldsymbol{\alpha}}(i)-\mu [e(i)^H S(i)]^T.
\]
This directly targets the beamformed distortion seen in the intended direction [1804.02178].

That viewpoint is architecture-specific rather than per-device-specific. The linearization target is the aggregate radiated distortion in the main beam, not the individual distortion of each PA. Under realistic measured PA models from the Lund University massive MIMO testbed, the combined-feedback DPD reduced main-beam EVM from \(3.17\%\) without DPD to \(1.85\%\), and improved ACLR L/R from \(40.48/40.58\) dBc to \(63.63/61.42\) dBc. The reference method that learned from one PA output only achieved \(2.09\%\) EVM and \(52.01/51.91\) dBc ACLR L/R, so the combined-feedback design delivered more than \(10\) dB ACLR gain relative to that single-PA-learning baseline [1804.02178].

A second architecture-aware linearization framework models PA distortion via spatio-frequency Bussgang decomposition for hybrid OFDM arrays. In fully connected architectures, the frequency-domain relation is
\[
\mathbf y_k^f=\mathbf A_k\mathbf x_k^f+\boldsymbol\eta_k,
\]
where \(\mathbf A_k\in\mathbb C^{N_t\times D}\) is a matrix-valued nonlinearized effective beamformer and \(\mathbf R_{\boldsymbol\eta}^k\) is a full spatial covariance. In partially connected arrays, the decomposition becomes subarray-wise and the distortion covariance is block diagonal. Anti-beamforming feedback is then used to reduce the \(N_t\)-dimensional PA output to \(D\) observed streams, enabling one DPD per RF chain. The analysis shows that classical per-RF-chain DPD is effective for partially connected arrays and best for fully digital arrays, but inadequate for fully connected arrays because each PA input is a superposition of all RF-chain signals and the nonlinear cross-terms cannot be inverted by branchwise DPD alone [2006.15930].

A recurring conclusion across these linearization papers is that hybrid beamforming changes the correct observation model. Distortion suppression is most effective when feedback reconstructs the spatially combined signal that the array actually radiates in the dominant direction.

## 4. Architecture-dependent distortion beyond PA nonlinearity

Not all distortion-aware hybrid beamforming papers treat distortion as additive nonlinear RF noise. In wireless fronthaul with one-bit analog phase shifters, the central distortion mechanism is the effective or distorted channel after analog precoding:
\[
\mathbf H_l^{\mathrm E}=\mathbf H_l\mathbf A.
\]
The digital precoder is computed from an SLNR generalized eigenproblem using only \(\mathbf H_l^{\mathrm E}\), not the inaccessible true \(\mathbf H_l\), and a genetic algorithm searches the discrete one-bit analog precoder space. This formulation is distortion-aware in the sense of being phase-quantization-aware and effective-channel-aware. Its main practical warning is that hybrid SLNR with one-bit phase shifters can create strong undesired beams and hence intercell interference in multicell wireless fronthaul, even when single-cell sum rate remains close to digital beamforming [1803.02741].

A different hardware-aware formulation models DAC quantization and finite-resolution phase shifters in single-user narrowband mmWave. The analog precoder is factorized as
\[
\mathbf F_{\mathrm{RF}}=\mathbf S_2\mathbf F_{\mathrm{ps}}\mathbf S_1,
\]
and DAC quantization is introduced through
\[
Q(\mathbf F_{\mathrm{BB}}x)=\Delta \mathbf F_{\mathrm{BB}}x+\varepsilon_b,
\]
with diagonal attenuation \(\Delta_{i,i}=1-\frac{\pi\sqrt{3}}{2}2^{-2b_i}\). Phase shifters are restricted to a discrete alphabet
\[
\Phi\in\left\{0,\frac{2\pi}{2^q},\ldots,\frac{2\pi(2^q-1)}{2^q}\right\}.
\]
The resulting energy-efficiency optimization jointly tunes digital precoding, switch connectivity, analog phases, and DAC resolution. The paper explicitly introduces additive quantization noise but then neglects it in later derivations because its covariance is argued to be very small for the DAC resolutions of interest; accordingly, its strongest distortion-aware content is the joint modeling of multiplicative DAC attenuation and discrete phase constraints, rather than full quantization-noise covariance propagation [2501.01684].

Electromagnetic hybrid beamforming shifts the locus of distortion from RF blocks to radiation physics. For a 3D holographic array, the emitted field is
\[
\mathbf x^{(c)}=\mathbf C\,\mathbf i\odot \mathbf g(\mathbf r,\theta,\phi)\odot \mathbf W_{BB}\mathbf t,
\]
where \(\mathbf C\) models mutual coupling and \(\mathbf i\) is the analog current vector. Coupling-aware directivity maximization yields
\[
\mathbf i^{(c)}_{\mathrm{opt}}=\gamma \mathbf C^{-1}\mathbf Z^{-1}\mathbf e(\theta,\phi).
\]
Here the distortion-aware step is not linearization of a nonlinear PA but compensation of pattern distortion induced by sector non-uniformity and coupling. The effective electromagnetic channel already contains the array pattern and coupling effects, so digital precoding is optimized on that channel rather than on an RF-only abstraction [2403.05970].

Hybrid reconfigurable parasitic arrays move further toward a circuit-theoretic interpretation. With one active antenna and multiple parasitic elements, the induced parasitic currents satisfy
\[
\mathbf i_P=-(\mathbf Z_{\mathsf P}+\mathbf Z_{\mathsf R})^{-1}\mathbf z_{0P}i_0,
\]
and the beam pattern is
\[
G(\theta,\mathbf Z_{\mathsf R})=\left|1-\mathbf a_P^T(\theta)(\mathbf Z_{\mathsf P}+\mathbf Z_{\mathsf R})^{-1}\mathbf z_{0P}\right|^2.
\]
The analog “weights” are therefore not free complex coefficients but functions of mutual impedances and realizable reactances. A shift-of-origin transformation maps the resulting coupled magnitude-phase constraint into a unit-modulus variable, enabling a closed-form reactance design. This suggests that, in parasitic hybrid arrays, distortion-awareness is essentially realizability-awareness [2502.17864].

## 5. Optimization objectives and algorithmic frameworks

Once the distortion source is specified, the optimization criterion changes materially. In distortion-aware ISAC with partially connected hybrid beamforming, the objective is the weighted sum
\[
\mathcal G(\mathbf F_{\mathrm A},\mathbf F_{\mathrm D})
=\varpi_c\sum_{k=1}^{K}R_k+\varpi_s R_s,
\]
subject to a power constraint imposed after nonlinear amplification:
\[
\mathbb E\left\{\|\phi(\mathbf F_{\mathrm A}\mathbf F_{\mathrm D}\mathbf s)\|^2\right\}=P_{tot}.
\]
The solution strategy first optimizes a fully digital beamformer \(\mathbf F\) via alternating optimization over \(\mathbf F\), \(\mathbf U\), and \(\mathbf V\); the \(\mathbf F\)-subproblem is solved on a constant-Frobenius-norm manifold with Riemannian conjugate-gradient updates and Armijo line search, while \(\mathbf U\) and \(\mathbf V\) admit closed-form updates. The optimized \(\mathbf F\) is then decomposed into block-diagonal unit-modulus \(\mathbf F_{\mathrm A}\) and digital \(\mathbf F_{\mathrm D}\) through a hybrid decomposition stage [2507.14018].

In effective-channel-aware fronthaul, the analog search variable is discrete and the digital stage is simple. For fixed one-bit analog precoder \(\mathbf A\), the digital beamformer is the dominant generalized eigenvector of
\[
\left(M_l\sigma^2\mathbf I+(\tilde{\mathbf H}_l^{\mathrm E})^H\tilde{\mathbf H}_l^{\mathrm E}\right)^{-1}
(\mathbf H_l^{\mathrm E})^H\mathbf H_l^{\mathrm E},
\]
and a genetic algorithm evolves the analog precoder population to maximize a low-SNR approximation of sum rate,
\[
f(\mathbf A)\approx \sum_{k=1}^{K}\log_2(1+\lambda_{\max}^{(k)}).
\]
The architecture is thus distortion-aware through the measured post-analog channel rather than through explicit RF impairment penalties [1803.02741].

Electromagnetic hybrid beamforming for 3D holographic arrays uses alternating optimization with semidefinite relaxation. The single-user closed-form updates
\[
\mathbf i_{\mathrm{opt}}=\alpha\,\operatorname{diag}(\mathbf G\mathbf s)^*\mathbf C^H\mathbf w_{BB}^*,\qquad
\mathbf w_{BB,\mathrm{opt}}=\beta\,\mathbf C^*\operatorname{diag}(\mathbf G\mathbf s)^*\mathbf i^*
\]
lead to multi-user formulations lifted into semidefinite variables and solved alternately. The analog stage is initialized from the coupling-aware directivity solution \(\mathbf i=\mu_i\mathbf C^{-1}\mathbf Z^{-1}\mathbf e\), making environmental and electromagnetic adaptation explicit [2403.05970].

In RIS-aided vehicular communication, the dominant impairment is mobility. The system first performs two-timescale grouped channel estimation, then optimizes active BS beamforming and grouped RIS coefficients. In the single-VUE narrowband case, active beamforming uses water-filling over the grouped equivalent channel, while grouped RIS phases are updated either in closed form for single-element groups or by gradient ascent for larger groups. In the broadband multi-VUE case, subcarrier assignment, power allocation, active beamforming, and passive RIS beamforming are jointly optimized under explicit Doppler-ICI terms, relaxed binary assignment, and difference-of-concave approximation. The beamforming problem is therefore distortion-aware because the resource-allocation variables and beamformers are optimized against ICI that depends on velocity and subcarrier spacing [2603.00407].

A plausible synthesis is that distortion-aware hybrid beamforming has no single canonical solver. Its algorithmic structure is inherited from the distortion mechanism: manifold optimization for nonlinear post-PA objectives, decorrelation learning for combined-radiation DPD, discrete search for quantized analog networks, SDR for coupling-aware electromagnetic synthesis, and two-timescale or D.C.-programming tools for Doppler-aware active/passive designs.

## 6. Empirical regularities, misconceptions, and scope limits

Several empirical patterns recur across the literature. First, larger arrays and narrower beams do not eliminate nonlinear distortion. In mmWave hybrid MIMO with nonlinear PAs, increasing transmit power as the number of transmit antennas grows large can be counter-effective in terms of energy efficiency, and at high transmit power analog beamforming can achieve higher spectral and energy efficiency than digital and hybrid beamforming because the architecture changes both PA operating conditions and total power consumption [1806.01602]. Second, linearization targeted at the beamformed aggregate signal can markedly outperform per-device surrogates: the combined-feedback subarray DPD reached \(1.85\%\) EVM and \(63.63/61.42\) dBc ACLR L/R, versus \(2.09\%\) and \(52.01/51.91\) dBc for single-PA-learning DPD [1804.02178]. Third, in distortion-aware ISAC, reducing nonlinear radiation toward important directions can be more valuable than maximizing linear beam power alone: the reported design produced about \(42\) dB reduction in nonlinear power toward the user direction while having about \(7\) dB weaker linear power than MRT because of the nonzero sensing weight [2507.14018].

The electromagnetic and hardware-aware literature reveals analogous tradeoffs. The 3D holographic EHB scheme was reported to achieve only around \(10\%\) average gain in single-user transmission at \(20\) dB SNR, but in multi-user settings it achieved an average sum-rate gain of over \(150\%\) relative to conventional beamforming algorithms because pattern programmability and coupling-aware design significantly reduced inter-user interference [2403.05970]. The dual-switch mmWave architecture with DAC-bit optimization reported spectral-efficiency loss below \(20\%\) relative to the fully connected architecture while improving energy efficiency; under imperfect CSI with \(\xi=0.6\) at \(20\) dB SNR, its energy efficiency was about \(3.4\) bits/Hz/J versus about \(3.7\) bits/Hz/J with perfect CSI, which the paper interpreted as around \(8\%\) loss [2501.01684]. In effective-channel-aware fronthaul, hybrid SLNR with one-bit or two-bit phase shifters was approximately \(15\%\) to \(7\%\) worse than digital SLNR from \(-12\) dB to \(0\) dB, with the gap reducing to about \(5\%\) in the moderate-SNR regime, yet the same coarse analog network could create undesired beams that matter in multicell operation [1803.02741].

A common misconception is that distortion-aware hybrid beamforming refers only to PA nonlinearity. The surveyed work shows otherwise: in some papers it denotes post-analog effective-channel distortion and beam-pattern distortion from finite phase resolution [1803.02741]; in others it denotes radiation-pattern distortion and mutual coupling [2403.05970], realizability constraints in parasitic arrays [2502.17864], or Doppler-induced ICI and training-overhead/channel-aging tradeoffs [2603.00407]. A second misconception is that more degrees of freedom necessarily improve distortion-limited performance. Coarse evidence runs in both directions: more RF chains or more flexible digital excitation can worsen PA operating conditions at high power [1806.01602], more subcarriers can worsen OFDM performance in high mobility because they reduce \(\Delta f\) and intensify ICI [2603.00407], and more parasitic elements do not necessarily keep improving spectral efficiency beyond \(N_P=2\) in the reported examples [2502.17864].

The scope limits are equally consistent. Much of the PA-nonlinearity literature is narrowband or single-carrier, assumes Gaussian signaling, and focuses on identical memoryless polynomial or PH PA models [1806.01602, 1804.02178, 2507.14018]. Some optimization stages are indirect: the ISAC design optimizes a fully digital target before hybrid decomposition [2507.14018], and the energy-efficient mmWave architecture introduces additive DAC quantization noise but later neglects it and approximates \(\Delta\approx\mathbf I\) in the beamformer block [2501.01684]. Electromagnetic formulations abstract several implementation details into real-time programmability of currents or circuit parameters [2403.05970], while the vehicular RIS work is “hybrid” in the active/passive sense and not in the conventional RF-chain/phase-shifter sense [2603.00407]. These limitations suggest that a fully unified distortion-aware hybrid beamforming theory—simultaneously covering PA memory, quantization, mutual coupling, wideband frequency selectivity, architectural connectivity, and imperfect CSI—remains only partially developed.

A plausible overall conclusion is that distortion-aware hybrid beamforming is best understood not as a single algorithmic doctrine but as a modeling stance: the hybrid architecture determines the structure of the relevant distortion, and any credible beamforming or linearization method must therefore be architecture-aware, direction-aware, and operating-point-aware.

Source: https://www.emergentmind.com/topics/distortion-aware-hybrid-beamforming