Interference-Aware Multiuser Hybrid Precoding for Coexistence with LEO Satellite Communication

Abstract: Interference from terrestrial networks can reduce the communication rate for low Earth orbit (LEO) satellites in the upper mid-band. To coexist in frequency, MIMO precoding can be used to reduce the signal that impinges on the LEO satellite. We present a beamforming algorithm designed for the hybrid architecture that incorporates a satellite interference penalty while optimizing the analog and digital precoders. Our algorithm optimizes the precoding at the base station (BS) within the set of precoders that null the interference to the satellite. Simulations demonstrate that our algorithm reduces the interference at the satellites and lowers the probability of violating prescribed LEO satellite protection thresholds, outperforming prior hybrid nulling algorithms. Results indicate that the algorithm maintains sum-rate within 3\% of the existing hybrid solutions, while effectively improving interference to noise power by 22.4 dB.