Detectable signals of post-Born lensing curl B-modes

Abstract: Curl lensing, also known as lensing field-rotation or shear B-modes, is a distinct post-Born observable caused by two lensing deflections at different redshifts (lens-lens coupling). For the Cosmic Microwave Background (CMB), the field-rotation is approximately four orders of magnitude smaller than the CMB lensing convergence. Direct detection is therefore challenging for near-future CMB experiments such as the Simons Observatory (SO) or CMB `Stage-4' (CMB-S4). Instead, the curl can be probed in cross-correlation between a direct reconstruction and a template formed using pairs of large-scale structure (LSS) tracers to emulate the lens-lens coupling. In this paper, we derive a new estimator for the optimal curl template specifically adapted for curved-sky applications, and test it against non-Gaussian complications using N-body cosmology simulations. We find non-foreground biases to the curl cross-spectrum are purely Gaussian at the sensitivity of SO. However, higher-order curl contractions induce non-Gaussian bias at the order of $1\sigma$ for CMB-S4 using quadratic estimators (QE). Maximum a-Posteriori (MAP) lensing estimators significantly reduce biases for both SO and CMB-S4, in agreement with our analytic predictions. We also show that extragalactic foregrounds in the CMB can bias curl measurements at order of the signal, and evaluate a variety of mitigation strategies to control these biases for SO-like experiments. Near-future observations will be able to measure post-Born lensing curl B-modes.