A Search for helium in the atmospheres of three sub-Neptunes and a super-Earth around M-dwarfs

Abstract: Thousands of sub-Neptunes have been discovered mainly through space-based surveys such as Kepler and TESS. Their bulk compositions and internal structures are thought to reflect their formation and evolutionary pathways, and atmospheric observations provide constraints on these processes. The near-infrared helium triplet is a potential tracer of extended, escaping H/He atmospheres. Recent models that include geometric effects suggest that planets orbiting nearby late M dwarfs may offer favorable conditions for detecting this signal. Nevertheless, helium has been reported for only three planets around M dwarfs to date. We conducted high-resolution transmission spectroscopy of three sub-Neptunes (TOI-2136b, TOI-654b, and LP 791-18c) and a super-Earth (TOI-1634b) orbiting M dwarfs with the InfraRed Doppler (IRD) spectrograph on the Subaru Telescope. We find no statistically significant helium absorption in any target; accordingly, we derive 95% confidence upper limits on the helium line depth of 1.36%, 0.60%, 2.07%, and 3.00%, and on the equivalent width of 7.3, 2.1, 7.4, and 9.1 mÅ, for TOI-2136b, TOI-1634b, TOI-654b, and LP 791-18c, respectively. We further explored constraints on the upper-atmospheric temperature and mass-loss rate by comparing these results with isothermal Parker-wind models. While we have compared with self-consistent ATES models of primordial H/He atmospheres spanning a range of assumed X-ray luminosities, changes in the assumed XUV flux do not appear to account for the non-detections. The results suggest that these planets have metal-enriched H/He primary atmospheres or non-primordial atmospheres, such as water-rich envelopes. Future observations of other absorption lines, such as Lyman-$α$, H-$α$, and H$_2$O, may provide further constraints on these atmospheres.

• The paper reports no statistically significant He I 10830 Å absorption in three sub-Neptunes and one super-Earth, establishing 95% upper limits of 0.62–2.53% in line depth.
• High-resolution Subaru/IRD transmission spectroscopy combined with Parker-wind and ATES modeling shows that predicted helium signals from primordial, near-solar H/He atmospheres exceed the observational limits, especially for LP 791-18c.
• The findings favor metal-enriched or water-rich atmospheres for the sub-Neptunes and indicate that XUV-flux uncertainty alone cannot explain the non-detections, motivating follow-up studies with JWST, Lyman-α, and Hα observations.

Motivation and target selection

The near-infrared He I triplet at 10830 Å is one of the few ground-accessible tracers of extended, escaping H/He atmospheres, and it is largely immune to interstellar absorption that complicates Lyman-α observations. Because the metastable population of the triplet depends on the balance between stellar XUV and mid-UV irradiation, early theoretical work predicted K-dwarf hosts to be optimal, and indeed most detections to date are around K stars. More recent models incorporating detailed outflow thermal structures, however, predict stronger He I signals for planets around M dwarfs [Biassoni+2024; Ballabio+2025]. Despite this, helium had been reported for only three planets orbiting M dwarfs, and only three M-dwarf-hosted sub-Neptunes (K2-25b, GJ 1214b, GJ 3090b) had been searched, most prior targets being super-Earths unlikely to retain H/He envelopes.

This paper presents high-resolution transmission spectroscopy with the InfraRed Doppler (IRD) spectrograph (R = 70,000, 970–1750 nm) on the Subaru 8.2 m telescope for four TESS-discovered planets: the sub-Neptunes TOI-2136b ($R_p = 2.20\,R_\oplus$, $T_{eq} = 378$ K), TOI-654b ($2.378\,R_\oplus$, 727 K), LP 791-18c ($2.438\,R_\oplus$, 324 K), and the super-Earth TOI-1634b ($1.749\,R_\oplus$, 842 K). Each system was observed for a single full transit with exposure times of 600–900 s, yielding per-pixel SNRs of roughly 89, 36, 21, and 63 at ~1000 nm respectively. The TOI-2136b data reanalyze the transit previously reported by Kawauchi et al. (2022), now with an improved telluric correction.

Data reduction

Telluric absorption was removed using theoretical telluric spectra built from HITRAN line parameters and the MIPAS atmospheric model, fitted frame-by-frame; telluric standard-star correction was rejected because the standard-star data lacked sufficient SNR. OH$^-$ emission near the triplet was corrected by fitting the strongest line at 1(0834.2873) Å with a Gaussian whose amplitude and width scale a three-line empirical template derived from CARMENES fiber-B sky spectra, motivated by the correlated variability of neighboring OH lines. Residuals after correction reach the noise level.

Wavelength shifts were corrected first with rvcorrect and then refined via least-squares deconvolution against VALD line lists over three spectral orders containing 426 lines, with shifts of −52.6, −64.3, −18.0, and −95.5 km/s for TOI-2136, TOI-1634, TOI-654, and LP 791-18 respectively; consistency with BT-Settl models validates the correction. Out-of-transit frames were combined into a template, divided from each frame, shifted into the planetary rest frame, and co-added. Planetary radial velocity excursions during transit range from ±1.5 km/s (TOI-2136b, LP 791-18c) up to +20.3/−15.8 km/s (TOI-1634b).

Transmission spectra and upper limits

No statistically significant He I absorption is detected in any target. Gaussian-profile fits with emcee (line center restricted between the two stronger triplet components; width uniform between 0.065 and 0.5 Å) yield the following 95% confidence upper limits:

Planet	Depth UL (%)	EW UL (mÅ)
TOI-2136b	< 1.54	< 7.4
TOI-1634b	< 0.62	< 2.7
TOI-654b	< 1.97	< 7.3
LP 791-18c	< 2.53	< 8.1

The TOI-2136b spectrum retains a marginal excess feature at ~10830.8 Å noted previously, but its width is narrower than the instrumental profile and it persists across multiple exposures, suggesting a stellar pseudo-signal; attempts to confirm a stellar-activity origin via Paschen-β failed because that line is contaminated by telluric OH emission, and the small planetary RV excursion (~0.06 Å) prevents the residual map from discriminating planetary from stellar origins. The authors also note that temporal variability of a telluric emission feature near 10833.3 Å (vacuum), detected near twilight, could imprint an apparent absorption signal if it contaminates the out-of-transit reference—an important caveat for any He I observation taken close to sunset or sunrise.

Phase-resolved equivalent widths of the stellar He I triplet show no orbital-phase variation in any system, ruling out coherent enhancements comparable to the giant tidal tails reported for HAT-P-32b and HAT-P-67b.

Model comparison: Parker wind and ATES

Isothermal Parker-wind models computed with p-winds, including tidal gravity terms (important for TOI-1634b and TOI-654b, where the Roche-lobe radius approaches the sonic point), an H/He ratio of 90/10, XUV scaled from the GJ 876 MUSCLES template, and the Hill radius as the effective upper boundary, exclude a region of low-temperature/high-mass-loss-rate parameter space at 95% confidence for all four planets. Phase integration used 100 bins, verified to introduce relative depth errors of only ~0.5–0.7% versus a 10,000-bin reference.

Self-consistent hydrodynamic models with ATES, assuming primordial solar-composition H/He atmospheres, predict post-instrumental-convolution line depths of 11.7%, 1.09%, 2.58%, and 24.0% for TOI-2136b, TOI-1634b, TOI-654b, and LP 791-18c respectively—all exceeding the observational upper limits, most dramatically for LP 791-18c, where the prediction exceeds the limit by nearly an order of magnitude. The corresponding ATES-predicted EWs (56.8, 6.68, 16.2, and 138.4 mÅ) are likewise ruled out at >95% confidence. This is the central quantitative tension of the paper: primordial, near-solar-composition H/He envelopes should have produced readily detectable helium absorption, yet none is seen.

Recomputing ATES models with the substantially lower coronal XUV luminosities of S2025 (e.g., $\log L_X = 26.20$ versus 27.47 erg/s for the GJ 876 template) actually increases the predicted depths (13.5%, 1.13%, 2.76%, 27.6%), so the discrepancy persists under both XUV prescriptions. A systematic scan of $\log(L_X/L_{bol})$ from −3.0 down to −6.5 (early M dwarfs) or −6.0 (LP 791-18), with EUV estimated via two alternative S2025 relations, finds no case in which both the predicted depth and EW fall below the detection thresholds. The authors conclude that XUV flux uncertainty alone cannot account for the non-detections—a strong claim, though they concede they cannot rule out X-ray luminosities below the observed M-dwarf range.

A methodological caveat emerges from comparing the two codes: best-fit isothermal temperatures systematically exceed metastable-helium-density-weighted mean temperatures from the non-isothermal ATES profiles, so isothermal Parker-wind inversions of detections may yield biased atmospheric parameters unless non-isothermal radiative transfer (e.g., sunbather) is employed.

Interpretation

For TOI-1634b, a rocky super-Earth with escape velocity ~26.9 km/s at ultra-short period, the non-detection is unsurprising and consistent with the absence of a retained H/He envelope, though a secondary atmosphere cannot be excluded. For the three low-density sub-Neptunes, the results favor either metal-enriched H/He primary atmospheres—which recent work suggests weaken the He I triplet—or non-primordial, water-rich envelopes. Notably, JWST transmission spectroscopy of LP 791-18c indicates an H$_2$-dominated but high-metallicity atmosphere, leaving open the possibility that this planet does retain a primordial envelope whose He I signal is suppressed by high metallicity, strong stellar winds, or vibrationally excited H$_2$. An alternative H/He ratio different from the assumed 90/10 could also mute the triplet.

Limitations and open questions

Several limitations qualify these conclusions. All constraints derive from single transits per target, with modest SNR for LP 791-18 (SNR ≈ 21); variability of escaping atmospheres, documented elsewhere, cannot be assessed. The 1D spherically symmetric models cannot capture 3D outflow structures such as cometary tails, and transmission spectra sampling limited orbital phase do not constrain such geometry. The X-ray luminosities of the host stars were never directly measured, and the analysis assumes the targets' X-ray emission lies within empirically observed ranges. Finally, the tension between isothermal and self-consistent temperature definitions means the excluded regions in mass-loss-rate–temperature space should be interpreted with care.

The specific questions left open are whether complementary tracers—Lyman-α, Hα, and molecular features such as H$T_{eq} = 378$0O observed with JWST—can independently establish atmospheric metallicity and H/He ratio for these planets, and whether mechanisms such as stellar-wind interaction or vibrationally excited H$T_{eq} = 378$1 can quantitatively reconcile LP 791-18c's JWST-inferred H$T_{eq} = 378$2 atmosphere with its absent He I signature.

Conclusion

This work reports non-detections of He I 10830 Å absorption in four small planets transiting M dwarfs, with 95% depth limits of 0.62–2.53%, and demonstrates through both isothermal Parker-wind and self-consistent ATES modeling—and across a wide grid of assumed XUV fluxes—that primordial near-solar H/He atmospheres are inconsistent with the data. The results point toward metal-enriched or non-primordial atmospheres for the sub-Neptunes, while underscoring that resolving the low He I detection rate around M dwarfs requires independent compositional constraints from other spectral diagnostics.