Measurements with a TRISTAN prototype detector system at the "Troitsk nu-mass" experiment in integral and differential mode

Abstract: Sterile neutrinos emerge in minimal extensions of the Standard Model which can solve a number of open questions in astroparticle physics. For example, sterile neutrinos in the keV-mass range are viable dark matter candidates. Their existence would lead to a kink-like distortion in the tritium $\beta$-decay spectrum. In this work we report about the instrumentation of the Troitsk nu-mass experiment with a 7-pixel TRISTAN prototype detector and measurements in both differential and integral mode. The combination of the two modes is a key requirement for a precise sterile neutrino search, as both methods are prone to largely different systematic uncertainties. Thanks to the excellent performance of the TRISTAN detector at high rates, a sterile neutrino search up to masses of about 6 keV could be performed, which enlarges the previous accessible mass range by a factor of 3. Upper limits on the neutrino mixing amplitude in the mass range < 5.6 keV (differential) and < 6.6 keV (integral) are presented. These results demonstrate the feasibility of a sterile neutrino search as planned in the upgrade of the KATRIN experiment with the final TRISTAN detector and read-out system.