Search for Higgs bosons decaying to $aa$ in the $μμττ$ final state in $pp$ collisions at $\sqrt{s} = $ 8 TeV with the ATLAS experiment

Abstract: A search for the decay to a pair of new particles of either the 125 GeV Higgs boson ($h$) or a second CP-even Higgs boson ($H$) is presented. The dataset correspods to an integrated luminosity of 20.3 fb$^{-1}$ of $pp$ collisions at $\sqrt{s}=$ 8 TeV recorded by the ATLAS experiment at the LHC in 2012. The search was done in the context of the next-to-minimal supersymmetric standard model, in which the new particles are the lightest neutral pseudoscalar Higgs bosons ($a$). One of the two $a$ bosons is required to decay to two muons while the other is required to decay to two $\tau$-leptons. No significant excess is observed above the expected backgrounds in the dimuon invariant mass range from 3.7 GeV to 50 GeV. Upper limits are placed on the production of $h\rightarrow{aa}$ relative to the Standard Model $gg\rightarrow{h}$ production, assuming no coupling of the $a$ boson to quarks. The most stringent limit is placed at 3.5\% for $m_a =$ 3.75 GeV. Upper limits are also placed on the production cross section of $H\rightarrow{aa}$ from 2.33 pb to 0.72 pb, for fixed $m_a$ = 5 GeV with $m_H$ ranging from 100 GeV to 500 GeV.