Search for exotic decays of the Higgs boson to a pair of pseudoscalars in the μμbb and ττbb final states

A search for exotic decays of the Higgs boson (H) with a mass of 125GeV to a pair of light pseudoscalars a ₁ is performed in final states where one pseudoscalar decays to two b quarks and the other to a pair of muons or τ leptons. A data sample of proton–proton collisions at s=13TeV corresponding to an integrated luminosity of 138fb ^-1 recorded with the CMS detector is analyzed. No statistically significant excess is observed over the standard model backgrounds. Upper limits are set at 95% confidence level (CL) on the Higgs boson branching fraction to μμbb and to ττbb, via a pair of a ₁s. The limits depend on the pseudoscalar mass ma ₁ and are observed to be in the range (0.17–3.3) ×10 ^-4 and (1.7–7.7) ×10 ^-2 in the μμbb and ττbb final states, respectively. In the framework of models with two Higgs doublets and a complex scalar singlet (2HDM+S), the results of the two final states are combined to determine upper limits on the branching fraction B(H→a ₁a ₁→ℓℓbb) at 95% CL, with ℓ being a muon or a τ lepton. For different types of 2HDM+S, upper bounds on the branching fraction B(H→a ₁a ₁) are extracted from the combination of the two channels. In most of the Type II 2HDM+S parameter space, B(H→a ₁a ₁) values above 0.23 are excluded at 95% CL for ma ₁ values between 15 and 60GeV.