Nonlinear interaction effects in a three-mode cavity optomechanical system

Abstract: We investigate the resonant enhancement of nonlinear interactions in a three-mode cavity optomechanical system with two mechanical oscillators. By using the Keldysh Green's function technique we find that nonlinear effects on the cavity density of states can be greatly enhanced by the resonant scattering of two phononic polaritons, due to their small effective dissipation. In the large detuning limit and taking into account an upper bound on the achievable dressed coupling, the optimal point for probing the nonlinear effect is obtained, showing that such three-mode system can exhibit prominent nonlinear features also for relatively small values of $g/\kappa$.