Unconventional Superconductivity near a Nematic Instability in a Multi-Orbital system

Abstract: We analyze superconductivity in a multi-orbital fermionic system near the onset of a nematic order, using doped FeSe as an example. We associate the nematic order with spontaneous polarization between $d_{xz}$ and $d_{yz}$ orbitals. We derive the pairing interaction, mediated by soft nematic fluctuations, and show that it is attractive, and that its strength depends on the position on the Fermi surface. As the consequence, right at the nematic quantum-critical point (QCP), superconducting gap opens up at $T_c$ only at special points and extends into finite arcs at $T < T_c$. In between the arcs the Fermi surface remains intact. This gives rise to highly unconventional behavior of the specific heat, with no jump at $T_c$ and an apparent finite offset at $T=0$, when extrapolated from a finite $T$. We argue that this behavior is consistent with the specific heat data for FeSe${1-x}$S$_x$ near critical $x$ for the onset of a nematic order. We discuss the behavior of the gap away from a QCP and the pairing symmetry, and apply the results to FeSe${1-x}$S$x$ and FeSe${1-x}$Te$_x$, which both show superconducting behavior near the QCP distinct from that in a pure FeSe.