Trace functions and fusion rules of diagonal coset vertex operator algebras

Abstract: In this paper, irreducible modules of the diagonal coset vertex operator algebra $C(L_{\mathfrak{g}}(k+l,0),L_{\mathfrak{g}}(k,0)\otimes L_{\mathfrak{g}}(l,0))$ are classified under the assumption that $C(L_{\mathfrak{g}}(k+l,0),L_{\mathfrak{g}}(k,0)\otimes L_{\mathfrak{g}}(l,0))$ is rational, $C_2$-cofinite and certain additional assumption. An explicit modular transformation formula of traces functions of $C(L_{\mathfrak{g}}(k+l,0),L_{\mathfrak{g}}(k,0)\otimes L_{\mathfrak{g}}(l,0))$ is obtained. As an application, the fusion rules of $C(L_{E_8}(k+2,0), L_{E_8}(k,0)\otimes L_{E_8}(2,0))$ are determined by using the Verlinde formula.