Boundary operator expansion and extraordinary phase transition in the tricritical O(N) model

Abstract: We study the boundary extraordinary transition of a three-dimensional (3D) tricritical $O(N)$ model. We first compute the mean-field Green's function with a general coupling of $|\vec \phi|<sup>{2n}$ (with $n=3$ corresponding to the tricritical model) at the extraordinary phase transition. Then, using layer susceptibility, we obtain the boundary operator expansion for the transverse and longitudinal modes within the $\epsilon=3 - d$ expansion. Based on these results, we demonstrate that the tricritical point exhibits an extraordinary transition characterized by an ordered boundary for any $N$. This provides the first nontrivial example of continuous symmetry breaking in 2D in the context of boundary criticality.