Heat kernel estimates for Markov processes in bounded sets with jump kernels decaying at the boundary

Abstract: In this paper, we study two types of purely discontinuous symmetric Markov processes $X$ in bounded smooth subsets of $\mathbb R^d$: conservative processes and processes killed either upon approaching the boundary of the set or by a killing potential $κ$. The jump kernel of $X$ is of the form $J(x,y)={\cal B}(x,y)|x-y|^{-d-α}$, $α\in (0,2)$, where the function ${\cal B}(x,y)$ decays to 0 at the boundary and is described in terms of two $O$-regularly varying functions and one slowly varying function. Under the conditions, introduced in \cite{CKSV24}, on ${\cal B}(x,y)$ and on the killing potential $κ$, we establish sharp two-sided estimates on the heat kernel of $X$: in Lipschitz sets when $X$ is conservative, and in $C^{1,1}$ open sets for the killed process.