Forward-mode automatic differentiation for the tensor renormalization group and its relation to the impurity method

Abstract: We propose a forward-mode automatic differentiation (AD) framework for tensor renormalization group (TRG) methods. In this approach, evaluating the derivatives of the partition function up to order $k$ increases the matrix-multiplication cost by a factor of $(k+1)(k+2)/2$ compared to computing the free energy alone, while the memory footprint is only $k$ times that of the original calculation. In the limit where the derivatives of the SVD are neglected, we establish a theoretical correspondence between our forward-mode AD and conventional impurity methods. Numerically, we find that the proposed AD algorithm can calculate internal energy and specific heat significantly higher accuracy than the impurity method at comparable computational cost. We also provide a practical procedure to extract critical exponents from derivatives of the renormalized tensor in TRG calculations in both two and three dimensions.