Estimation Error: Distribution and Pointwise Limits

Abstract: In this paper, we examine the distribution and convergence properties of the estimation error $W = X - \hat{X}(Y)$, where $\hat{X}(Y)$ is the Bayesian estimator of a random variable $X$ from a noisy observation $Y = X +\sigma Z$ where $\sigma$ is the parameter indicating the strength of noise $Z$. Using the conditional expectation framework (that is, $\hat{X}(Y)$ is the conditional mean), we define the normalized error $\mathcal{E}\sigma = \frac{W}{\sigma}$ and explore its properties. Specifically, in the first part of the paper, we characterize the probability density function of $W$ and $\mathcal{E}\sigma$. Along the way, we also find conditions for the existence of the inverse functions for the conditional expectations. In the second part, we study pointwise (i.e., almost sure) convergence of $\mathcal{E}\sigma$ under various assumptions about the noise and the underlying distributions. Our results extend some of the previous limits of $\mathcal{E}\sigma$ studied under the $L^2$ convergence, known as the \emph{mmse dimension}, to the pointwise case.