Improved Hoeffding's Lemma and Hoeffding's Tail Bounds

Abstract: The purpose of this letter is to improve Hoeffding's lemma and consequently Hoeffding's tail bounds. The improvement pertains to left skewed zero mean random variables $X\in[a,b]$, where $a<0$ and $-a>b$. The proof of Hoeffding's improved lemma uses Taylor's expansion, the convexity of $\exp(sx), s\in {\bf R}$ and an unnoticed observation since Hoeffding's publication in 1963 that for $-a>b$ the maximum of the intermediate function $\tau(1-\tau)$ appearing in Hoeffding's proof is attained at an endpoint rather than at $\tau=0.5$ as in the case $b>-a$. Using Hoeffding's improved lemma we obtain one sided and two sided tail bounds for $P(S_n\ge t)$ and $P(|S_n|\ge t)$, respectively, where $S_n=\sum_{i=1}^nX_i$ and the $X_i\in[a_i,b_i],i=1,...,n$ are independent zero mean random variables (not necessarily identically distributed). It is interesting to note that we could also improve Hoeffding's two sided bound for all ${X_i: a_i\ne b_i,i=1,...,n}$. This is so because here the one sided bound should be increased by $P(-S_n\ge t)$, wherein the left skewed intervals become right skewed and vice versa.