Solutions and stability of a generalization of Wilson's equation

Abstract: In this paper we study the solutions and stability of the generalized Wilson's functional equation $\int_{G}f(xty)d\mu(t)+\int_{G}f(xt\sigma(y))d\mu(t)=2f(x)g(y),\; x,y\in G$, where $G$ is a locally compact group, $\sigma$ is a continuous involution of $G$ and $\mu$ is an idempotent complex measure with compact support and which is $\sigma$-invariant. We show that $\int_{G}g(xty)d\mu(t)+\int_{G}g(xt\sigma(y))d\mu(t)=2g(x)g(y),\; x,y\in G$ if $f\neq 0$ and $\int_{G}f(t.)d\mu(t)\neq 0$. We also study some stability theorems of that equation and we establish the stability on noncommutaive groups of the classical Wilson's functional equation $f(xy)+\chi(y)f(x\sigma(y))=2f(x)g(y)\; x,y\in G$, where $\chi$ is a unitary character of $G$.