Optimal Algorithms for Nonlinear Estimation with Convex Models

Abstract: A linear functional of an object from a convex symmetric set can be optimally estimated, in a worst-case sense, by a linear functional of observations made on the object. This well-known fact is extended here to a nonlinear setting: other simple functionals of the object can be optimally estimated by functionals of the observations that share a similar simple structure. This is established for the maximum of several linear functionals and even for the $\ell$th largest among them. Proving the latter requires an unusual refinement of the analytical Hahn--Banach theorem. The existence results are accompanied by practical recipes relying on convex optimization to construct the desired functionals, thereby justifying the term of estimation algorithms.