Fat-shattering characterization of general convex optimization complexity

Determine whether the offline first-order oracle complexity of general convex Lipschitz optimization can always be upper bounded, up to logarithmic factors, by the fat-shattering dimension of the associated linear pairing class, and whether such a bound can be achieved by an optimization algorithm.

Background

The paper proves an upper bound in terms of sequential fat-shattering dimension and shows that, in several examples—including mixed-norm, Lipschitz, Sobolev, and orthonormal-basis settings—the sequential and ordinary fat-shattering dimensions differ by only logarithmic factors. This does not resolve the broader question for arbitrary feasible and subgradient sets or arbitrary fat-shattering profiles.

The unresolved issue concerns whether ordinary fat-shattering dimension, rather than the sequential variant, controls the offline first-order oracle complexity in general, and whether an optimization method can attain the resulting bound. The paper notes that a literal dimension-only formulation fails in low dimensions because known oracle complexities may include an additional logarithmic dependence on the accuracy.