Differentiation of the HTMC norm

Ascertain a principled method to differentiate the Harder than Monte Carlo (HTMC) norm ∥f∥_{H^γ}—defined via infinite sequences of binary circuits—so that gradient-based optimization methods can be applied to circuit-size minimization in the HTMC regime.

Background

Although the authors establish a sandwich bound relating the HTMC norm and a ResNet-based function norm, the HTMC norm itself is defined through discrete binary circuits and lacks an evident differential structure. This obstructs the use of gradient-based methods that would otherwise exploit the convexity properties established in the HTMC regime.