Optimal Threshold Functions

• The paper demonstrates that any n-variable threshold function can be ε-approximated using a function dependent on at most Inf(f)^2 · poly(1/ε) variables, exponentially sharpening Friedgut’s theorem.
• It introduces a novel method leveraging anti-concentration inequalities and precise rounding techniques to reduce integer weight bounds from 2^(O(1/ε^2)) to 2^(O(1/ε^(2/3))).
• These advancements have practical implications in learning theory, property testing, and circuit complexity by enabling more efficient algorithms and tighter complexity bounds.

Optimal threshold functions are a central concept in the analysis and approximation of Boolean and real-valued threshold functions, with foundational importance in learning theory, circuit complexity, and the geometry of Boolean functions. For linear threshold functions (LTFs) $f(x) = \sign(w \cdot x - \theta)$, the notion of optimality often refers to maximizing approximation quality under resource constraints such as dependence on few variables (“junta size”) or integer weight bounds; it also concerns the structural connection between total influence, sensitivity, and approximability.

1. Approximation by Low-Variable Threshold Functions: Junta Structure

A principal result is that any $n$-variable threshold function $f$ is $\epsilon$-close (in Hamming distance) to a threshold function depending only on $\Inf(f)^2 \cdot \mathrm{poly}(1/\epsilon)$ variables, where $\Inf(f)$ denotes the total influence or average sensitivity of $f$:

$f(x) = \sign(w \cdot x - \theta),$

can be $\epsilon$-approximated by a function $g(x)$ that depends on at most $\Inf(f)^2 \cdot \mathrm{poly}(1/\epsilon)$ coordinates.

This is an exponential sharpening of Friedgut’s theorem, which, for general Boolean functions, yields only a $2^{O(\Inf(f)/\epsilon)}$ junta-size bound. The result is essentially tight up to lower-order terms: any such function may require

$\Omega\left(\Inf(f)^2 + \frac{1}{\epsilon^2}\right)$

variables for $\epsilon$-approximation.

This exponential improvement exploits the additive structure and anti-concentration properties specific to threshold (halfspace) functions, rather than general Boolean functions.

2. Integer-Weight Approximators: Anti-Concentration and Weight Bounds

The second principal result is that every $n$-variable threshold function is $\epsilon$-approximated by a threshold function with integer weights bounded in magnitude by

$\mathrm{poly}(n) \cdot 2^{O(1/\epsilon^{2/3})}$

previously, only bounds of $\mathrm{poly}(n) \cdot 2^{O(1/\epsilon^{2})}$ were achieved. The improvement is accomplished via a new proof technique that leverages strong anti-concentration inequalities, specifically Halász's result, to tightly control the probability that the weighted sum $w \cdot x$ is close to the threshold value.

For a precise construction, rounding techniques are applied: Each weight $w_i$ is rounded to the nearest integer multiple of a carefully selected granularity, for instance,

$\frac{r}{V_n \log(1/\epsilon)}$

for uniform distributions; $r/n$ for constant-biased products. The analysis demonstrates that the error incurred through rounding weights is controlled by the anti-concentration of $w \cdot x$.

This two-stage regularization—first reducing to “regular” (i.e., well-spread) weights via structural representation (see Lemma 26), then rounding—enables polynomially bounded approximators with integer weights.

3. Proof Techniques: Anti-Concentration and Representation Theorems

The novel methodological feature is the combination of probabilistic and Fourier-analytic techniques with anti-concentration tools. Specifically, Halász's inequality shows that, if weight differences $|w_i - w_j|$ are lower-bounded, the probability that the sum $w \cdot x$ falls into small intervals is $O(k^{-3/2})$ with $k$ the number of nonzero weights. This enables fine-grained control of the error rates for approximators produced by rounding.

A crucial step is establishing that every threshold function admits “almost optimal” representations in which consecutive weights differ by a substantial amount, thereby allowing anti-concentration bounds to be evoked directly.

The approach treats “regular” threshold functions first (no dominating weight) and extents to arbitrary LTFs by constructing regular approximants.

4. Quantitative Table: Junta and Integer Weight Bounds

Approximation	Upper Bound Size	Lower Bound Size
$\epsilon$-junta for LTF	$\Inf(f)^2 \cdot \mathrm{poly}(1/\epsilon)$	$\Omega(\Inf(f)^2 + 1/\epsilon^2)$
Integer-weight LTF ($\epsilon$-close)	$\mathrm{poly}(n) \cdot 2^{O(1/\epsilon^{2/3})}$	(No explicit lower bound given)

5. Applications and Significance

• Learning Theory: These results yield efficient algorithms for learning halfspaces. Smaller junta and weight representations enable faster sample and time complexity in both proper and improper learning models.
• Property Testing: The ability to approximate threshold functions with few variables or small integer weights yields stronger testers for function classes under uniform or product distributions.
• Circuit Complexity and Derandomization: Threshold circuits benefit from low-weight representations, which translate directly to improved circuit size upper bounds, circuit compression, and explicit constructions of pseudo-random generators using $k$-wise independence.
• Hardness of Approximation: Sharp characterizations of Fourier spectrum and influence for threshold functions underpin hardness proofs for various approximation problems.

6. Open Problems and Future Directions

Several questions arise:

• Weight Bounds: Is it possible to further reduce the integer weight dependence to

$$\mathrm{poly}(n) \cdot 2^{\mathrm{polylog}(1/\epsilon)}$$

for all LTFs?

• PTFs (Polynomial Threshold Functions): Extending the exponential sharpening from degree-1 threshold functions to degree-$d$ PTFs remains open, with the conjecture that junta size should be exponential (in $d$) in average sensitivity.
• Sharper Anti-Concentration: Leveraging deeper anti-concentration results could yield improvements if even “nicer” representations for threshold functions can be constructed; some results of Vu or Tao–Vu are suggested as potential avenues.
• Beyond Uniform Distributions: Some generalizations already hold for constant-biased and $k$-wise independent distributions; further extension to arbitrary product distributions is of interest.

7. Conclusion

Optimal threshold functions, as formalized in this context, are threshold functions or approximators which, subject to an error parameter $\epsilon$, achieve minimal complexity—either in the number of relevant variables (sharp junta size), or in integer weight magnitude. By unifying harmonic analytic methods, probabilistic anti-concentration bounds, and linear programming-based rounding strategies, this body of work provides tight (or near-tight) quantitative results. These in turn inform algorithm design in learning theory, provide stronger lower bounds in circuit complexity, and chart methodological advances in Boolean function analysis.