Analytical characterization of optimal regularization for hinge-loss SVM on RAF data

Derive a closed-form analytical characterization of the optimal regularization parameter λ_opt that minimizes the generalization error for support vector machines with hinge loss trained on the Rules-and-Facts (RAF) data model, as a function of the sample complexity α, the fraction of facts ε, and the kernel geometry (μ1, μ⋆).

Background

For kernel ridge regression on RAF data, the paper derives an explicit formula for the optimal regularization λ_opt that minimizes the generalization error and characterizes when perfect memorization is compatible with optimal generalization. For hinge-loss SVMs, the authors rely on numerical cross-validation to select λ and observe improved generalization at the expense of memorization but lack an analytical solution.

An analytical λ_opt for the hinge loss would parallel the square-loss result, enabling precise trade-off characterizations between memorization and generalization for SVMs on RAF data.