Analyze random-feature learners

Analyze in-context learning by random-feature learners for single-index targets, extending the spherical-harmonic kernel-learner analysis to random feature maps.

Background

The kernel learner is analyzed using a fixed feature map consisting of finitely many spherical-harmonic sectors. The paper notes that random features are a more practical alternative that should lead to related architectures, but their behavior is not analyzed.

Resolving this problem would determine whether the paper’s kernel-versus-feature comparison and context-length conclusions extend beyond the analytically convenient spherical-harmonic construction.

References

We leave the analysis of such random-feature learners to future work.

— In-context Learning of Single-index Targets: Comparing Kernel and Feature Learners  (2610.01712 - Gu et al., 1 Oct 2026) in Section 2, subsection “Kernel learner”