Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Conditional Gradients for the Approximate Vanishing Ideal (2202.03349v16)

Published 7 Feb 2022 in cs.LG

Abstract: The vanishing ideal of a set of points $X\subseteq \mathbb{R}n$ is the set of polynomials that evaluate to $0$ over all points $\mathbf{x} \in X$ and admits an efficient representation by a finite set of polynomials called generators. To accommodate the noise in the data set, we introduce the pairwise conditional gradients approximate vanishing ideal algorithm (PCGAVI) that constructs a set of generators of the approximate vanishing ideal. The constructed generators capture polynomial structures in data and give rise to a feature map that can, for example, be used in combination with a linear classifier for supervised learning. In PCGAVI, we construct the set of generators by solving constrained convex optimization problems with the pairwise conditional gradients algorithm. Thus, PCGAVI not only constructs few but also sparse generators, making the corresponding feature transformation robust and compact. Furthermore, we derive several learning guarantees for PCGAVI that make the algorithm theoretically better motivated than related generator-constructing methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. On the equivalence between herding and conditional gradient algorithms. In Proceedings of the International Conference on Machine Learning, pages 1355–1362. PMLR.
  2. Decomposition-invariant conditional gradient for general polytopes with line search. In Proceedings of Advances in Neural Information Processing Systems, pages 2687–2697.
  3. Skin segmentation dataset. UCI Machine Learning Repository.
  4. Multi-output polynomial networks and factorization machines. In Proceedings of Advances in Neural Information Processing Systems, pages 3349–3359.
  5. Weakly-supervised alignment of video with text. In Proceedings of the International Conference on Computer Vision, pages 4462–4470. IEEE.
  6. Lazifying conditional gradient algorithms. Journal of Machine Learning Research, 20(71):1–42.
  7. Boosting frank-wolfe by chasing gradients. In Proceedings of the International Conference on Machine Learning, pages 2111–2121. PMLR.
  8. Optimal transport for domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(9):1853–1865.
  9. Ideals, varieties, and algorithms: an introduction to computational algebraic geometry and commutative algebra. Springer Science & Business Media.
  10. UCI machine learning repository.
  11. Fassino, C. (2010). Almost vanishing polynomials for sets of limited precision points. Journal of Symbolic Computation, 45(1):19–37.
  12. An algorithm for quadratic programming. Naval Research Logistics Quarterly, 3(1-2):95–110.
  13. An extended Frank-Wolfe method with “in-face” directions, and its application to low-rank matrix completion. SIAM Journal on optimization, 27(1):319–346.
  14. Faster rates for the Frank-Wolfe method over strongly-convex sets. In Proceedings of the International Conference on Machine Learning. PMLR.
  15. A linearly convergent variant of the conditional gradient algorithm under strong convexity, with applications to online and stochastic optimization. SIAM Journal on Optimization, 26(3):1493–1528.
  16. Linear-memory and decomposition-invariant linearly convergent conditional gradient algorithm for structured polytopes. volume 29, pages 1001–1009.
  17. Optimizing over the growing spectrahedron. In European Symposium on Algorithms, pages 503–514. Springer.
  18. Some comments on Wolfe’s ‘away step’. Mathematical Programming, 35(1):110–119.
  19. Large-scale image classification with trace-norm regularization. In Proceedings of the Conference on Computer Vision and Pattern Recognition, pages 3386–3393. IEEE.
  20. Approximate computation of zero-dimensional polynomial ideals. Journal of Symbolic Computation, 44(11):1566–1591.
  21. Principal variety analysis. In Proceedings of the Conference on Robot Learning, pages 97–108.
  22. Jaggi, M. (2013). Revisiting Frank-Wolfe: Projection-free sparse convex optimization. In Proceedings of the International Conference on Machine Learning, number CONF, pages 427–435. PMLR.
  23. A simple algorithm for nuclear norm regularized problems. In Proceedings of the International Conference on Machine Learning. PMLR.
  24. Efficient image and video co-localization with frank-wolfe algorithm. In Computer Vision – ECCV 2014, pages 253–268, Cham. Springer International Publishing.
  25. Spurious vanishing problem in approximate vanishing ideal. IEEE Access, 7:178961–178976.
  26. Gradient boosts the approximate vanishing ideal. In Proceedings of the Conference on Artificial Intelligence, number 04, pages 4428–4435.
  27. Vanishing ideal genetic programming. In 2016 IEEE Congress on Evolutionary Computation (CEC), pages 5018–5025.
  28. Affine invariant analysis of frank-wolfe on strongly convex sets. In Proceedings of the International Conference on Machine Learning, pages 5398–5408. PMLR.
  29. Computational Commutative Algebra. Springer Berlin Heidelberg.
  30. An affine invariant linear convergence analysis for Frank-Wolfe algorithms. arXiv preprint arXiv:1312.7864.
  31. On the global linear convergence of Frank-Wolfe optimization variants. In Proceedings of Advances in Neural Information Processing Systems, pages 496–504.
  32. Probability in Banach Spaces: isoperimetry and processes, volume 23. Springer Science & Business Media.
  33. Constrained minimization methods. USSR Computational Mathematics and Mathematical Physics, 6(5):1–50.
  34. Limbeck, J. (2013). Computation of approximate border bases and applications.
  35. Vanishing component analysis. In Proceedings of the International Conference on Machine Learning, pages 597–605. PMLR.
  36. Sinkhorn barycenters with free support via Frank-Wolfe algorithm. In Proceedings of Advances in Neural Information Processing Systems, pages 9318–9329.
  37. Fifty years of pulsar candidate selection: from simple filters to a new principled real-time classification approach. Monthly Notices of the Royal Astronomical Society, 459(1):1104–1123.
  38. Foundations of machine learning. MIT press.
  39. The construction of multivariate polynomials with preassigned zeros. In European Computer Algebra Conference, pages 24–31. Springer.
  40. Nesterov, Y. (1983). A method for unconstrained convex minimization problem with the rate of convergence O⁢(1/k2)𝑂1superscript𝑘2O(1/k^{2})italic_O ( 1 / italic_k start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ). In Doklady an USSR, volume 269, pages 543–547.
  41. Subspace robust wasserstein distances. In Proceedings of the International Conference on Machine Learning, pages 5072–5081. PMLR.
  42. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12:2825–2830.
  43. Weakly-supervised learning of visual relations. In Proceedings of the International Conference on Computer Vision, pages 5179–5188. IEEE.
  44. Learning infinite rbms with Frank-Wolfe. In Proceedings of Advances in Neural Information Processing Systems, pages 3063–3071.
  45. Least squares support vector machine classifiers. Neural Processing Letters, 9(3):293–300.
  46. Nonlinear blind source separation unifying vanishing component analysis and temporal structure. IEEE Access, 6:42837–42850.
  47. The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients. Expert Systems with Applications, 36(2):2473–2480.
  48. Zhang, X. (2018). Improvement on the vanishing component analysis by grouping strategy. EURASIP Journal on Wireless Communications and Networking, 2018(1):111.
  49. Hand posture recognition using approximate vanishing ideal generators. In Proceedings of the International Conference on Image Processing, pages 1525–1529. IEEE.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now