Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Exact objectives of random linear programs and mean widths of random polyhedrons (2403.03637v1)

Published 6 Mar 2024 in math.OC, cs.IT, math.IT, math.PR, and stat.ML

Abstract: We consider \emph{random linear programs} (rlps) as a subclass of \emph{random optimization problems} (rops) and study their typical behavior. Our particular focus is on appropriate linear objectives which connect the rlps to the mean widths of random polyhedrons/polytopes. Utilizing the powerful machinery of \emph{random duality theory} (RDT) \cite{StojnicRegRndDlt10}, we obtain, in a large dimensional context, the exact characterizations of the program's objectives. In particular, for any $\alpha=\lim_{n\rightarrow\infty}\frac{m}{n}\in(0,\infty)$, any unit vector $\mathbf{c}\in{\mathbb R}n$, any fixed $\mathbf{a}\in{\mathbb R}n$, and $A\in {\mathbb R}{m\times n}$ with iid standard normal entries, we have \begin{eqnarray*} \lim_{n\rightarrow\infty}{\mathbb P}{A} \left ( (1-\epsilon) \xi{opt}(\alpha;\mathbf{a}) \leq \min_{A\mathbf{x}\leq \mathbf{a}}\mathbf{c}T\mathbf{x} \leq (1+\epsilon) \xi_{opt}(\alpha;\mathbf{a}) \right ) \longrightarrow 1, \end{eqnarray*} where \begin{equation*} \xi_{opt}(\alpha;\mathbf{a}) \triangleq \min_{x>0} \sqrt{x2- x2 \lim_{n\rightarrow\infty} \frac{\sum_{i=1}{m} \left ( \frac{1}{2} \left (\left ( \frac{\mathbf{a}i}{x}\right )2 + 1\right ) \mbox{erfc}\left( \frac{\mathbf{a}_i}{x\sqrt{2}}\right ) - \frac{\mathbf{a}_i}{x} \frac{e{-\frac{\mathbf{a}_i2}{2x2}}}{\sqrt{2\pi}} \right ) }{n} }. \end{equation*} For example, for $\mathbf{a}=\mathbf{1}$, one uncovers \begin{equation*} \xi{opt}(\alpha) = \min_{x>0} \sqrt{x2- x2 \alpha \left ( \frac{1}{2} \left ( \frac{1}{x2} + 1\right ) \mbox{erfc} \left ( \frac{1}{x\sqrt{2}}\right ) - \frac{1}{x} \frac{e{-\frac{1}{2x2}}}{\sqrt{2\pi}} \right ) }. \end{equation*} Moreover, $2 \xi_{opt}(\alpha)$ is precisely the concentrating point of the mean width of the polyhedron ${\mathbf{x}|A\mathbf{x} \leq \mathbf{1}}$.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (77)
  1. S. Agmon. The relaxation method for linear inequalities. Canadian Journal of Mathematics, 6:382–392, 1954.
  2. D. Alonso-Gutierrez and J. Prochno. On the Gaussian behavior of marginals and the mean width of random polytopes. Proc. Amer. Math. Soc, 143(2):821–832, 2015.
  3. M. M. Babbar. Distributions of solutions of a set of linear equations (with an application to linear programming). J. Amer. Statist. Assoc., 50(1):854–869, 1955.
  4. On the optimal objective value of random linear programs. 2023. available online at http://arxiv.org/abs/2401.17530.
  5. Properties of the geometry of solutions and capacity of multilayer neural networks with rectified linear unit activations. Phys. Rev. Lett., 123:170602, October 2019.
  6. P. Baldi and S. Venkatesh. Number od stable points for spin-glasses and neural networks of higher orders. Phys. Rev. Letters, 58(9):913–916, Mar. 1987.
  7. Broken symmetries in multilayered perceptrons. Phys. Rev. A, 45(6):4146, March 1992.
  8. E. M. Beale. On minimizing a convex function subject to linear inequalities. Journal of the Royal Statistical Society: Series B (Methodological), 17(2):173–184, 1955.
  9. A. Ben-Tal and A. Nemirovski. Lectures on Modern Convex Optimization. Analysis, Algorithms, and Engineering Applications. Society for Industrial and Applied Mathematics, 2001.
  10. D. P. Bertsekas. Nonlinear Programming. Athena Scientific, second edition, 1999.
  11. D. Bertsimas and J. N. Tsitsiklis. Introduction to Linear Optimization. Athena Scientific, 1997.
  12. Gardner formula for Ising perceptron models at small densities. Proceedings of Thirty Fifth Conference on Learning Theory, PMLR, 178:1787–1911, 2022.
  13. K.-H. Borgwardt. The average number of pivot steps required by the simplex-method is polynomial. Z. Oper. Res. Ser. A-B, 26(5):A157–A177, 1982.
  14. K.-H. Borgwardt. A sharp upper bound for the expected number of shadow vertices in lp-polyhedra under orthogonal projection on two-dimensional planes. Math. Oper. Res., 24(3):544–603, 1999.
  15. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2003.
  16. K. Chandrasekaran and S. S. Vempala. Integer feasibility of random polytopes: random integer programs. In Proceedings of the 5th conference on Innovations in theoretical computer science, pages 449–458, 2014.
  17. S. Chatterjee. A generalization of the Lindenberg principle. The Annals of Probability, 34(6):2061–2076.
  18. T. Cover. Geomretrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Transactions on Electronic Computers, (EC-14):326–334, 1965.
  19. D. Dadush and S. Huiberts. A friendly smoothed analysis of the simplex method. STOC 18-Proceedings of the 50th Annual ACM Symposium on Theory of Computing, pages 390–403, 2018.
  20. G. B. Dantzig. Linear programming under uncertainty. Management Science, 1:197–206, 1955.
  21. J. Ding and N. Sun. Capacity lower bound for the Ising perceptron. STOC 2019: Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, pages 816–827, 2019.
  22. D. Donoho and J. Tanner. Counting the face of randomly projected hypercubes and orthants, with application. Discrete and Computational Geometry, 43:522–541, 2010.
  23. J. Dupacova. Stochastic programming with incomplete information: a surrey of results on postoptimization and sensitivity analysis. Optimization, 18(4):507–532, 1987.
  24. J. Dupacova and R. Wets. Asymptotic behavior of statistical estimators and of optimal solutions of stochastic optimization problems. The Annals of Statistics, pages 1517–1549, 1988.
  25. Y. C. Eldar and D. Needell. Acceleration of randomized Kaczmarz method via the Johnson-Lindenstrauss lemma. Numer. Algorithms, 58(2):163–177, 2011.
  26. Storage capacity and learning algorithms for two-layer neural networks. Phys. Rev. A, 45(10):7590, May 1992.
  27. The allocation of aircraft to routes: An example of linear programming under uncertain demand. Management Science, 3(1):45–73, 1956.
  28. R. M. Durbin G. J. Mitchison. Bounds on the learning capacity of some multi-layer networks. Biological Cybernetics, 60:345–365, 1989.
  29. E. Gardner. The space of interactions in neural networks models. J. Phys. A: Math. Gen., 21:257–270, 1988.
  30. Asymptotic shape of the convex hull of isotropic log-concave random vectors. Adv. in Appl. Math., 75:116–143, 2016.
  31. E. D. Gluskin. Extremal properties of orthogonal parallelepipeds and their applications to the geometry of banach spaces. Mat. Sb. (N.S.), 136(178)(1):85–96, 1988.
  32. Y. Gordon. On Milman’s inequality and random subspaces which escape through a mesh in Rnsuperscript𝑅𝑛{R}^{n}italic_R start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT. Geometric Aspect of of functional analysis, Isr. Semin. 1986-87, Lect. Notes Math, 1317, 1988.
  33. H. Gutfreund and Y. Stein. Capacity of neural networks with discrete synaptic couplings. J. Physics A: Math. Gen, 23:2613, 1990.
  34. Upper and lower bounds on the smoothed complexity of the simplex method. STOC 23-Proceedings of the 55th Annual ACM Symposium on Theory of Computing, pages 1904–1917, 2023.
  35. S Kaczmarz. Angenaherte auflosung von systemen linearer gleichungen. Bulletin international de lacademie polonaise des sciences et des lettres, 1937.
  36. P. Kall and J. Mayer. Stochastic linear programming, volume 7. Springer, 1979.
  37. Limit laws for empirical optimal solutions in random linear programs. Annals of Operations Research, 315(1):251–278, 2022.
  38. W. Krauth and M. Mezard. Storage capacity of memory networks with binary couplings. J. Phys. France, 50:3057–3066, 1989.
  39. H. W. Kuhn. Nonlinear programming. A historical view. In R. W. Cottle and C. E. Lemke, editors, Nonlinear Programming, volume 9 of SIAM-AMS Proceedings. American Mathematical Society, 1976.
  40. Nonlinear programming. In J. Neyman, editor, Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, pages 481–492. University of California Press, 1951.
  41. J. W. Lindeberg. Eine neue herleitung des exponentialgesetzes in der wahrscheinlichkeitsrechnung. Math. Z., 15:211–225, 1922.
  42. Smallest singular value of random matrices and geometry of random polytopes. Adv. Math., 195(2):491–523, 2005.
  43. Asymptotic confidence sets for random linear programs. 2023. available online at http://arxiv.org/abs/2302.12364.
  44. Y. Liu and C. Q. Gu. On greedy randomized block Kaczmarz method for consistent linear systems. Linear Algebra Appl., 616:178–200, 2021.
  45. D. G. Luenberger. Linear and Nonlinear Programming. Addison-Wesley, second edition, 1984.
  46. Sampling Kaczmarz-Motzkin method for linear feasibility problems: generalization and acceleration. Math. Program., 194(1-2):719–779, 2022.
  47. The relaxation method for linear inequalities. Canadian Journal of Mathematics, 6:393–404, 1954.
  48. S. Nakajima and N. Sun. Sharp threshold sequence and universality for Ising perceptron models. Proceedings of the 2023 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 638–674, 2023.
  49. D. Needell and J. A. Tropp. Paved with good intentions: analysis of a randomized block Kaczmarz method. Linear Algebra Appl., 441:199–221, 2014.
  50. A. Prekopa. On the probability distribution of the optimum of a random linear program. SIAM Journal on Control, 4(1):211–222, 1966.
  51. R. T. Rockafellar. Convex Analysis. Princeton University Press, 1970.
  52. A. Ruszczynski and A. Shapiro. Stochastic programming models. Handbooks in Operations Research and Management Science, pages 841–858, 1989.
  53. A. Shapiro. Asymptotic properties of statistical estimators in stochastic programming. The Annals of Statistics, pages 841–858, 1989.
  54. A. Shapiro. Asymptotic behavior of optimal solutions in stochastic programming. Mathematics of Operations Research, 18(4):829–845, 1993.
  55. M. Shcherbina and B. Tirozzi. Rigorous solution of the Gardner problem. Comm. on Math. Physics, (234):383–422, 2003.
  56. S. Smale. On the average number of steps of the simplex method of linear programming. Math. Programming, 27(3):241–262, 1983.
  57. Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time. J. ACM, 51(3):385–468, 2004.
  58. M. Stojnic. Various thresholds for ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-optimization in compressed sensing. available online at http://arxiv.org/abs/0907.3666.
  59. M. Stojnic. ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT optimization and its various thresholds in compressed sensing. ICASSP, IEEE International Conference on Acoustics, Signal and Speech Processing, pages 3910–3913, 14-19 March 2010. Dallas, TX.
  60. M. Stojnic. Recovery thresholds for ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT optimization in binary compressed sensing. ISIT, IEEE International Symposium on Information Theory, pages 1593 – 1597, 13-18 June 2010. Austin, TX.
  61. M. Stojnic. Another look at the Gardner problem. 2013. available online at http://arxiv.org/abs/1306.3979.
  62. M. Stojnic. Discrete perceptrons. 2013. available online at http://arxiv.org/abs/1303.4375.
  63. M. Stojnic. Meshes that trap random subspaces. 2013. available online at http://arxiv.org/abs/1304.0003.
  64. M. Stojnic. Regularly random duality. 2013. available online at http://arxiv.org/abs/1303.7295.
  65. M. Stojnic. Generic and lifted probabilistic comparisons – max replaces minmax. 2016. available online at http://arxiv.org/abs/1612.08506.
  66. M. Stojnic. Bilinearly indexed random processes – stationarization of fully lifted interpolation. 2023. available online at http://arxiv.org/abs/2311.18097.
  67. M. Stojnic. Binary perceptrons capacity via fully lifted random duality theory. 2023. available online at http://arxiv.org/abs/2312.00073.
  68. M. Stojnic. Fl RDT based ultimate lowering of the negative spherical perceptron capacity. 2023. available online at http://arxiv.org/abs/2312.16531.
  69. M. Stojnic. Exact capacity of the wide hidden layer treelike neural networks with generic activations. 2024. available online at http://arxiv.org/abs/2402.05719.
  70. T. Strohmer and R. Vershynin. A randomized kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl., 15(2):262–278, 2004.
  71. M. Talagrand. Intersecting random half cubes. Random Structures Algorithms, 15(3-4):436–449, 1999.
  72. M. Talagrand. Mean field models and spin glasse: Volume II. A series of modern surveys in mathematics 55, Springer-Verlag, Berlin Heidelberg, 2011.
  73. M. Talagrand. Mean field models and spin glasses: Volume I. A series of modern surveys in mathematics 54, Springer-Verlag, Berlin Heidelberg, 2011.
  74. G. Tintner. A note on stochastic linear programming). Econometrica, 28(1):490–495, 1960.
  75. J. G. Wendel. A problem in geometric probability. Mathematica Scandinavica, 1:109–111, 1962.
  76. R. O. Winder. Single stage threshold logic. Switching circuit theory and logical design, pages 321–332, Sep. 1961. AIEE Special publications S-134.
  77. J. A. Zavatone-Veth and C. Pehlevan. Activation function dependence of the storage capacity of treelike neural networks. Phys. Rev. E, 103:L020301, February 2021.

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com