Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Generic proving of replica symmetry breaking (2311.18106v1)

Published 29 Nov 2023 in cond-mat.stat-mech, cs.IT, math-ph, math.IT, math.MP, and math.PR

Abstract: We study the replica symmetry breaking (rsb) concepts on a generic level through the prism of recently introduced interpolating/comparison mechanisms for bilinearly indexed (bli) random processes. In particular, \cite{Stojnicnflgscompyx23} introduced a \emph{fully lifted} (fl) interpolating mechanism and \cite{Stojnicsflgscompyx23} developed its a \emph{stationarized fully lifted} (sfl) variant. Here, we present a sfl \emph{matching} mechanism that shows that the results obtained in \cite{Stojnicnflgscompyx23,Stojnicsflgscompyx23} completely correspond to the ones obtained by a statistical physics replica tool with the replica symmetry breaking (rsb) form suggested by Parisi in \cite{Par79,Parisi80,Par80}. The results are very generic as they allow to handle pretty much all bilinear models at once. Moreover, given that the results of \cite{Stojnicnflgscompyx23,Stojnicsflgscompyx23} are extendable to many other forms, the concepts presented here automatically extend to any such forms as well.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (107)
  1. D. Achlioptas and Y. Peres. The threshold for random k-SAT is 2⁢k⁢l⁢n⁢k−O⁢(k)2𝑘𝑙𝑛𝑘𝑂𝑘2klnk-{O}(k)2 italic_k italic_l italic_n italic_k - italic_O ( italic_k ). Journal of the AMS, 17:947–973, 2004.
  2. R. J. Adler. An introduction to Continuity, Extrema, and Related Topic for General Gaussian Processes. Institute of Mathematical Statistics, 1990.
  3. Analogue neural networks on correlated random graphs. J. Phys. A: Math. Theor., 45:365001, 2012.
  4. Parallel processing in immune networks. Phys. Rev. E, 2012.
  5. Multitasking associative networks. Phys. Rev. Lett, 2012.
  6. A. E. Alaoui and M. Sellke. Algorithmic pure states for the negative spherical perceptron. 2020. available online at http://arxiv.org/abs/2010.15811.
  7. D. J. Aldous. Asymptotics in the random assignment problem. Probab Theory Related Fields, 93:507–534, 1992.
  8. D. J. Aldous. The zeta(2) limit in the random assignment problem. Random Structures Algorithms, 18:381–418, 2001.
  9. Exact expectations and distributions for the random assignment problem. Combinat. Probab. Comput., 11(3):217–248, 2002.
  10. Phys. Rev. A, 32:1007, 1985.
  11. P. Baldi and S. Venkatesh. Number od stable points for spin-glasses and neural networks of higher orders. Phys. Rev. Letters, 58(9):913–916, Mar. 1987.
  12. The replica symmetric approximation of the analogical neural network. J. Stat. Physics, July 2010.
  13. Equlibrium statistical mechanics of bipartite spin systems. Journal of Physics A: Mathematical and Theoeretical, 44(245002), 2011.
  14. How glassy are neural networks. J. Stat. Mechanics: Thery and Experiment, July 2012.
  15. M. Bayati and A. Montanari. The lasso risk of gaussian matrices. 2010. available online at http://arxiv.org/abs/1008.2581.
  16. M. Bayati and A. Montanari. The dynamics of message passing on dense graphs, with applications to compressed sensing. IEEE Transactions on Information Theory, 57(2), 2011.
  17. Asymmetric Little spin glas model. Physical Review B, 46(9), September 1992.
  18. Eigenstates and limit cycles in the SK model. Journal of Physics A: Mathematical and General, 21:4201, 1988.
  19. S. H. Cameron. Tech-report 60-600. Proceedings of the bionics symposium, pages 197–212, 1960. Wright air development division, Dayton, Ohio.
  20. A. Coja-Oghlan. The asymptotic k-SAT threshold. Proceedings of the forty-fifth annual ACM symposium on theory of computing (STOC), pages 804–813, 2014.
  21. D. Coppersmith and G. Sorkin. Constructive bounds and exact expectations for the random assignment problem. Random Structures Algorithms, 15:113–144, 1999.
  22. D. Coppersmith and G. Sorkin. On the expected incremental cost of a minimum assignment. Contemporary combinatories, Bolyai Society Mathematical Studies, 10(8), 2002. B. Bollobas (Editor), Springer, New York.
  23. T. Cover. Geomretrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Transactions on Electronic Computers, (EC-14):326–334, 1965.
  24. Satisfiability Threshold for Random Regular nae-sat. Communications in Mathematical Physics, 341(2):435–489, 2015.
  25. Satisfiability threshold for random regular NAE-SAT. Proceedings of the forty-sixth annual ACM symposium on theory of computing (STOC), pages 814–822, 2015.
  26. D. Donoho. High-dimensional centrally symmetric polytopes with neighborlines proportional to dimension. Disc. Comput. Geometry, 35(4):617–652, 2006.
  27. D. Donoho and J. Tanner. Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing. Preprint, 2009. available at arXiv:0906.2530.
  28. X. Fernique. Des resultats nouveaux sur les processus Gaussiens. C.R. Acad. Sci. Paris Ser A-B, 278:A363–A365, 1974.
  29. X. Fernique. Regularite des trajectoires des fonctions aleatoires Gaussiens. Springer Lecture notes, 480:1–96, 1975.
  30. Jamming in multilayer supervised learning models. Phys. Rev. Lett., 123(16):160602, 2019.
  31. S. Franz and G. Parisi. The simplest model of jamming. Journal of Physics A: Mathematical and Theoretical, 49(14):145001, 2016.
  32. Universality of the SAT-UNSAT (jamming) threshold in non-convex continuous constraint satisfaction problems. SciPost Physics, 2:019, 2017.
  33. Critical jammed phase of the linear perceptron. Phys. Rev. Lett., 123(11):115702, 2019.
  34. Surfing on minima of isostatic landscapes: avalanches and unjamming transition. SciPost Physics, 9:12, 2020.
  35. A. Frieze and N. Wormald. Random k-Sat: a tight threshold for moderately growing k. Combinatorica, 28:297–305, 2005.
  36. E. Gardner. The space of interactions in neural networks models. J. Phys. A: Math. Gen., 21:257–270, 1988.
  37. E. Gardner and B. Derrida. Optimal storage properties of neural networks models. J. Phys. A: Math. Gen., 21:271–284, 1988.
  38. Y. Gordon. Some inequalities for Gaussian processes and applications. Israel Journal of Mathematics, 50(4):265–289, 1985.
  39. F. Guerra. Broken replica symmetry bounds in the mean field spin glass model. Comm. Math. Physics, 233:1–12, 2003.
  40. H. Gutfreund and Y. Stein. Capacity of neural networks with discrete synaptic couplings. J. Physics A: Math. Gen, 23:2613, 1990.
  41. D. O. Hebb. Organization of behavior. New York: Wiley, 1949.
  42. J. J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Science, 79:2554, 1982.
  43. R. D. Joseph. The number of orthants in n𝑛nitalic_n-space instersected by an s𝑠sitalic_s-dimensional subspace. Tech. memo 8, project PARA, 1960. Cornel aeronautical lab., Buffalo, N.Y.
  44. J. P. Kahane. Une inegualite du type de Slepian et Gordon sur les processus Gaussiens. Israel Journal of Mathematics, 55(1):109–110, 1986.
  45. Covering cubes by random half cubes with applications to biniary neural networks. Journal of Computer and System Sciences, 56:223–252, 1998.
  46. W. Krauth and M. Mezard. Storage capacity of memory networks with binary couplings. J. Phys. France, 50:3057–3066, 1989.
  47. M. Ledoux and M. Talagrand. Probability in Banach spaces: Isopermetry and Processes. Springer (New York), 1991.
  48. M. A. Lifshits. Gaussian random functions. Kluwer, Boston, 1995.
  49. S. Linusson and J. Wastlund. A proof of Parisi’s conjecture on the random assignment problem. Probabil Theory Related Fields, 128(3):419–440, 2004.
  50. W. A. Little. The existence of persistent states in the brain. Math. Biosci., 19(1-2):101–120, 1974.
  51. Threshold values of random K-SAT from the cavity method. Random Struct. Alg., 28:340–373, 2006.
  52. M. Mezard and G. Parisi. On the solution of the random link matching problem. J Physique, 48:1451–1459, 1987.
  53. Analytic and algorithmic solution of random satisfiability problems. Science, 297:812–815, 2002.
  54. M. Molloy. The freezing threshold for k-colourings of a random graph. Proceedings of the forty-third annual ACM symposium on theory of computing (STOC), pages 921–930, 2012.
  55. Proofs of the Parisi and Coppersmith-Sorkin random assignment conjectures. Random Structures and Algorithms, 27(4):413–444, 2005.
  56. R. O.Winder. Threshold logic. Ph. D. dissertation, Princetoin University, 1962.
  57. D. Panchenko. A connection between the Ghirlanda-Guerra identities and ultrametricity. The Annals of Probability, 38(1):327–347, 2010.
  58. D. Panchenko. The Ghirlanda-Guerra identities for mixed p-spin model. Comptes Rendus Mathematique, 348(3-4):189–192, 2010.
  59. D. Panchenko. The Parisi ultrametricity conjecture. Ann. Math., 77(1):383–393, 2013.
  60. D. Panchenko. The Sherrington-Kirkpatrick model. Springer Science & Business Media, 2013.
  61. D. Panchenko. On the replica symmetric solution of the l-sat model. Electronic Journal of Probability, 19, 2014.
  62. G. Parisi. Infnite number of order parameters for spin-glasses. Phys. Rev. Lett., 43:1754–1756, 1979.
  63. G. Parisi. Breaking the symmetry in SK model. J. Physics, A13:1101, 1980.
  64. G. Parisi. A sequence of approximated solutions to the SK model for spin glasses. Journal of Physics A: Mathematical and General, 13(4):L115, 1980.
  65. G. Parisi. Order parameter for spin glasses. Phys. Rev. Lett., 50:1946, 1983.
  66. L. Pastur and A. Figotin. On the theory of disordered spin systems. Theory Math. Phys., 35(403-414), 1978.
  67. The replica-symmetric solution without the replica trick for the Hopfield model. Journal of Statistical Physics, 74(5/6), 1994.
  68. L. Schlafli. Gesammelte Mathematische AbhandLungen I. Basel, Switzerland: Verlag Birkhauser, 1950.
  69. M. Shcherbina and B. Tirozzi. The free energy of a class of Hopfield models. Journal of Statistical Physics, 72(1/2), 1993.
  70. M. Shcherbina and B. Tirozzi. On the volume of the intrersection of a sphere with random half spaces. C. R. Acad. Sci. Paris. Ser I, (334):803–806, 2002.
  71. M. Shcherbina and B. Tirozzi. Rigorous solution of the Gardner problem. Comm. on Math. Physics, (234):383–422, 2003.
  72. D. Sherrington and S. Kirkpatrick. Solvable model of a spin glass. Phys. Rev. Letters, 35:1792–1796, 1972.
  73. D. Slepian. The one sided barier problem for Gaussian noise. Bell System Tech. Journal, 41:463–501, 1962.
  74. M. Stojnic. A framework for perfromance characterization of LASSO algortihms. available online at http://arxiv.org/abs/1303.7291.
  75. M. Stojnic. Upper-bounding ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-optimization weak thresholds. available online at http://arxiv.org/abs/1303.7289.
  76. M. Stojnic. Various thresholds for ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-optimization in compressed sensing. available online at http://arxiv.org/abs/0907.3666.
  77. M. Stojnic. Block-length dependent thresholds for ℓ2/ℓ1subscriptℓ2subscriptℓ1\ell_{2}/\ell_{1}roman_ℓ start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT / roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-optimization in block-sparse compressed sensing. ICASSP, IEEE International Conference on Acoustics, Signal and Speech Processing, pages 3918–3921, 14-19 March 2010. Dallas, TX.
  78. M. Stojnic. ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT optimization and its various thresholds in compressed sensing. ICASSP, IEEE International Conference on Acoustics, Signal and Speech Processing, pages 3910–3913, 14-19 March 2010. Dallas, TX.
  79. M. Stojnic. Recovery thresholds for ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT optimization in binary compressed sensing. ISIT, IEEE International Symposium on Information Theory, pages 1593 – 1597, 13-18 June 2010. Austin, TX.
  80. M. Stojnic. Towards improving ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT optimization in compressed sensing. ICASSP, IEEE International Conference on Acoustics, Signal and Speech Processing, pages 3938–3941, 14-19 March 2010. Dallas, TX.
  81. M. Stojnic. Another look at the Gardner problem. 2013. available online at http://arxiv.org/abs/1306.3979.
  82. M. Stojnic. Asymmetric Little model and its ground state energies. 2013. available online at http://arxiv.org/abs/1306.3978.
  83. M. Stojnic. Bounds on restricted isometry constants of random matrices. 2013. available online at http://arxiv.org/abs/1306.3779.
  84. M. Stojnic. Discrete perceptrons. 2013. available online at http://arxiv.org/abs/1303.4375.
  85. M. Stojnic. Lifting ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-optimization strong and sectional thresholds. 2013. available online at http://arxiv.org/abs/1306.3770.
  86. M. Stojnic. Lifting/lowering Hopfield models ground state energies. 2013. available online at http://arxiv.org/abs/1306.3975.
  87. M. Stojnic. Negative spherical perceptron. 2013. available online at http://arxiv.org/abs/1306.3980.
  88. M. Stojnic. Regularly random duality. 2013. available online at http://arxiv.org/abs/1303.7295.
  89. M. Stojnic. Spherical perceptron as a storage memory with limited errors. 2013. available online at http://arxiv.org/abs/1306.3809.
  90. M. Stojnic. Fully bilinear generic and lifted random processes comparisons. 2016. available online at http://arxiv.org/abs/1612.08516.
  91. M. Stojnic. Generic and lifted probabilistic comparisons – max replaces minmax. 2016. available online at http://arxiv.org/abs/1612.08506.
  92. M. Stojnic. Bilinearly indexed random processes – stationarization of fully lifted interpolation. 2023. available online at arxiv.
  93. M. Stojnic. Fully lifted interpolating comparisons of bilinearly indexed random processes. 2023. available online at arxiv.
  94. V. N. Sudakov. Gaussian random processes and measures of solid angles in Hilbert space. Soviet Math. Dokl., 12(1):412–415, 1971.
  95. M. Talagrand. Rigorous results for the Hopfield model with many patterns. Prob. theory and related fields, 110:177–276, 1998.
  96. M. Talagrand. An assignment problem at high temperature. Ann. Probab., 31(2):818–848, 2003.
  97. M. Talagrand. The Generic Chaining. Springer-Verlag, 2005.
  98. M. Talagrand. The Parisi formula. Annals of mathematics, 163(2):221–263, 2006.
  99. M. Talagrand. Mean field models and spin glasse: Volume II. A series of modern surveys in mathematics 55, Springer-Verlag, Berlin Heidelberg, 2011.
  100. M. Talagrand. Mean field models and spin glasses: Volume I. A series of modern surveys in mathematics 54, Springer-Verlag, Berlin Heidelberg, 2011.
  101. S. Venkatesh. Epsilon capacity of neural networks. Proc. Conf. on Neural Networks for Computing, Snowbird, UT, 1986.
  102. J. Wastlund. A proof of a conjecture of Buck, Chan, and Robbins on the expected value of the minimum assignment. Random Structures Algorithms, 26(1-2):237–251, 2004.
  103. J. Wastlund. An easy proof of the ζ⁢(2)𝜁2\zeta(2)italic_ζ ( 2 ) limit in the random assignment problem. Electronic Communications in Probability, 14:1475, 2009.
  104. J. Wastlund. Replica symmetry of the minimum matching. Annals of Mathematics, 175(3):1061–1091, 2012.
  105. J. G. Wendel. A problem in geometric probability. Mathematica Scandinavica, 1:109–111, 1962.
  106. J. G. Wendel. A problem in geometric probablity. Mathematics Scandinavia, 11:109–111, 1962.
  107. R. O. Winder. Single stage threshold logic. Switching circuit theory and logical design, pages 321–332, Sep. 1961. AIEE Special publications S-134.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now