Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Probabilistic programming interfaces for random graphs: Markov categories, graphons, and nominal sets (2312.17127v1)

Published 28 Dec 2023 in cs.PL, cs.LO, and math.PR

Abstract: We study semantic models of probabilistic programming languages over graphs, and establish a connection to graphons from graph theory and combinatorics. We show that every well-behaved equational theory for our graph probabilistic programming language corresponds to a graphon, and conversely, every graphon arises in this way. We provide three constructions for showing that every graphon arises from an equational theory. The first is an abstract construction, using Markov categories and monoidal indeterminates. The second and third are more concrete. The second is in terms of traditional measure theoretic probability, which covers 'black-and-white' graphons. The third is in terms of probability monads on the nominal sets of Gabbay and Pitts. Specifically, we use a variation of nominal sets induced by the theory of graphs, which covers Erd\H{o}s-R\'enyi graphons. In this way, we build new models of graph probabilistic programming from graphons.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (95)
  1. Nathanael Ackerman. 2015. Representations of Aut(M)-Invariant Measures. arXiv e-print 1509.06170 (2015).
  2. Properly ergodic structures. arXiv e-print 1710.09336 (2017).
  3. Invariant measures concentrated on countable structures. Forum Math. Sigma 4 (2016), e17.
  4. On computable representations of exchangeable data. Workshop on Probabilistic Programming Semantics (PPS 2017). https://pps2017.luddy.indiana.edu/files/2016/12/compAH.pdf
  5. Algorithmic barriers to representing conditional independence. In Proc. 34th ACM/IEEE Symp. Logic in Comp. Sci. (LICS 2019). 1–13.
  6. Exchangeable Random Primitives. Workshop on Probabilistic Programming Semantics (PPS 2016). http://pps2016.luddy.indiana.edu/files/2015/12/xrp.pdf
  7. Wilhelm Ackermann. 1937. Die Widerspruchsfreiheit der allgemeinen Mengenlehre. Math. Ann. 114, 1 (1937), 305–315.
  8. J Adámek and J Rosický. 1994. Locally presentable and accessible categories. Cambridge University Press.
  9. Universal Properties of Partial Quantum Maps. In Proc QPL 2022.
  10. Random orderings and unique ergodicity of automorphism groups. J. Eur. Math. Soc. (JEMS) 16, 10 (2014), 2059–2095.
  11. Pedro Henrique Azevedo de Amorim. 2023. A Higher-Order Language for Markov Kernels and Linear Operators. In Proc. FOSSACS 2023.
  12. Linear λ𝜆\lambdaitalic_λ-calculus and categorical models revisited. In Proc. CSL 1992.
  13. Automata theory in nominal sets. Logical Methods in Computer Science 10, 3 (2014).
  14. M. Bojańczyk and T. Place. 2012. Towards model theory with data values. In Proc. ICALP 2012.
  15. Stefan Bornholdt and Hans Georg Schuster (Eds.). 2002. Handbook of Graphs and Networks. Wiley.
  16. Testing for high-dimensional geometry in random graphs. Random Structures & Algorithms 49, 3 (2016), 503–532.
  17. Local exchangeability. Bernoulli 29, 3 (2023), 2084–2100.
  18. Olivia Caramello. 2013. Topological Galois Theory. Advances in Mathematics 291 (2013).
  19. Olivia Caramello. 2014. Fraïssé’s Construction from a Topos-Theoretic Perspective. Logica Universalis 8 (2014), 261–281.
  20. Introduction to extensive and distributive categories. J. Pure Appl. Algebra 84, 2 (1993), 145–158.
  21. François Caron and Emily B. Fox. 2017. Sparse Graphs Using Exchangeable Random Measures. Journal of the Royal Statistical Society Series B: Statistical Methodology 79, 5 (09 2017), 1295–1366.
  22. J R B Cockett. 1993. Introduction to distributive categories. Math. Struct. Comput. Sci. 3, 3 (1993), 277–307.
  23. Harry Crane and Henry Towsner. 2018. Relatively exchangeable structures. J. Symbolic Logic 83, 2 (2018), 416–442.
  24. Categorical Foundations of Gradient-Based Learning. In Proc. ESOP 2021.
  25. Layer by layer: composing monads. In Proc. ICTAC 2018.
  26. Affine monads and lazy structures for Bayesian programming. In Proc. POPL 2023.
  27. Measurable cones and stable, measurable functions: a model for probabilistic higher-order programming. Proc. ACM Program. Lang. 2, POPL (2018), 59:1–59:28.
  28. Thomas Ehrhard and Christine Tasson. 2019. Probabilistic call by push value. Logical Methods in Computer Science 15, 1 (2019).
  29. P. Erdős and A. Rényi. 1959. On Random Graphs I. Publicationes Mathematicae Debrecen 6 (1959), 290.
  30. M P Fiore. 2005. Mathematical models of computational and combinatorial structures. In Proc. FOSSACS 2005.
  31. Backprop as Functor: A Compositional Perspective on Supervised Learning. In Proceedings of the 34th Annual ACM/IEEE Symposium on Logic in Computer Science (Vancouver, Canada) (LICS ’19). IEEE Press, Article 11, 13 pages.
  32. Cameron Freer and Daniel Roy. 2010. Posterior distributions are computable from predictive distributions. In Proc. 13th Int. Conf. Artificial Intelligence and Statistics (AISTATS 2010). 233–240.
  33. Cameron E. Freer and Daniel M. Roy. 2012. Computable de Finetti measures. Annals of Pure and Applied Logic 163, 5 (2012), 530–546.
  34. T. Fritz. 2020. A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics. Adv. Math. 370 (2020).
  35. Weakly affine monads. arXiv e-print 2303.14049 (2023).
  36. Murdoch Gabbay and Andrew M Pitts. 1999. A New Approach to Abstract Syntax Involving Binders. In Proc. LICS 1999. 214–224.
  37. Haim Gaifman. 1964. Concerning measures in first order calculi. Israel J. Math. 2 (1964), 1–18.
  38. Michèle Giry. 1980. A categorical approach to probability theory. In Categorical aspects of topology and analysis (Lecture Notes in Math., Vol. 915). 68–85.
  39. Church: A universal language for generative models. In Proc. 24th Conf. Uncertainty in Artificial Intelligence (UAI 2008). AUAI Press, 220–229.
  40. N. D. Goodman and J. B. Tenenbaum. 2023. Probabilistic Models of Cognition. http://v1.probmods.org
  41. A Domain-Theoretic Approach to Statistical Programming Languages. arXiv e-print 2106.16190 (2021).
  42. Claudio Hermida and Robert Tennent. 2012. Monoidal indeterminates and categories of possible worlds. Theoretical Computer Science 430 (2012).
  43. A convenient category for higher-order probability theory. In Proc. LICS 2017.
  44. Hongde Hu and Walter Tholen. 1995. Limits in free coproduct completions. Journal of Pure and Applied Algebra 105 (1995), 277–291.
  45. Mathieu Huot and Sam Staton. 2018. Universal Properties in Quantum Theory. In Proc. QPL 2018.
  46. Bart Jacobs. 1994. Semantics of weakening and contraction. Ann. Pure & Appl. Logic 69, 1 (1994), 73–106.
  47. Bart Jacobs. 2016. Introduction to coalgebra: Towards mathematics of states and observations. CUP.
  48. Bart Jacobs. 2018. From probability monads to commutative effectuses. J. Log. Algebr. Methods Program. 94 (2018), 200–237.
  49. Bart Jacobs and Jorik Mandemaker. 2012. The Expectation Monad in Quantum Foundations. In Proceedings 8th International Workshop on Quantum Physics and Logic (Electronic Proceedings in Theoretical Computer Science, Vol. 95). Open Publishing Association, 143–182.
  50. Svante Janson. 2013. Graphons, cut norm and distance, couplings and rearrangements. New York Journal of Mathematics Monographs 4 (2013).
  51. C Barry Jay. 1993. Tail recursion through universal invariants. Theoret. Comput. Sci 115, 1 (1993), 151–189.
  52. Commutative Monads for Probabilistic Programming Languages. In 36th Annual ACM/IEEE Symposium on Logic in Computer Science, LICS 2021, Rome, Italy, June 29 - July 2, 2021. IEEE, 1–14.
  53. Peter T. Johnstone. 2002. Sketches of an Elephant – A Topos Theory Compendium. OUP.
  54. A generalization of hierarchical exchangeability on trees to directed acyclic graphs. Annales Henri Lebesgue 4 (2021).
  55. Younesse Kaddar and Sam Staton. 2023. A model of stochastic memoization and name generation in probabilistic programming: categorical semantics via monads on presheaf categories. In Proc. MFPS 2023. To appear.
  56. Olav Kallenberg. 2010. Foundations of Modern Probability (2nd ed.). Springer New York.
  57. Ohad Kammar. 2014. An algebraic theory of type-and-effect systems. Ph. D. Dissertation. University of Edinburgh.
  58. Ohad Kammar and Dylan McDermott. 2018. Factorisation Systems for Logical Relations and Monadic Lifting in Type-and-effect System Semantics. In Proc. MFPS 2018. 239–260.
  59. Fraïssé limits, Ramsey theory, and topological dynamics of automorphism groups. Geom. Funct. Anal. 15, 1 (2005), 106–189.
  60. Oleg Kiselyov and Chung-Chieh Shan. 2010. Probabilistic programming using first-class stores and first-class continuations. In Proc. 2010 ACM SIGPLAN Workshop on ML.
  61. Homomorphism Problems for First-Order Definable Structures. In Proc. FSTTCS 2016.
  62. Anders Kock. 1970. Monads on symmetric monoidal closed categories. Arch. Math. 21 (1970), 1–10.
  63. Anders Kock. 2012. Commutative monads as a theory of distributions. Theory Appl. Categ. 26, 4 (2012), 97–131.
  64. Dexter Kozen. 1981. Semantics of probabilistic programs. J. Comput. System Sci. 22, 3 (1981), 328–350.
  65. H. Lindner. 1979. Affine parts of monads. Arch. Math. (1979).
  66. László Lovász. 2012. Large networks and graph limits. Amer. Math. Soc., Providence, RI. xiv+475 pages.
  67. László Lovász and Balázs Szegedy. 2006. Limits of dense graph sequences. Journal of Combinatorial Theory, Series B 96, 6 (2006), 933–957.
  68. Saunders Mac Lane. 1998. Categories for the Working Mathematician. Springer.
  69. Venture: a higher-order probabilistic programming platform with programmable inference. arXiv e-print 1404.0099 (2014).
  70. David Marker. 2002. Model Theory : An Introduction. Springer Science & Business Media.
  71. Concrete categories and higher-order recursion: With applications including probability, differentiability, and full abstraction. In LICS ’22: 37th Annual ACM/IEEE Symposium on Logic in Computer Science, Haifa, Israel, August 2 - 5, 2022, Christel Baier and Dana Fisman (Eds.). ACM, 57:1–57:14.
  72. E. Moggi. 1989. Computational Lambda-Calculus and Monads. In Proceedings of the Fourth Annual Symposium on Logic in Computer Science (Pacific Grove, California, USA). IEEE Press, 14–23.
  73. Peter Orbanz and Daniel M. Roy. 2015. Bayesian Models of Graphs, Arrays and Other Exchangeable Random Structures. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 2 (2015), 437–461.
  74. Duško Pavlović. 1997. Categorical logic of names and abstraction in action calculi. Mathematical Structures in Computer Science 7, 6 (1997), 619–637.
  75. Mathew Penrose. 2003. Random geometric graphs. Oxford University Press.
  76. Andrew M Pitts. 2001. Categorical logic. In Handbook of Logic in Computer Science. Vol. 5. OUP.
  77. Andrew M. Pitts. 2013. Nominal Sets: names and symmetry in computer science. CUP.
  78. Gordon D. Plotkin and John Power. 2002. Notions of Computation Determine Monads. In Proc. FOSSACS 2002.
  79. A J Power. 2006a. Semantics for local computational effects. In Proc. MFPS 2006.
  80. John Power. 2006b. Generic models for computational effects. Theor. Comput. Sci. 364, 2 (2006), 254–269.
  81. R. Rado. 1964. Universal Graphs and Universal Functions. Acta Arithmetica 9 (1964), 331–340. Issue 4.
  82. A stochastic programming perspective on nonparametric Bayes. In Workshop on Nonparametric Bayes (co-located with ICML 2008).
  83. Dan Shiebler. 2021. Categorical Stochastic Processes and Likelihood. Compositionality 3 (April 2021). Issue 1.
  84. Sam Staton. 2017. Commutative Semantics for Probabilistic Programming. In Proc. ESOP 2017. 855–879.
  85. The Beta-Bernoulli Process and Algebraic Effects. In Proc. ICALP 2018. Appendix at arXiv:1802.09598.
  86. Exchangeable Random Processes and Data Abstraction. Workshop on Probabilistic Programming Semantics (PPS 2017). https://pps2017.luddy.indiana.edu/files/2017/01/staton-yang-ackerman-freer-roy.pdf
  87. Dario Stein. 2021. Structural Foundations for Probabilistic Programming Languages. Ph. D. Dissertation. University of Oxford.
  88. Ross Street and Robert Walters. 1978. Yoneda structures on 2-categories. Journal of Algebra 50, 2 (1978), 350–379.
  89. Terence Tao. 2013. Ultraproducts as a Bridge Between Discrete and Continuous Analysis. https://terrytao.wordpress.com/2013/12/07/
  90. A domain theory for statistical probabilistic programming. Proc. ACM Program. Lang. 3, POPL (2019), 36:1–36:29.
  91. An Introduction to Probabilistic Programming. arXiv e-print 1809.10756 (2018).
  92. Victor Veitch and Daniel M. Roy. 2019. Sampling and estimation for (sparse) exchangeable graphs. The Annals of Statistics 47, 6 (2019), 3274 – 3299.
  93. R F C Walters. 1989. Data types in distributive categories. Bulletin of the Australian Mathematical Society 40 (1989), 79–82.
  94. A New Approach to Probabilistic Programming Inference. In Proc. 17th Int. Conf. on Artificial Intelligence and Statistics (AISTATS 2014). 1024–1032.
  95. Jeff Wu. 2013. Reduced Traces and JITing in Church. Master’s thesis. Mass. Inst. of Tech.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now