Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
134 tokens/sec
GPT-4o
9 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Probabilistic Programming with Exact Conditions (2312.17141v1)

Published 28 Dec 2023 in cs.PL, cs.LO, and math.PR

Abstract: We spell out the paradigm of exact conditioning as an intuitive and powerful way of conditioning on observations in probabilistic programs. This is contrasted with likelihood-based scoring known from languages such as Stan. We study exact conditioning in the cases of discrete and Gaussian probability, presenting prototypical languages for each case and giving semantics to them. We make use of categorical probability (namely Markov and CD categories) to give a general account of exact conditioning which avoids limits and measure theory, instead focusing on restructuring dataflow and program equations. The correspondence between such categories and a class of programming languages is made precise by defining the internal language of a CD category.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Categories in control. Theory Appl. Categ. 30 (2015), 836–881.
  2. Kantorovich problems and conditional measures depending on a parameter. Journal of Mathematical Analysis and Applications (2020).
  3. Graphical affine algebra. In Proc. LICS 2019 (2019).
  4. The calculus of signal flow diagrams I: linear relations on streams. Inform. Comput. 252 (2017).
  5. Stan: A probabilistic programming language. Journal of statistical software 76, 1 (2017).
  6. Disintegration and Bayesian inversion via string diagrams. Mathematical Structures in Computer Science 29 (2019), 938 – 971.
  7. Profunctor optics, a categorical update. arxiv:2001.07488, 2020.
  8. Probabilistic (logic) programming concepts. Mach. Learn. 100 (2015).
  9. Eaton, M. L. Multivariate statistics: A vector space approach. Lecture Notes-Monograph Series 53 (2007), i–512.
  10. Fong, B. Causal Theories: A Categorical Perspective on Bayesian Networks (MSc thesis). PhD thesis, University of Oxford, 2012.
  11. Fritz, T. A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics. Adv. Math. 370 (2020).
  12. Representable Markov categories and comparison of statistical experiments in categorical probability, 2020.
  13. Infinite products and zero-one laws in categorical probability. Compositionality 2 (Aug. 2020).
  14. Führmann, C. Varieties of effects. In Proc. FOSSACS 2002 (2002), pp. 144–159.
  15. PSI: Exact symbolic inference for probabilistic programs. In Proc. CAV 2016 (2016).
  16. Giry, M. A categorical approach to probability theory. Categorical Aspects of Topology and Analysis. Lecture Notes in Mathematics (1982).
  17. Golubtsov, P. V. Monoidal kleisli category as a background for information transformers theory. Inf. Process. 2, 1 (2002), 62–84.
  18. The Design and Implementation of Probabilistic Programming Languages. http://dippl.org, 2014. Accessed: 2021-8-3.
  19. Probabilistic Models of Cognition. http://probmods.org, 2016. Accessed: 2021-3-26.
  20. Monoidal indeterminates and categories of possible worlds. Theoret. Comput. Sci. 430 (2012).
  21. Bennett and Stinespring, together at last. In Proceedings of the 18th International Conference on Quantum Physics and Logic (QPL 2021) (Sept. 2021), no. 343 in Electronic Proceedings in Theoretical Computer Science, EPTCS, pp. 102–118.
  22. Universal properties in quantum theory. In Proc. QPL 2018 (2018).
  23. Infer.net tutorial 3: Learning a gaussian. https://dotnet.github.io/infer/userguide/Learning%20a%20Gaussian%20tutorial.html.
  24. Jacobs, B. A channel-based perspective on conjugate priors. Mathematical Structures in Computer Science 30, 1 (2020), 44–61.
  25. Jacobs, J. Paradoxes of probabilistic programming. In Proc. POPL 2021 (2021).
  26. Jaynes, E. T. Probability Theory: The Logic of Science. CUP, 2003.
  27. The geometry of tensor calculus, i. Advances in Mathematics 88 (1991), 55–112.
  28. Kallenberg, O. Foundations of Modern Probability. Springer, New York, 1997.
  29. Algebraic foundations for effect-dependent optimisations. In Proc. POPL 2012 (2012), pp. 349–360.
  30. Stable local computation with conditional Gaussian distributions. Statistics and Computing 11 (11 1999).
  31. Modelling environments in call-by-value programming languages. Information and Computation 185, 2 (2003), 182–210.
  32. MacLane, S. Categories for the Working Mathematician. Springer-Verlag, 1971. Graduate Texts in Mathematics, Vol. 5.
  33. Infer.NET 0.3, 2018. Microsoft Research Cambridge.
  34. Moggi, E. Computational lambda-calculus and monads. In Proceedings of the Fourth Annual Symposium on Logic in Computer Science (1989), IEEE Press, p. 14–23.
  35. Moggi, E. Notions of computation and monads. Information and Computation 93, 1 (1991), 55 – 92. Selections from 1989 IEEE Symposium on Logic in Computer Science.
  36. Delayed sampling and automatic Rao-Blackwellization of probabilistic programs. In Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics (2018), pp. 1037–1046.
  37. Applications of a disintegration transformation. In Workshop on program transformations for machine learning (2019).
  38. Power, J. Generic models for computational effects. Theor. Comput. Sci. 364, 2 (2006), 254–269.
  39. Premonoidal categories and notions of computation. Math. Struct. Comput. Sci. 7 (1997), 453–468.
  40. Expect the unexpected from conditional expectation. The American Statistician 52, 3 (1998).
  41. Selinger, P. A Survey of Graphical Languages for Monoidal Categories. Springer Berlin Heidelberg, Berlin, Heidelberg, 2011, pp. 289–355.
  42. Exact Bayesian inference by symbolic disintegration. In Proc. POPL 2017 (2017).
  43. St Clere Smithe, T. Bayesian updates compose optically, 2020.
  44. Staton, S. An algebraic presentation of predicate logic. In Foundations of Software Science and Computation Structures (Berlin, Heidelberg, 2013), F. Pfenning, Ed., Springer Berlin Heidelberg, pp. 401–417.
  45. Staton, S. Commutative semantics for probabilistic programming. In Programming Languages and Systems (Berlin, Heidelberg, 2017), H. Yang, Ed., Springer Berlin Heidelberg, pp. 855–879.
  46. The beta-bernoulli process and algebraic effects. Proceedings of 45th International Colloquium on Automata, Languages and Programming (ICALP ’18) (02 2018).
  47. Stein, D. GaussianInfer. https://github.com/damast93/GaussianInfer, 2021.
  48. Stein, D. Structural Foundations for Probabilistic Programming Languages. PhD thesis, University of Oxford, 2021.
  49. Compositional Semantics for Probabilistic Programs with Exact Conditioning. Association for Computing Machinery, New York, NY, USA, 2021.
  50. An introduction to probabilistic programming, 2018.
  51. From high-level inference algorithms to efficient code. Proc. ACM Program. Lang. 3, ICFP (July 2019).
  52. Z., Z. The Schur Complement and Its Applications. 01 2005.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now