Expressive power loss of the LH-safe PPL fragment

Determine the loss of expressive power incurred by restricting the core probabilistic programming language to the LH-safe fragment L that bans the score construct, forbids sums of densities for distributions, and enforces single-use observation of each data variable; specifically, establish which probabilistic models expressible in the unrestricted core language become inexpressible in this LH-safe fragment and characterize the extent of this expressiveness gap.

Background

The paper introduces a safe probabilistic programming language fragment (denoted L) that prevents likelihood hacking by enforcing three syntactic constraints: each data variable must be observed exactly once (and only as an argument to observe), the score construct is forbidden, and sums of densities are disallowed. This fragment is shown to be LH-safe in the sense that any program written in it induces a properly normalized joint over data and latent variables.

While the authors demonstrate empirical coverage (e.g., many Stan models comply) and provide soundness guarantees for LH-safety, they explicitly note that the impact of these constraints on expressiveness has not been formally analyzed. Understanding the expressiveness gap would clarify which classes of models are excluded by the safety conditions and inform future language design that balances safety with modeling flexibility.