Lecture notes: Probability with Measure

Abstract: Lecture notes as per the title. In the first part, the concepts of a measurable space, measurable maps between measurable spaces and that of a measure on a measurable space are introduced, after which the fundamentals of the theory of Lebesgue integration are developed: convergence theorems, product spaces and Tonelli-Fubini, indefinite integration and absolute continuity, L-spaces and integral inequalities. Everything is set up so that in the second part the fundamental concepts of probability (such as those of random elements and their laws, independence, conditioning) can be cast swiftly in the measure-theoretic setting. Some emphasis is placed on monotone class and Dynkin's lemma type arguments. Products of arbitrary families of probabilities and Kolmogorov's extension theorem are treated.

• The paper establishes the measure-theoretic basis for probability by rigorously defining sigma-algebras, measures, and their properties.
• It develops essential tools such as Carathéodory's extension theorem, Lebesgue integration, product measures, and convergence theorems.
• The framework supports advanced topics in stochastic processes, conditional expectations, and statistical inference, underpinning modern probabilistic models.

Measure-Theoretic Probability: A Rigorous Introduction

Scope, Motivation, and Style

The notes "Lecture notes: Probability with Measure" (2604.01685) provide a technically precise and comprehensive foundation for probability theory from the standpoint of measure theory. The material is formal, omitting motivational intuition and graphical exposition, instead focusing strictly on definitions, theorems, and proofs. The content is structured to serve practitioners seeking the minimal set of rigorous arguments necessary for advanced work in probability and related fields—stochastic processes, analysis, and mathematical statistics. The author deliberately restricts the scope to the essential machinery, relegating advanced topics and illustrative commentary to referenced literature.

Sigma-Algebras and Measure Spaces

The initial development centers on $\sigma$-algebras and their role in delineating measurable events, measurable functions, and the structure underpinning measures. Key properties, such as closure under complements and countable unions, are rigorously defined, with standard constructions (e.g., discrete/trivial $\sigma$-fields, countable–co-countable fields, $\sigma$-fields generated by partitions) detailed. The necessity of considering measures on $\sigma$-fields rather than arbitrary set classes is justified by balancing tractability with richness—enabling the deployment of countable additivity and ensuring the availability of nontrivial measures.

The treatment of measures as countably additive nonnegative set functions null at the empty set is canonical, encompassing finiteness, probability measures, and $\sigma$-finiteness. Examples (counting measure, Dirac measure, and Borel-Cantelli lemma) are included to illustrate the general applicability. Notably, the development emphasizes the naturality and consequences of countable additivity, with references to the philosophical debates on its primacy in probability theory.

Measurability, Generated Fields, and Structure

The construction of $\sigma$-fields via generators, especially through the pullback (initial structures) and pushforward (final structures) along measurable maps, is established. The formalism clarifies the relationship between measurability and information flow, ultimately underpinning crucial results in probability (e.g., modeling filtrations and information in stochastic processes). The stability of measurability under composition and restriction is proven and leveraged throughout.

A key result is the Doob-Dynkin factorization lemma, stating that any measurable function relative to the $\sigma$-field generated by another is almost surely a function of that generator. This property is foundational in both classical probability and modern areas such as machine learning (feature engineering, sufficient statistics).

Borel Sets, Borel Measurability, and Real-Valued Functions

The Borel $\sigma$-field on the extended real line and intervals is defined through explicit generators. The framework for Borel measurability of numerical functions is systematically constructed, with an emphasis on closure under algebraic operations and uniform limit processes. Central results, such as the measurability of supremum, infimum, limsup, and liminf of sequences of functions, are established, supporting later developments in integration and convergence theorems.

Monotone Classes, Dynkin Systems, and Uniqueness Theorems

The technical core includes the monotone class theorem and the formally rigorous $\pi$-$\lambda$ theorem. These instruments are fundamental for extending properties (established on generating classes) to full $\sigma$0-fields, notably in proving the uniqueness of measures agreeing on generating $\sigma$1-systems and for verifying equality between measures constructed via different methods.

Extension Theorems and Construction of Measures

Carathéodory's extension theorem is presented in detail, providing the bedrock for extending premeasures (countably additive on algebras) to full measures on generated $\sigma$2-fields. The development includes detailed proofs, the Tsirelson compactness criterion for countable additivity, and the explicit construction of Lebesgue-Stieltjes measures, including the Lebesgue measure as a special case. The paper also highlights pathological cases (e.g., non-extension to $\sigma$3), relying on set-theoretic considerations (axiom of choice, Vitali sets).

Integration, the Lebesgue Integral, and Convergence Theorems

The machinery of the Lebesgue integral for real-valued measurable functions is built from simple nonnegative functions, with care taken to address issues of well-definedness ($\sigma$4 conventions, "morally" measurable functions via completions). The integral is characterized by monotone approximation and proven to obey additivity, homogeneity, linearity, monotonicity, and dominated convergence properties.

Strong convergence theorems (Fatou, monotone, dominated convergence) are derived with full technical rigor. These results form the analytical and probabilistic backbone for further developments in stochastic analysis, ergodic theory, and mathematical statistics.

Product Measures, Fubini-Tonelli, and Disintegration

The product measure construction via $\sigma$5-finiteness leads directly to Fubini and Tonelli's theorems, enabling the interchange of iterated integration. The text addresses subtle issues arising from nonunique completions and the failure of completeness under product measures. The disintegration of measures and pushforward-pullback constructions are introduced with explicit formal properties, critical in conditional probability, Markov processes, and Bayesian inference.

Absolute Continuity, Radon-Nikodym Theorem, and Change of Variables

Measures are systematically compared via absolute continuity, with the Radon-Nikodym theorem proved in full generality (under $\sigma$6-finiteness). Explicit formulas for densities and changes of variables, including multi-dimensional Jacobian transformations, are provided, clarifying the measure-theoretic underpinnings of probability densities and law transformations.

$\sigma$7-Spaces, Inequalities, and Functional Analysis

The $\sigma$8-spaces are constructed, with norms, completeness, and density of simple functions precisely established. Classical inequalities (Markov, Jensen, Hölder, Minkowski, Cauchy-Schwarz) are proved, highlighting their measure-theoretic origins and their role in controlling integrability and convergence.

Probability as Normalized Measure: Random Elements and Independence

Probability spaces are cast as normalized measures, linking random variables, vectors, and elements to the measure-theoretic framework. The law of an element is identified with the pushforward measure, and explicit connections are drawn between distribution functions, densities, and expectations. Quantile representations and coupling arguments are formalized (e.g., Skorokhod representation, inverse transform samples).

Independence is given a rigorous measure-theoretic definition via independent $\sigma$9-fields and collections of random elements. The Kolmogorov extension theorem is proved for projective families, enabling the construction of infinite product spaces—foundational for stochastic process theory (Brownian motion, Markov chains, infinite coin tosses).

Conditional Expectation and Regular Conditional Probabilities

Conditional expectation is defined as an $\sigma$0-projection onto a sub-$\sigma$1-field, realized as a Radon-Nikodym derivative. Concrete computation methods (discrete partitions, absolutely continuous cases, independence, image measures) are provided, together with key properties (tower property, linearity, "taking out what is known," monotonicity, Jensen's inequality for conditional expectation).

The existence and uniqueness (up to a.s.~equality) of regular conditional probabilities are established for Borel spaces, supporting both the theoretical understanding and practical implementation of stochastic conditioning and Bayesian inference.

Implications and Directions

The notes provide an indispensable technical foundation supporting modern probability theory, stochastic analysis, ergodic theory, and measure-valued statistical inference. The measure-theoretic methods underpin advanced work in stochastic processes, martingale theory, and the probabilistic analysis of algorithms. The explicit handling of technical subtleties—including issues of extension, completion, regularity, and nonmeasurable sets—ensures rigor in contexts where intuition may fail.

Future work and theoretical advances in AI—especially those leveraging stochastic processes, Bayesian learning, and the rigorous analysis of randomized procedures—will continue to require facility with these constructions. Developments such as probabilistic programming, measure-theoretic deep learning, and the formal verification of probabilistic algorithms fundamentally rely on this paradigm.

Conclusion

"Lecture notes: Probability with Measure" (2604.01685) systematically constructs the measure-theoretic scaffolding beneath probability theory, emphasizing minimalism, rigor, and directness. The treatment facilitates both foundational understanding and precision in application, offering clear pathways to advanced probabilistic modeling and analysis. The notes form a precise launching point for graduate coursework, research in mathematical statistics and probability, and foundational studies in machine learning and stochastic systems.