Nonparametric Bayesian Policy Learning

• Nonparametric Bayesian Policy Learning (NBPL) is a framework that places a Dirichlet process prior on a reduced-form distribution to infer optimal treatment assignments and welfare outcomes.
• The approach leverages the Bayesian bootstrap for tractable posterior computation, linking empirical welfare maximization with Bayesian decision theory.
• NBPL demonstrates practical relevance by comparing policy classes in applications such as Job Training and bednet subsidy experiments under uncertainty.

Nonparametric Bayesian Policy Learning (NBPL) is a Bayesian nonparametric framework for policy learning in which a decision-maker places a Dirichlet process prior on a reduced-form distribution that determines welfare-relevant policy objects, and then uses the induced posterior to conduct inference on optimal treatment assignments, optimal welfare, welfare regret, and comparisons across policy classes. In the formulation introduced under that name, NBPL is explicitly designed for uncertainty-aware treatment choice: for a fixed welfare criterion and feasible policy class, all decision-relevant uncertainty is taken to be induced by uncertainty about a reduced-form distribution, and posterior computation is implemented tractably via the Bayesian bootstrap (Ye, 16 May 2026).

1. Decision-theoretic formulation

The canonical NBPL setup studies a decision-maker who assigns a binary treatment using observed covariates to maximize welfare in a target population. Potential outcomes and covariates satisfy

$(Y_i(1),Y_i(0),X_i)\sim P_0^\star,$

with $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$ and $X_i\in\mathcal X\subseteq \mathbb R^{d_x}$. A policy is represented by a measurable set $G\subseteq\mathcal X$: individuals with $X\in G$ are treated, and others are not. Feasible policies belong to a class $\mathcal G\subseteq 2^{\mathcal X}$ (Ye, 16 May 2026).

With utilitarian welfare, the policy value is

$W(P_0^\star;G)\coloneqq E_{P_0^\star}\big[Y(1)\mathds 1\{X\in G\}+Y(0)\mathds 1\{X\notin G\}\big].$

The framework then normalizes welfare relative to assigning no treatment, yielding

$W(P_0^\star;G)=E_{P_0^\star}\big[(Y(1)-Y(0))\mathds 1\{X\in G\}\big].$

The policy problem is therefore to choose $G\in\mathcal G$ to maximize expected welfare gains.

Observed data are

$$\mathcal D_n=\{D_i\}_{i=1}^n,\qquad D_i=(Y_i,T_i,X_i),$$

with $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$0 the realized treatment and

$Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$1

Let $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$2 denote the joint distribution of $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$3, and define the propensity score

$Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$4

Identification of welfare relies on four assumptions: external validity, unconfoundedness, outcome moments, and strict overlap, together with known $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$5. Under these assumptions, welfare is point identified through inverse-probability weighting: $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$6

A central move in NBPL is to define the reduced-form vector

$Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$7

and let $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$8 denote the distribution of $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$9. Welfare then becomes a deterministic functional of $X_i\in\mathcal X\subseteq \mathbb R^{d_x}$0. This reduced-form orientation is fundamental: NBPL does not place its prior on a full structural model of potential outcomes, but on the reduced-form distribution that suffices for the welfare criterion under study.

2. Bayesian nonparametric construction

For a fixed policy class $X_i\in\mathcal X\subseteq \mathbb R^{d_x}$1, NBPL targets several posterior objects. The set of optimal rules is

$X_i\in\mathcal X\subseteq \mathbb R^{d_x}$2

and the corresponding optimal welfare is

$X_i\in\mathcal X\subseteq \mathbb R^{d_x}$3

The framework also studies welfare regret, defined as the loss from using the rule optimal under $X_i\in\mathcal X\subseteq \mathbb R^{d_x}$4 when the true distribution is $X_i\in\mathcal X\subseteq \mathbb R^{d_x}$5: $X_i\in\mathcal X\subseteq \mathbb R^{d_x}$6

NBPL places a Dirichlet process prior on the reduced-form distribution,

$X_i\in\mathcal X\subseteq \mathbb R^{d_x}$7

where $X_i\in\mathcal X\subseteq \mathbb R^{d_x}$8 is a finite base measure. The Dirichlet process is defined in the usual way: for every finite measurable partition $X_i\in\mathcal X\subseteq \mathbb R^{d_x}$9,

$G\subseteq\mathcal X$0

The total mass $G\subseteq\mathcal X$1 governs concentration, while the normalized base measure acts as a base distribution. When $G\subseteq\mathcal X$2 has full support on the reduced-form sample space, the prior has large weak support (Ye, 16 May 2026).

Posterior inference proceeds by pushforward from the posterior on $G\subseteq\mathcal X$3. The posterior on optimal rules is

$G\subseteq\mathcal X$4

and the posterior on optimal welfare is

$G\subseteq\mathcal X$5

For two policy classes $G\subseteq\mathcal X$6, the same posterior induces a welfare-comparison probability,

$G\subseteq\mathcal X$7

which the framework interprets as posterior evidence that $G\subseteq\mathcal X$8 can achieve higher optimal welfare.

Conjugacy is one of the principal computational advantages. If $G\subseteq\mathcal X$9, then

$X\in G$0

where

$X\in G$1

is the empirical measure. This posterior underwrites both the theory and the Bayesian-bootstrap implementation.

3. Computation via the Bayesian bootstrap

The recommended implementation uses the Bayesian bootstrap, corresponding to the posterior $X\in G$2. Its key representation is

$X\in G$3

where

$X\in G$4

This reduces posterior computation to random reweighting of the observed sample (Ye, 16 May 2026).

The practical NBPL procedure is therefore a sequence of weighted welfare-maximization problems. For posterior draw $X\in G$5, one samples exponential weights

$X\in G$6

One then computes

$X\in G$7

followed by posterior-draw summaries such as

$X\in G$8

and treatment share

$X\in G$9

Posterior summaries are then formed empirically from $\mathcal G\subseteq 2^{\mathcal X}$0. Equal-tailed credible intervals come from empirical quantiles; posterior treatment-share summaries come from the empirical distribution of $\mathcal G\subseteq 2^{\mathcal X}$1; and posterior class comparison is estimated by

$\mathcal G\subseteq 2^{\mathcal X}$2

The framework emphasizes tractability: computation reuses existing welfare-maximization solvers, replacing deterministic sample weights $\mathcal G\subseteq 2^{\mathcal X}$3 with random Bayesian-bootstrap weights. This suggests a close operational relationship between NBPL and empirical welfare maximization, even though the inferential objects differ.

4. Theoretical properties

The theory centers on finite policy-class complexity, summarized by Assumption VC, which requires the policy class $\mathcal G\subseteq 2^{\mathcal X}$4 to have finite VC dimension $\mathcal G\subseteq 2^{\mathcal X}$5. This covers, among other examples mentioned in the paper, linear rules and shallow decision trees (Ye, 16 May 2026).

A key discrepancy measure is

$\mathcal G\subseteq 2^{\mathcal X}$6

The regret decomposition

$\mathcal G\subseteq 2^{\mathcal X}$7

separates sampling error from posterior reweighting error.

The first headline result is posterior regret contraction at the minimax-optimal rate: $\mathcal G\subseteq 2^{\mathcal X}$8 for any sequence $\mathcal G\subseteq 2^{\mathcal X}$9 arbitrarily slowly. The paper states that this matches the minimax-optimal regret rate for empirical welfare maximization in Kitagawa and Tetenov (2018), implying that NBPL does not lose first-order learning efficiency relative to that frequentist benchmark. For growing classes $W(P_0^\star;G)\coloneqq E_{P_0^\star}\big[Y(1)\mathds 1\{X\in G\}+Y(0)\mathds 1\{X\notin G\}\big].$0 with

$W(P_0^\star;G)\coloneqq E_{P_0^\star}\big[Y(1)\mathds 1\{X\in G\}+Y(0)\mathds 1\{X\notin G\}\big].$1

the corresponding result is

$W(P_0^\star;G)\coloneqq E_{P_0^\star}\big[Y(1)\mathds 1\{X\in G\}+Y(0)\mathds 1\{X\notin G\}\big].$2

The second headline result is pointwise consistency of posterior class comparison. For two policy classes $W(P_0^\star;G)\coloneqq E_{P_0^\star}\big[Y(1)\mathds 1\{X\in G\}+Y(0)\mathds 1\{X\notin G\}\big].$3, if

$W(P_0^\star;G)\coloneqq E_{P_0^\star}\big[Y(1)\mathds 1\{X\in G\}+Y(0)\mathds 1\{X\notin G\}\big].$4

then

$W(P_0^\star;G)\coloneqq E_{P_0^\star}\big[Y(1)\mathds 1\{X\in G\}+Y(0)\mathds 1\{X\notin G\}\big].$5

If the classes are tied at the truth,

$W(P_0^\star;G)\coloneqq E_{P_0^\star}\big[Y(1)\mathds 1\{X\in G\}+Y(0)\mathds 1\{X\notin G\}\big].$6

then for any $W(P_0^\star;G)\coloneqq E_{P_0^\star}\big[Y(1)\mathds 1\{X\in G\}+Y(0)\mathds 1\{X\notin G\}\big].$7,

$W(P_0^\star;G)\coloneqq E_{P_0^\star}\big[Y(1)\mathds 1\{X\in G\}+Y(0)\mathds 1\{X\notin G\}\big].$8

The paper also proves a proposition linking empirical welfare maximization (EWM) and Bayes decision theory. Under the Bayesian-bootstrap posterior $W(P_0^\star;G)\coloneqq E_{P_0^\star}\big[Y(1)\mathds 1\{X\in G\}+Y(0)\mathds 1\{X\notin G\}\big].$9, the Bayes rule minimizing posterior expected welfare regret coincides with the empirical welfare maximizer. This gives EWM a Bayesian interpretation as the rule that averages welfare over posterior uncertainty and then optimizes, whereas NBPL retains the full posterior over optimal assignments rather than collapsing immediately to a single rule. The paper presents this as one of its main novelties and, specifically, as the first posterior welfare-regret guarantee for policy learning.

5. Identification, uncertainty, and policy-class comparison

NBPL sharply distinguishes identified scalar welfare targets from possibly set-valued policy targets. Under external validity, unconfoundedness, overlap, and known propensity score, the welfare of any fixed rule $W(P_0^\star;G)=E_{P_0^\star}\big[(Y(1)-Y(0))\mathds 1\{X\in G\}\big].$0 is point identified by equation $W(P_0^\star;G)=E_{P_0^\star}\big[(Y(1)-Y(0))\mathds 1\{X\in G\}\big].$1. Consequently, for each fixed policy class $W(P_0^\star;G)=E_{P_0^\star}\big[(Y(1)-Y(0))\mathds 1\{X\in G\}\big].$2, the optimal welfare value

$W(P_0^\star;G)=E_{P_0^\star}\big[(Y(1)-Y(0))\mathds 1\{X\in G\}\big].$3

is point identified (Ye, 16 May 2026).

By contrast, the optimal rule itself may fail to be unique: $W(P_0^\star;G)=E_{P_0^\star}\big[(Y(1)-Y(0))\mathds 1\{X\in G\}\big].$4 The optimal-rule object is therefore naturally set-valued. The framework explicitly notes that this gives the rule component an affinity with partial-identification problems. Posterior inference on $W(P_0^\star;G)=E_{P_0^\star}\big[(Y(1)-Y(0))\mathds 1\{X\in G\}\big].$5 is then inference on an identified set of optimal rules rather than on a unique decision rule.

This distinction matters for the interpretation of posterior uncertainty. In NBPL, posterior uncertainty is statistical uncertainty about the reduced-form distribution $W(P_0^\star;G)=E_{P_0^\star}\big[(Y(1)-Y(0))\mathds 1\{X\in G\}\big].$6, and therefore about treatment assignments, optimal welfare, regret, and class rankings induced by $W(P_0^\star;G)=E_{P_0^\star}\big[(Y(1)-Y(0))\mathds 1\{X\in G\}\big].$7. It is not a posterior over a full structural causal model for $W(P_0^\star;G)=E_{P_0^\star}\big[(Y(1)-Y(0))\mathds 1\{X\in G\}\big].$8, nor does it directly address ambiguity arising from unsupported identifying assumptions. The paper is explicit that the framework handles statistical uncertainty rather than broader ambiguity or partial identification beyond the nonuniqueness of optimal rules.

Policy-class comparison is one of the framework’s most distinctive inferential objects. The decision-relevant comparison is not model fit but achievable welfare: $W(P_0^\star;G)=E_{P_0^\star}\big[(Y(1)-Y(0))\mathds 1\{X\in G\}\big].$9 This comparison remains meaningful for non-nested classes and can assign high posterior probability to one class even when the resulting optimal rules differ sharply only on small regions of the covariate space. The criterion is therefore explicitly welfare-based rather than representation-based.

6. Empirical applications

NBPL is illustrated in two randomized experiments: the Job Training Partnership Act (JTPA) study and the bednet subsidy experiment of Bhattacharya and Dupas. In both cases the framework compares linear rules with depth-2 decision trees and reports posterior summaries for optimal welfare, treatment share, and posterior class dominance (Ye, 16 May 2026).

Application	Setup	Main posterior findings
JTPA	$G\in\mathcal G$0, $G\in\mathcal G$1, $G\in\mathcal G$2	Tree class dominates linear with posterior probability $G\in\mathcal G$3 without treatment cost and $G\in\mathcal G$4 with $G\in\mathcal G$5 cost
Bednet subsidy	$G\in\mathcal G$6, $G\in\mathcal G$7, $G\in\mathcal G$8	Tree class outperforms linear with posterior probability $G\in\mathcal G$9, with and without a 70% treatment-capacity constraint

In the JTPA application, treatment is program eligibility and outcome is earnings 30 months after assignment. Without treatment cost, the posterior distribution of $$\mathcal D_n=\{D_i\}_{i=1}^n,\qquad D_i=(Y_i,T_i,X_i),$$0 first-order stochastically dominates that of $$\mathcal D_n=\{D_i\}_{i=1}^n,\qquad D_i=(Y_i,T_i,X_i),$$1. Appendix summaries report NBPL medians with 95% credible intervals: without treatment cost, linear rules yield treatment share $$\mathcal D_n=\{D_i\}_{i=1}^n,\qquad D_i=(Y_i,T_i,X_i),$$2 and welfare gain $$\mathcal D_n=\{D_i\}_{i=1}^n,\qquad D_i=(Y_i,T_i,X_i),$$3 $$\mathcal D_n=\{D_i\}_{i=1}^n,\qquad D_i=(Y_i,T_i,X_i),$$4. With $$\mathcal D_n=\{D_i\}_{i=1}^n,\qquad D_i=(Y_i,T_i,X_i),$$5 $$\mathcal D_n=\{D_i\}_{i=1}^n,\qquad D_i=(Y_i,T_i,X_i),$$6 for linear rules, versus treatment share $$\mathcal D_n=\{D_i\}_{i=1}^n,\qquad D_i=(Y_i,T_i,X_i),$$7 and welfare gain $$\mathcal D_n=\{D_i\}_{i=1}^n,\qquad D_i=(Y_i,T_i,X_i),$$8 for trees. The paper also reports that EWM welfare estimates tend to lie at relatively low posterior quantiles of the NBPL welfare posterior, that NBPL credible intervals are typically tighter than corresponding EWM bootstrap confidence intervals, and that NBPL tends to choose lower treatment shares than EWM.

In the bednet application, treatment is the offer of a highly subsidized insecticide-treated bednet and the outcome is bednet coverage or usage. Without a capacity constraint, both classes treat almost everyone and welfare differences are small. Appendix values are treatment share $$\mathcal D_n=\{D_i\}_{i=1}^n,\qquad D_i=(Y_i,T_i,X_i),$$9, welfare $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$00, CI $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$01 for linear rules, and treatment share $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$02, welfare $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$03, CI $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$04 for trees. Under a 70% treatment-capacity constraint, treatment shares become $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$05 for both classes, while welfare is $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$06, CI $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$07 for linear rules and $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$08, CI $Y_i(1),Y_i(0)\in\mathcal Y\subseteq \mathbb R$09 for trees. The paper interprets these results as evidence that more adaptive classes become more valuable when policy constraints bind.

These applications are also substantively important because they sit in settings where randomization makes the identifying assumptions especially credible: treatment assignment is randomized, and propensity scores are known by design. That feature aligns closely with the reduced-form emphasis of NBPL.

7. Scope, adjacent formulations, and limitations

In its 2026 formulation, NBPL is specifically a reduced-form Dirichlet-process framework for treatment choice under uncertainty (Ye, 16 May 2026). Adjacent work shows that closely related Bayesian nonparametric policy-learning ideas appear in several other architectures, but these should not be conflated with the reduced-form NBPL construction.

One adjacent line studies dynamic treatment regimes by combining a Dirichlet process mixture model for longitudinal dynamics with policy-search algorithms over an interpretable constrained regime class; this is model-based, simulation-driven policy search rather than reduced-form posterior inference on optimal assignments (Guan et al., 2018). A second line studies policy recognition and state abstraction from demonstrations, using CRP and ddCRP priors to infer the number of local controllers and task-appropriate partitions of the state space (Šošić et al., 2016). A third line uses an Indian Buffet Process prior to infer an unknown number of latent features, with one categorical policy component per feature, in Bayesian nonparametric imitation learning (Hahn et al., 2017). In decentralized partially observable reinforcement learning, stick-breaking priors over finite-state-controller nodes provide a Bayesian nonparametric treatment of policy memory complexity in Dec-POMDPs (Shih, 2021, Shih et al., 2021). These formulations are all recognizably Bayesian and nonparametric, but they place their priors on different objects: dynamics, state partitions, latent features, or controller memory states.

Two further papers are especially relevant as methodological neighbors. "Nonparametric Uniform Inference in Binary Classification and Policy Values" develops a nonparametric frequentist framework based on a strictly convex surrogate loss, with Gaussian inference for both the learned policy and the optimal policy value; it provides a benchmark for nonparametric policy-value inference rather than a Bayesian posterior construction (Liu et al., 18 Nov 2025). "General Bayesian Policy Learning" proposes a loss-based generalized posterior over policies using a squared-loss surrogate and a working Gaussian pseudo-likelihood interpretation; it is directly policy-targeted and highly compatible with flexible function classes, but it is not, in the strict classical sense, a Dirichlet-process or Gaussian-process nonparametric Bayes formulation (Kato, 27 Feb 2026).

Several limitations of NBPL are explicit. The main theory and applications assume known propensity scores, external validity, unconfoundedness, and strict overlap. The regret analysis relies on finite VC dimension. The Bayesian interpretation is intentionally reduced-form and limited-information: it does not produce a full structural posterior over potential outcomes. The paper identifies unknown propensity scores and the integration of statistical uncertainty with ambiguity/partial identification as important extensions (Ye, 16 May 2026).

A further source of confusion is bibliographic rather than conceptual. A 2020 arXiv entry titled "Batch Reinforcement Learning with a Nonparametric Off-Policy Policy Gradient" does not provide relevant technical content for NBPL in the supplied text; it is described there as an IEEEtran template with no reinforcement-learning or Bayesian policy-learning method (Tosatto et al., 2020). This underscores a broader terminological point: NBPL, as a named framework, refers specifically to the reduced-form Dirichlet-process approach of (Ye, 16 May 2026), while the broader phrase can also evoke neighboring Bayesian nonparametric approaches to policy learning in imitation learning, causal policy learning, and partially observable reinforcement learning.