Prescriptive Neural Networks Overview

• Prescriptive Neural Networks (PNNs) are frameworks that directly optimize treatment assignments by integrating causal estimation with neural network outputs.
• They employ diverse architectures—from shallow 0–1 networks using MILP to deep, multimodal models—yielding competitive performance against traditional decision trees.
• PNNs balance predictive accuracy and interpretability via rule extraction and fairness-oriented feature selection, enabling personalized interventions in healthcare and policy-making.

Prescriptive Neural Networks (PNNs) are a class of neural-based frameworks designed to optimize treatment assignment or intervention decisions by directly modeling the prescription policy, rather than solely predicting outcomes. PNNs aim to maximize the utility of prescribed actions in contexts, such as personalized medicine or policy-making, where optimal decisions depend on both individual-level characteristics and estimated counterfactual outcomes under various interventions. Emerging in recent years, PNNs encompass a spectrum from shallow, interpretable Boolean architectures trained by mixed-integer programming to expressive deep multimodal networks, unified by their explicit policy optimization objective and direct incorporation of causal estimation. PNNs have demonstrated superior or competitive performance compared to tree-based and classical prescriptive models across synthetic and real-world tabular, image, and text-rich datasets (Patil et al., 2024, Sun et al., 2023, Bertsimas et al., 24 Jan 2025).

1. Core Principles and Architectures

PNNs are fundamentally prescription-centric, outputting for each input feature vector $x$ either a treatment assignment in the discrete set $\mathcal T$ or a probability distribution over $\mathcal T$ (via softmax), trained to optimize an expected counterfactual outcome or utility. Key PNN variants include:

• 0–1 PNN: Shallow networks (L=2 layers, typically 1 hidden) with binary activations (0–1) in the hidden layer and a linear output, with all activation decisions and prescription assignments encoded as integer variables. Such networks employ mixed-integer linear programming (MILP) to directly maximize average causal utility estimated via doubly robust (DR) counterfactual estimators. Each hidden neuron represents a linear half-space in input space (Patil et al., 2024).
• P-ReLU (Prescriptive ReLU Network): Piecewise linear deep networks with ReLU activations; for each $x$, outputs predictions for all $K=|\mathcal T|$ treatments and prescribes the argmin. The architecture partitions input space into convex polyhedra with homogeneous prescriptions. Trained with a unified loss consisting of a prediction error and an approximate prescriptive regret term (Sun et al., 2023).
• Multimodal PNN: Deep feedforward networks leveraging embeddings of tabular, text, and/or image data, with feature fusion and prescription layers (softmax over $\mathcal T$). The input is a concatenation of preprocessed/embedded modalities, and the loss optimizes a differentiable surrogate of expected reward based on estimated counterfactuals (Bertsimas et al., 24 Jan 2025).

2. Mathematical Formalization and Policy Optimization

The central optimization in PNNs is the prescription objective. For observed data $\{(x_i, t_i, y_i)\}_{i=1}^n$, PNNs estimate a reward (or utility) matrix $\Gamma_{i, t}$ for each potential assignment. In classical 0–1 PNNs, the MILP maximizes average doubly robust outcomes:

$$\max_{\alpha, \beta, h, z}~\frac{1}{n}\sum_{i=1}^n\sum_{t\in T}h_{i,t,2}\,\hat\psi_t(x_i,t_i,y_i)$$

where $h_{i,t,2}$ is a binary indicator for assignment and $\mathcal T$0 is the DR estimator:

$\mathcal T$1

Policy constraints are imposed through integrality and big-M linearization for bilinear terms, one-hot assignment constraints, and explicit regularization (e.g., $\mathcal T$2 for sparsity).

In deep PNNs, the differentiable surrogate leverages softmax outputs:

$\mathcal T$3

where $\mathcal T$4 denotes the softmax assignment probability, $\mathcal T$5 counterfactual reward (e.g., via DR estimator or predicted potential outcome), and $\mathcal T$6 controls weight decay (Bertsimas et al., 24 Jan 2025).

3. Counterfactual Estimation and Integration

PNNs require accurate estimation of counterfactual outcomes for each candidate action. Standard approaches include:

• Direct regression: Fit $\mathcal T$7 for each $\mathcal T$8.
• Doubly robust (DR) estimation: Combine a propensity model $\mathcal T$9 and regression $\mathcal T$0 in

$\mathcal T$1

• Surrogate reward matrices: Constructed for all treatments per individual; for continuous interventions, discretize the space and fit outcome regressions.

Counterfactual integrations are fundamental to PNN training objectives, ensuring the policy is optimized for expected real-world utility under unobserved assignments (Patil et al., 2024, Bertsimas et al., 24 Jan 2025).

4. Interpretability and Rule Extraction

PNNs address the interpretability–performance tradeoff through both architectural design and post-hoc distillation:

• 0–1 PNN: The logical structure induced by binary activations maps each hidden neuron to a linear half-space, and the treatment assignment is a function of logical combinations of these half-spaces. This enables extraction of rule sets or Boolean expressions mapping features to prescriptions. Feature importances can be assessed via absolute weight magnitudes or SHAP applied to the optimized MIP solution. These importances have demonstrated greater stability than deep nets and improved clinical plausibility over tree-based competitors (Patil et al., 2024).
• P-ReLU: Any configuration of ReLU activations defines a unique convex polyhedron in input space where prescription is fixed; the entire network is equivalent to an oblique decision tree, whose splits correspond to hidden-layer hyperplanes and leaves to treatment assignments. Trained sparse networks can be exactly converted to small, interpretable trees without loss of prescription accuracy (Sun et al., 2023).
• Deep PNNs with Knowledge Distillation: Interpretability is recovered by fitting an Optimal Classification Tree (OCT) to match PNN prescriptions, yielding small decision trees that mirror the network’s policy, with less than 1.4 pp drop in improvement across multiple datasets (Bertsimas et al., 24 Jan 2025).

5. Multimodal Data Fusion

The recent extension of PNNs to multimodal data (Bertsimas et al., 24 Jan 2025) incorporates diverse sources such as tabular records, free-text clinical notes, and medical images by:

• Preprocessing each modality independently (scaling, one-hot, embedding).
• Extracting embeddings from pretrained large models (e.g., ClinicalLongformer for text, CNNs for images).
• Reducing high-dimensional features (e.g., PCA to 32D) and concatenating to form a composite feature vector.
• Feeding the fused embedding into the PNN for prescription optimization.

This framework enables robust prescriptive modeling in environments where actionable information is distributed across multiple heterogeneous data sources, such as electronic health records with structured labs and imaging.

6. Empirical Assessment and Comparative Performance

Empirical results highlight PNN efficacy across synthetic and real-world scenarios:

• Synthetic (tabular, discrete actions): 0–1 PNNs match or exceed tree/forest-based benchmarks (e.g., J-PT, B-PT, K-PT, causal forest) in out-of-sample probability of correct treatment (OOSP) over a wide range of covariate dimensions and signal regimes (e.g., PNN ∼91% correct at $\mathcal T$2, $\mathcal T$3; causal forest and J-PT ∼91%, B-PT/K-PT often <90%) (Patil et al., 2024). P-ReLU networks outperform axis-aligned and oblique trees, especially as the number of treatments increases (Sun et al., 2023).
• Healthcare (clinical outcomes):
  • 0–1 PNNs reduce peak blood pressure by 5.47 mm Hg ($\mathcal T$4) over existing practice in postpartum hypertension, and by 2 mm Hg ($\mathcal T$5) over the next best prescriptive model; 95% CI for mean SBP with PNN: [143.65, 147.12] mm Hg versus [146.59, 150.06] for practice (Patil et al., 2024).
  • Multimodal PNNs in TAVR procedures achieve a 32% reduction in estimated postoperative complication rates and a 40% reduction in estimated mortality rates for liver trauma injuries (Bertsimas et al., 24 Jan 2025).
• Additional domains: In tasks such as diabetes management, grocery pricing, splenic injury care, and trauma intervention, PNNs consistently outperform or match state-of-the-art baselines (regress-and-compare, causal forest, optimal policy tree), with mirrored trees closely tracking PNN performance (Bertsimas et al., 24 Jan 2025).
• Stability and realism: Prescription assignments remain stable across randomized splits (σ ≈ 0.10–0.45), prescription realism (mean absolute change from historical treatment 0.31–0.65) is on par with classical methods (Bertsimas et al., 24 Jan 2025).

7. Fairness and Feature Selection

PNNs address potential biases and encourage meaningful feature selection by:

• Formulating policies that rarely depend on protected or spurious covariates such as insurance status or race. 0–1 PNNs, aided by $\mathcal T$6 regularization in MILP, identify clinically plausible drivers (prenatal BMI, chronic/gestational hypertension, mode of delivery, gestational age, pre-eclampsia), in contrast to personalized trees or causal forests that sometimes select social variables (Patil et al., 2024).
• Knowledge-distilled trees retain these fairness properties, providing transparent certification of the features used in prescription.

A plausible implication is that the explicit regularization and constraint framework in PNNs facilitates the design of fair and clinically interpretable prescription policies, supporting their adoption in settings with regulatory and ethical requirements.

---

References:

(Patil et al., 2024): "Applications of 0-1 Neural Networks in Prescription and Prediction" (Sun et al., 2023): "Learning Prescriptive ReLU Networks" (Bertsimas et al., 24 Jan 2025): "Multimodal Prescriptive Deep Learning"