---
title: Decision Trees under Partial Observability
url: https://www.emergentmind.com/topics/decision-trees-under-po
type: topic
---

# Decision Trees under Partial Observability

Decision trees under partial observability (PO) refer to tree-based models for sequential decision making, prediction, or control where the agent does not have full access to the true underlying state but only to some observation or feature set. The PO setting fundamentally alters both the representation of state, the policy class, and the computational procedures for optimal or interpretable tree construction. Research has developed formal frameworks for PO in statistical learning, sequential decision processes, and interpretable model extraction. This article surveys structural foundations, key algorithmic paradigms, formal complexity results, and practical strategies for learning and interpreting decision trees under partial observability, with special attention to recent advances.

## 1. Formal Foundations: Observations, State Abstraction, and Policy Classes

Partial observability is defined by a mapping from the underlying state space $S$ to a set of observable features or variables $Obs$. In general, each observation $o \in Obs$ corresponds to an equivalence class of states $\omega^{-1}(o)$ that are indistinguishable given the observable features. A policy under PO is then a mapping $\sigma : Obs \to \mathcal{A}$, where $\mathcal{A}$ is the action set. 

Two major approaches to policy representation and deployment arise:

- **Tabular or table-based memoryless PO**: The policy is directly a mapping from $Obs$ to actions, ignoring unobservable state history altogether [2412.05476].
- **History-augmented PO**: The policy is represented as $\pi(a \mid z_t, h_t)$, where $z_t$ is the current observation and $h_t$ is an embedding of the past observation/action history, typically learned or constructed recursively [2203.08057].

Partial observability induces nontrivial equivalence classes among states, leading to the requirement of "consistency"—the set of enabled actions under $o$ must be identical across all underlying states with the same observation $o$ [2412.05476]. Strategies/model extraction methods must respect this property.

## 2. Methodologies for Strategy Construction under PO

Several algorithmic routes to decision-tree policies under PO have been formalized:

### 2.1 Sampling and Reinforcement Learning-Based Extraction

One approach [2412.05476] proceeds as follows:
- Use lightweight strategy sampling (LSS) or table-based Q-learning to learn a memoryless PO policy $\sigma : Obs \to \mathcal{A}$ in a Markov automata model.
- Extract the resulting mapping from simulation trajectories: collect all observed $(o, a)$ pairs and deduplicate.
- Input this tabular policy into a decision tree learner (e.g., dtControl tool) to obtain an axis-aligned tree that predicts $a$ from $o$ only.

The resulting tree is minimal with respect to observable features, highly compact, and interpretable. Empirical evidence on industrial-scale models (e.g., open-pit mining logistics) shows that trees extracted from PO policies are 1–2 orders of magnitude smaller than those from full observability (FO) strategies, rapidly capturing core domain rules [2412.05476].

### 2.2 Differentiable and Adaptive Tree Architectures

The POETREE framework [2203.08057] introduces fully-differentiable soft decision trees optimized via imitation learning in PO Markov Decision Processes (POMDPs). A decision policy $\pi(a \mid z_t, h_t)$ is modeled by a binary probabilistic tree:
- Every inner node computes a soft gate (logistic function) over the observation or observation-history embedding, allowing differentiable routing.
- Each leaf associates a softmax over actions, parameterizing a (possibly stochastic) policy.
- The tree is grown adaptively via validation-based splitting and pruned for parsimony. A history embedding $h_t$ is updated in the leaves via a learned recurrence. Parameters are tuned by backpropagation to match observed expert behavior.
- This architecture bridges PO constraints with interpretability: the final tree is a transparent, evolving policy depending on observations and encoded history.

## 3. Policy Extraction, Tree Representation, and Interpretability under PO

PO introduces essential representational and interpretability advantages and challenges:

- **Feature Selection and Model Size**: Restricting trees to observable features yields dramatically more compact, interpretable decision rules. Under FO, irrelevant or unobservable variables needlessly increase tree width and depth [2412.05476].
- **Axis-Aligned Trees via dtControl and Related Tools**: Extracted trees operate only on observable features, facilitating rule-based policy diagrams like "if full=true then to_dump_1 else towards_shovel_0". Domain experts can directly validate or deploy these rules.
- **Differentiability and Policy Flexibility**: Fully-differentiable trees allow continuous optimization and automatic architecture selection under imitation constraints in POETREE [2203.08057].

In medical decision-support applications under PO, POETREE achieves superior performance and interpretability metrics compared to recurrent neural network (RNN)-based baselines and competitive methods, with clinician-rated clarity scores approaching that of classical decision trees [2203.08057].

## 4. Theoretical Properties: Consistency, Complexity, and Limitations

PO trees and associated extraction methods admit the following structural and computational guarantees:

- **Policy Consistency**: All extracted trees must respect action-set consistency within each observable equivalence class. The strategy sampling and Q-learning algorithms enforce this at the data collection and mapping stages [2412.05476].
- **Learning and Evaluation**: Lightweight strategy sampling leverages deterministic hashing and empirical evaluations, efficiently searching the policy space under PO. Q-learning on $Obs$ requires only a hash table over visited $(o,a)$ pairs.
- **Model Size and Generalizability**: Trees under PO are provably smaller and avoid overfitting to unobservable or irrelevant state-space dimensions. Empirical results show strong scalability and interpretability even with hundreds of thousands of possible system states [2412.05476].
- **Limitations**: No method can recover unmeasured latent state variables. Offline-only data and lack of interventions limit guarantees on outcome optimality. In high-dimensional observation spaces, additional regularization may be required for human readability [2203.08057].

## 5. Practical Applications and Empirical Evidence

Recent studies document practical deployment and validation of PO trees in several domains:

- **Industrial Logistics**: Open-pit mine scheduling demonstrates the scaling of PO trees to models with 80+ trucks and large grids, with empirically tight confidence intervals for maximized/minimized objectives [2412.05476].
- **Clinical Policy Modeling**: POETREE yields interpretable, history-adaptive treatment or intervention policies from large EHR or registry datasets. Metrics such as AUROC, AUPRC, Brier score, and interpretability ratings demonstrate effective transfer of physician decision rules under PO structures [2203.08057].
- **Reinforcement Learning and Control**: Decision trees over observable variables serve as transparent, memoryless controllers in Markov automata with partial observability, outperforming naïve or uniform strategies [2412.05476].

## 6. Extensions: Robustness, Alternative Tree Representations, and Variable Importance

Advanced work has considered the implications of PO for variable importance, predictive equivalence, and robustness to missing data:

- **Rewriting Trees via Boolean Logical Forms**: Decision boundaries can be represented canonically as Disjunctive Normal Forms (DNFs) over observable features, eliminating predictive equivalence among different trees yielding the same behavior, and enabling robust prediction in the presence of missing features via logical entailment [2506.14143].
- **Variable Importance under PO**: DNF-based global importance metrics, counting the presence of variables in prime implicants across the DNF, yield missingness-robust, canonical importances irrespective of evaluation order [2506.14143].
- **Cost-Sensitive Feature Acquisition**: Dynamic programming and Q-learning methods on partial assignments to observable features enable cost-efficient querying until a robust (i.e., logically entailed) prediction can be made under the PO tree [2506.14143].

## 7. Summary Table: Representative PO Decision Tree Frameworks

| Framework                | Policy Representation      | Extraction Method              | Interpretability          |
|--------------------------|---------------------------|-------------------------------|--------------------------|
| Q-Learning + dtControl [2412.05476] | Axis-aligned tree over $Obs$ | Simulation/driven, tabular → tree | High                     |
| POETREE [2203.08057]     | Differentiable soft-tree  | Backpropagation, imitation learning | High (after pruning/adaptation) |
| DNF-Tree [2506.14143]    | Minimal DNF over $Obs$    | Symbolic simplification        | Canonical/logical        |

In conclusion, decision trees under partial observability define a principled, technically robust, and empirically validated pathway for extracting compact, interpretable, and effective policies in a wide spectrum of sequential, causal, and control domains. Their foundations, methods of training and extraction, and robust representations position them as a central tool for interpretable policy and control learning under incomplete information.

Source: https://www.emergentmind.com/topics/decision-trees-under-po