Indecision Trees: Uncertainty-Aware Models

• Indecision Trees are tree-based models that integrate uncertainty directly into both learning and inference through probabilistic routing and soft assignments.
• They employ mechanisms like probabilistic splits and uncertainty zones to handle measurement error, sensor noise, and data ambiguity effectively.
• Empirical evidence shows these models improve accuracy and calibration in both classification and regression tasks compared to classical decision trees.

Indecision Trees are a class of tree-based models that generalize classical decision trees by incorporating measured, modeled, or locally-estimated uncertainty directly into the tree learning and inference process. This “softens” the standard hard routing of classical trees, propagating examples probabilistically or flagging regions of ambiguity, and yields improved prediction calibration, principled uncertainty estimation, and more nuanced argument-based reasoning under uncertainty across classification and regression settings. Several architectures have been developed, including indecision trees with probabilistic routing, ternary trees with locally-adaptive uncertainty zones, and input-uncertainty-sensitive regression trees. These models are motivated by the demands of real-world domains in which measurement error, sensor noise, and data ambiguity are ubiquitous.

1. Formal Model Classes

Indecision Trees encompass several related modeling frameworks unified by their explicit treatment of uncertainty during either tree construction, inference, or both.

• Uncertainty-Weighted (Probabilistic) Decision Trees: Each input sample $x_i$ at node $q$ is characterized by a feature distribution (e.g., for continuous features, $x_{ij}\sim N(\mu_{ij},\sigma^2_{ij})$; for discrete features, a categorical pmf). Given a candidate split (clause) $C$, the probability for a sample to follow either branch is determined by integrating (for continuous) or summing (for discrete) the input’s uncertainty, e.g. $p(C(x_i))=\int_{t}^\infty N(z;\mu, \sigma^2)dz$ for a threshold test. Each instance flows down both branches with probability weights, so leaves store class mixtures weighted over mass received. At prediction time, inference consists of computing the total probability over decision paths, yielding a full output distribution (Kent et al., 2022).
• Ternary (Uncertainty-Zone) Trees: Each node induces, in addition to traditional binary splits, a local “undecided” or “boundary-uncertain” region, defined by a half-width $\delta$ around the optimal split threshold $\theta^*$. Rather than assign all $x$ with $x_f = \theta^*$ to either child, the algorithm routes instances in $[\theta^*-\delta, \theta^*+\delta]$ to both child subtrees, blending outputs and flagging these instances as boundary-uncertain. The uncertainty zone half-width, $q$0, is estimated locally from split statistics using one of several proposed, easily-computable methods (Smits, 21 May 2026).
• Uncertain Regression Trees: Each observed input $q$1 is modeled as a noisy measurement of a latent “true” value $q$2, with $q$3. Membership in each region is soft—the probability that $q$4 belongs to leaf $q$5 is $q$6, computed as a product of one-dimensional normal CDF differences. Predictions are weighted sums across all leaves, and splits are selected by expected loss (quadratic) under these soft assignments (Tami et al., 2018).

2. Uncertainty Quantification Mechanisms

Different models within the indecision tree framework use distinct, mathematically explicit schemes for propagating and quantifying uncertainty.

• Probability Routing: At each node, the probability for each instance to take the left or right branch is the probability, under its input distribution, that it falls on that side of the threshold. This is computed via integrals/CDFs for continuous, or simple pmf summations for discrete variables (Kent et al., 2022).
• Boundary-Uncertain Flagging (Ternary Trees): For each internal node with split threshold $q$7 and zone half-width $q$8, the instance is:
  • routed left if $q$9
  • routed right if $x_{ij}\sim N(\mu_{ij},\sigma^2_{ij})$0
  • blended (and flagged as boundary-uncertain) if $x_{ij}\sim N(\mu_{ij},\sigma^2_{ij})$1, with weights $x_{ij}\sim N(\mu_{ij},\sigma^2_{ij})$2, $x_{ij}\sim N(\mu_{ij},\sigma^2_{ij})$3 (Smits, 21 May 2026).
• Input Uncertainty in Regression: For region $x_{ij}\sim N(\mu_{ij},\sigma^2_{ij})$4, $x_{ij}\sim N(\mu_{ij},\sigma^2_{ij})$5. Predictions are $x_{ij}\sim N(\mu_{ij},\sigma^2_{ij})$6 (Tami et al., 2018).

3. Learning Algorithms and Split Criteria

• Greedy Construction under Uncertainty (Weighted Entropy): At each node, candidate split clauses are evaluated by a generalized information gain metric defined on the uncertainty-weighted empirical distribution: $x_{ij}\sim N(\mu_{ij},\sigma^2_{ij})$7, where $x_{ij}\sim N(\mu_{ij},\sigma^2_{ij})$8 (Kent et al., 2022).
• Locally-Adaptive $x_{ij}\sim N(\mu_{ij},\sigma^2_{ij})$9 Estimation in Ternary Trees: Five methods are used for adaptive boundary width estimation:
  • Quality-Plateau: Width of near-optimal split thresholds in the split criterion curve.
  • Class-Overlap: Region width over which empirical class-conditional distributions overlap.
  • Gain-Ratio: Measures split ambiguity.
  • Node-Bootstrap: Standard deviation of $C$0 under bootstrap resampling.
  • Margin: Minimal gap to nearest opposite-class sample on either side of $C$1 (Smits, 21 May 2026).
• Soft Assignment in Regression Trees: Training minimizes expected quadratic loss under soft assignments, i.e., $C$2. Splitting proceeds by choosing the variable and threshold yielding minimal expected risk reduction (Tami et al., 2018).

4. Theoretical Properties and Guarantees

• Monotonic Entropy Decrease: In uncertainty-weighted trees, the greedy split at each node never increases overall expected leaf entropy $C$3; training is guaranteed to terminate in finite depth (either on class-pure leaves or prescribed max depth) (Kent et al., 2022).
• Calibration Properties: In ternary trees, the boundary-uncertain rate $C$4 can be compared to the Bayes error $C$5 on synthetic benchmarks. The margin-based method yields $C$6 close to 1 on simple, clean data, indicating self-calibration, while more conservative methods over-flag uncertain regions (i.e., $C$7) (Smits, 21 May 2026).
• Upper and Lower Bounds: Top-down induction of decision trees using influence/purity-based heuristics (e.g., ID3/CART/C4.5) admits upper bounds such as $C$8 for $C$9 with optimal decision tree size $p(C(x_i))=\int_{t}^\infty N(z;\mu, \sigma^2)dz$0; but also lower bounds $p(C(x_i))=\int_{t}^\infty N(z;\mu, \sigma^2)dz$1 for certain function classes, showing potential super-polynomial blow-up in tree size even for ε-approximations (Blanc et al., 2019).

5. Argument Extraction and Interpretability

Each leaf in an uncertainty-weighted indecision tree corresponds to a logical “argument” or rule with antecedent (conjunction of split clauses) and associated class probability vector. Redundant or implied tests along a path can be minimized to yield a compact, disjoint set of probabilistic rules, directly integrable into argument-based and symbolic reasoning systems. Interpretability is inherited from the explicit path structure, while the output distribution at each leaf encodes quantified uncertainty (Kent et al., 2022).

6. Empirical Performance and Applications

• Classification: On synthetic data with explicit input uncertainty, indecision trees yield marked accuracy improvements ($p(C(x_i))=\int_{t}^\infty N(z;\mu, \sigma^2)dz$2 absolute, or $p(C(x_i))=\int_{t}^\infty N(z;\mu, \sigma^2)dz$3 relative error reduction) over classical trees that ignore uncertainty (Kent et al., 2022).
• Real-World Datasets: On OpenML-CC18 (71 datasets), ternary decision trees with each of the five proposed $p(C(x_i))=\int_{t}^\infty N(z;\mu, \sigma^2)dz$4 methods outperform standard CART in decided accuracy, with the margin method achieving the best efficiency ($p(C(x_i))=\int_{t}^\infty N(z;\mu, \sigma^2)dz$5 accuracy gain per unit flagging rate) and winning on 42/72 datasets; boundary-uncertain rates range from $p(C(x_i))=\int_{t}^\infty N(z;\mu, \sigma^2)dz$6 to $p(C(x_i))=\int_{t}^\infty N(z;\mu, \sigma^2)dz$7 depending on the method (Smits, 21 May 2026).
• Regression: Soft-assignment regression trees significantly outperform classical single-tree and even large Random Forests when input noise is present. For instance, an uncertain Random Forest with 15 trees outperforms a standard RF with 500 trees on perturbed data (Tami et al., 2018).
• Specialized Domains: In medical (e.g., mammography) and financial prediction, node-bootstrap and quality-plateau δ methods yield improved accuracy at modest boundary-uncertain rates and provide practical boundary uncertainty flagging for downstream triage (e.g., triggering human review) (Smits, 21 May 2026).

7. Context, Extensions, and Theoretical Connections

Indecision Trees generalize the divide-and-conquer principle of classical decision trees by integrating either quantifiable measurement/modeling uncertainty, adversarial ambiguity near split regions, or explicit distributional representations over both training and inference stages. They connect to Bayesian CART and BART models in quantifying predictive uncertainty and to regression splines/MARS for representing region boundaries via basis expansions (Zhang, 2021). From a logic perspective, the extracted probabilistic argument set is suitable for downstream symbolic or probabilistic-logic reasoning. Operationally, indecision tree algorithms resemble and extend impurity-based top-down heuristics (e.g., ID3, C4.5, CART) but avoid pathological super-polynomial blowups on certain function classes by embracing probabilistic routing and uncertainty modeling (Blanc et al., 2019).

A plausible implication is that indecision trees serve as an intermediate paradigm between crisp, fully-deterministic tree models and full Bayesian nonparametric trees, offering computational tractability, statistical robustness, and explicit uncertainty awareness directly relevant for safety-critical or ambiguity-rich real-world tasks.