Psi-Decision Trees in Convex Analysis

• Psi-Decision Trees are decision trees defined through a Bregman divergence framework using differentiable, strictly convex functions to generate impurity measures.
• They unify traditional CART criteria by deriving variance, entropy, and Gini measures as special cases from a common convex analytic construction.
• The framework adapts to diverse response types by selecting a matching convex potential, enabling efficient split optimization and guaranteeing statistical consistency.

Searching arXiv for the cited papers and closely related work to ground the article. Psi-Decision Trees, in the Bregman-divergence formulation, are decision trees whose node impurity and splitting criteria are induced by a differentiable, strictly convex potential $\psi$ via its associated Bregman divergence. They generalize the CART paradigm by replacing ad hoc impurity choices with a unified construction grounded in convex analysis and information geometry, and they subsume squared error, entropy, Gini, Poisson deviance, and Itakura–Saito as special cases (Bourel, 12 Jun 2026). The notation $\psi$ is also used in other decision-tree literatures to denote a bounded path-complexity measure for deterministic and nondeterministic trees (Ostonov et al., 2023) and a user-specified requirement for black-box policy synthesis (Demirović et al., 2024); a related decision-theoretic method, PSICA, uses trees to summarize probability-of-best treatment vectors in randomized trials with categorical treatments (Sysoev et al., 2018).

1. Formal definition in the Bregman framework

Let $\psi$ be a differentiable strictly convex function defined on a convex set $C \subset \mathbb{R}^d$. The associated Bregman divergence is

$$D_{\psi}(x, y) = \psi(x) - \psi(y) - \nabla \psi(y)^{\top}(x-y).$$

It measures how much $\psi$ deviates from its first-order tangent approximation at $y$ when evaluated at $x$. It satisfies $D_\psi \ge 0$ and equals $0$ iff $\psi$0 (Bourel, 12 Jun 2026).

For a dataset $\psi$1 in $\psi$2, node impurity is defined as

$\psi$3

The minimizer $\psi$4 is the Bregman centroid of $\psi$5, characterized by

$\psi$6

This is dual averaging: one averages in the dual coordinates induced by $\psi$7 and then maps back through $\psi$8.

For a candidate split $\psi$9, the impurity gain is

$\psi$0

Splits are chosen to maximize $\psi$1. Because $\psi$2 is an expected convex Jensen gap, $\psi$3.

A technical distinction arises between $\psi$4 and $\psi$5 in exponential-family modeling. In natural-parameter space, $\psi$6 often denotes the log-partition. In mean space, the canonical prediction loss is typically generated by $\psi$7. When the response lives in mean space, one uses $\psi$8 and replaces $\psi$9 with $C \subset \mathbb{R}^d$0 where appropriate. This is not a separate algorithmic idea; it is a coordinate choice within the same convex-analytic framework.

2. Recovery of classical CART criteria

The unifying claim of the framework is that many standard impurity measures are instances of particular convex generators. CART’s variance, entropy, and Gini criteria therefore appear not as unrelated design choices but as special cases derived from a common divergence construction (Bourel, 12 Jun 2026).

Setting	Generator	Induced impurity or divergence
Regression	$C \subset \mathbb{R}^d$1	$C \subset \mathbb{R}^d$2; impurity is mean squared error/within-node variance
Multiclass classification	$C \subset \mathbb{R}^d$3	$C \subset \mathbb{R}^d$4; impurity simplifies to entropy $C \subset \mathbb{R}^d$5
Gini-like classification	$C \subset \mathbb{R}^d$6 on the simplex	impurity becomes $C \subset \mathbb{R}^d$7, proportional to Gini
Count data	$C \subset \mathbb{R}^d$8	$C \subset \mathbb{R}^d$9; impurity is average Poisson deviance
Positive heteroscedastic data	$$D_{\psi}(x, y) = \psi(x) - \psi(y) - \nabla \psi(y)^{\top}(x-y).$$0	$$D_{\psi}(x, y) = \psi(x) - \psi(y) - \nabla \psi(y)^{\top}(x-y).$$1; Itakura–Saito divergence

For multiclass classification, taking $$D_{\psi}(x, y) = \psi(x) - \psi(y) - \nabla \psi(y)^{\top}(x-y).$$2 over the simplex yields $$D_{\psi}(x, y) = \psi(x) - \psi(y) - \nabla \psi(y)^{\top}(x-y).$$3. With one-hot labels, the centroid is the empirical class-proportion vector $$D_{\psi}(x, y) = \psi(x) - \psi(y) - \nabla \psi(y)^{\top}(x-y).$$4, and node impurity reduces to $$D_{\psi}(x, y) = \psi(x) - \psi(y) - \nabla \psi(y)^{\top}(x-y).$$5. Entropy reduction is thus a Bregman impurity reduction. If instead one uses a quadratic potential on the simplex, the same construction produces a quantity proportional to the Gini index.

For counts and positive responses, the framework moves naturally to mean-space generators. The Poisson choice $$D_{\psi}(x, y) = \psi(x) - \psi(y) - \nabla \psi(y)^{\top}(x-y).$$6 induces the Poisson deviance, while $$D_{\psi}(x, y) = \psi(x) - \psi(y) - \nabla \psi(y)^{\top}(x-y).$$7 induces Itakura–Saito, which is scale-invariant and is therefore suited to multiplicative or heteroscedastic positive responses.

The exponential-family interpretation is central. For a regular exponential family with density

$$D_{\psi}(x, y) = \psi(x) - \psi(y) - \nabla \psi(y)^{\top}(x-y).$$8

Banerjee’s representation gives

$$D_{\psi}(x, y) = \psi(x) - \psi(y) - \nabla \psi(y)^{\top}(x-y).$$9

Optimizing node impurity with $\psi$0 then corresponds to likelihood optimization, while $\psi$1 in dual coordinates aligns with KL between family members. This suggests that the choice of $\psi$2 is not merely computational; it encodes a statistical model and an underlying geometry.

3. Algorithmic construction and computational structure

The generic tree-construction procedure mirrors CART but replaces the impurity calculation and node representative with their Bregman analogues (Bourel, 12 Jun 2026). At a node containing sample set $\psi$3, one computes the centroid $\psi$4 from

$\psi$5

evaluates

$\psi$6

and stops if standard criteria hold: depth at least $\psi$7, node size below $\psi$8, impurity below $\psi$9, or no split with gain at least $y$0. Otherwise, one scans candidate feature-threshold splits and selects the split maximizing $y$1.

Efficient threshold scanning follows from sufficient statistics. The centroid depends on

$y$2

During a left-to-right scan along a sorted feature, one updates $y$3 and $y$4 incrementally as points move across the threshold. Child centroids are recomputed as $y$5 and $y$6, while impurities are updated using streaming sums of $y$7 and inner products through

$y$8

This yields $y$9 scanning per feature after an $x$0 sort, comparable to CART.

Prediction at leaves depends on the coordinate system. In $x$1-space one predicts the representative $x$2 or maps it to the mean parameter $x$3. In $x$4-space, the leaf prediction is directly $x$5. For multiclass classification this is the leaf class-probability vector; for regression it is the mean; for Poisson models it is the rate; for Gamma or Exponential settings it is the positive mean parameter.

Pruning extends in the expected way. Cost-complexity pruning is formulated with the $x$6-induced impurity:

$x$7

The penalty parameter $x$8 is selected by cross-validation, optionally using the $x$9-SE rule, under the same $D_\psi \ge 0$0-induced loss.

4. Geometry, bounds, and consistency

The gain of a split admits an exact centroid decomposition. If $D_\psi \ge 0$1, $D_\psi \ge 0$2, and $D_\psi \ge 0$3 are the centroids of $D_\psi \ge 0$4, $D_\psi \ge 0$5, and $D_\psi \ge 0$6, then

$D_\psi \ge 0$7

Equivalently, if a random variable $D_\psi \ge 0$8 takes values $D_\psi \ge 0$9 and $0$0 with probabilities $0$1 and $0$2, then

$0$3

The gain is therefore a Jensen gap and is nonnegative (Bourel, 12 Jun 2026).

Curvature properties of $0$4 translate directly into gain bounds. If $0$5 is $0$6-strongly convex,

$0$7

then

$0$8

Splits that separate child centroids in Euclidean norm therefore have guaranteed nontrivial gain. If $0$9 has $\psi$00-Lipschitz gradient,

$\psi$01

and one obtains the upper bounds

$\psi$02

Consistency follows the classical partition-regularity route. Under shrinking cell diameters, growing sample sizes per leaf, and finite second moment of the response, the Bregman risk

$\psi$03

is minimized by

$\psi$04

If $\psi$05 is $\psi$06-smooth, then

$\psi$07

This places Psi-Decision Trees under the same partition regularity used for CART, provided $\psi$08 is smooth.

The information-geometric interpretation is equally explicit. Natural and mean coordinates satisfy $\psi$09 and $\psi$10. Centroids are averages in dual space, while the Bregman Pythagorean theorem explains why projections onto convex sets minimize expected divergence. A plausible implication is that the framework is best understood not as a catalogue of impurity formulas but as a recursive-partitioning method defined on dual affine geometries.

5. Other meanings of $\psi$11 in decision-tree research

A distinct line of work uses $\psi$12 not as a loss generator but as a bounded complexity measure on paths in deterministic and nondeterministic decision trees for many-valued decision tables (Ostonov et al., 2023). In that setting, $\psi$13 satisfies positivity, commutativity, monotonicity, and boundedness from above, with the additional bounded-complexity condition $\psi$14. For a tree $\psi$15, complexity is

$\psi$16

and one studies the minimum deterministic and nondeterministic complexities, $\psi$17 and $\psi$18, as well as the worst-case comparison function $\psi$19. The main structural result is a dichotomy: if $\psi$20 is everywhere defined, then it is either bounded above by a constant or satisfies $\psi$21 for infinitely many $\psi$22. The same paper also proves that for any nondecreasing $\psi$23 with $\psi$24 and $\psi$25, one can construct a closed class $\psi$26 and a bounded complexity measure $\psi$27 such that

$\psi$28

Here, “$\psi$29-decision tree” refers to a cost model on attribute queries rather than to Bregman impurity.

A separate synthesis literature treats $\psi$30 as a specification or cost functional for control. Decision-tree policies are synthesized for deterministic black-box systems under a finite predicate discretization and bounded tree size, with optimality defined with respect to a trace-based objective such as steps-to-goal (Demirović et al., 2024). The search enumerates finite tree hypotheses and prunes them through trace equivalence and branch-and-bound. Its key device is a distance-based pruning rule: after evaluating a predicate $\psi$31 at a node, one records the minimum value of $\psi$32 among states that took the true branch and skips all later thresholds $\psi$33, because under determinism those trees are trace-equivalent on the evaluated initial states. Under deterministic dynamics, finite predicate discretization, bounded tree size or depth, and finite-horizon evaluation, the method is complete for the finite hypothesis class and returns the smallest optimal tree among ties. The reported runtime reduction from trace-based pruning is $\psi$34–$\psi$35, with examples including CartPole $\psi$36 for $\psi$37 and MountainCar $\psi$38 for $\psi$39.

PSICA is related but different again. It addresses randomized trials with more than two categorical treatments by fitting treatment-wise predictive models, estimating for each covariate vector the probability that each treatment is best, and then growing a decision tree on those probability vectors (Sysoev et al., 2018). For a subgroup $\psi$40, the leaf reports

$\psi$41

where $\psi$42 is the probability that treatment $\psi$43 is best, and the leaf label is the set of possible best treatments after $\psi$44-based exclusion of low-probability options. Splitting is based on a loss tied to expected misassignment cost, and an optional significance mask uses a chi-square test to suppress spurious splits. In simulations, PSICA methods typically achieved near-perfect accuracy, about $\psi$45–$\psi$46, while uncertainty and suspect-split behavior depended on whether probability estimation used bootstrap or infinitesimal-jackknife uncertainty propagation. The leaf-level ability to say “I don’t know” marks a decision-theoretic extension of tree outputs beyond single-label prediction.

6. Practical use, limitations, and extensions

Within the Bregman framework, the operational recommendation is to choose $\psi$47 to match the response type and noise structure (Bourel, 12 Jun 2026). Squared error is appropriate for Gaussian-like homoscedastic regression, negative entropy for categorical or multiclass responses, quadratic potentials on the simplex for Gini-like criteria, Poisson divergence for counts, and Itakura–Saito for positive heteroscedastic data with multiplicative or scale noise. The same source notes that gains in generalization can occur when $\psi$48 matches the data-generating process, whereas performance is indistinguishable from CART when both optimize the same criterion.

The framework has domain and regularity constraints. Responses must lie in the domain of $\psi$49; on the boundary of the probability simplex, gradients may be undefined, so smoothing is needed. For probabilities, Laplace smoothing avoids $\psi$50 when computing logarithms. For positive responses, clipping at $\psi$51 avoids $\psi$52 and division by zero under Itakura–Saito. Numerical stabilization may require log-sum-exp for KL calculations and stabilized gradients more generally.

Model mismatch is a statistical limitation rather than a formal failure mode. Using squared error under multiplicative noise, for example, reduces statistical efficiency. The text explicitly states that one should choose $\psi$53 aligned with the data’s exponential-family model or error structure. A related misconception is that every convex loss fits the framework: absolute deviation is not a Bregman divergence. Extensions to non-differentiable objectives are possible through generalized Bregman divergences with subgradients or by replacing the loss with other convex risks such as pinball loss, but the paper notes that one then loses closed-form centroids.

The framework extends naturally to ensembles. Psi-Decision Trees can be inserted into random forests and boosting by replacing impurity or loss with the $\psi$54-induced version. The same source states that diversity or ambiguity decompositions extend to Bregman divergences, supporting ensemble analysis under the $\psi$55 framework. This suggests a broad research program in which recursive partitioning, convex duality, and model-aware loss design are treated as a single object rather than as separate design decisions.

Across the alternate meanings of $\psi$56, the main source of confusion is terminological. In Bregman-based trees, $\psi$57 is a convex potential; in complexity theory, it is a bounded complexity measure; in policy synthesis, it is a specification or cost functional; and in PSICA the emphasis is on probability-of-best treatment vectors rather than on a literal $\psi$58 generator. The commonality is structural rather than definitional: each framework uses trees as interpretable recursive partitions while elevating a mathematically explicit objective—divergence, path cost, trace specification, or decision-theoretic loss—to the central organizing role.