Adaptive Decision Learning (ADL)

• Adaptive Decision Learning (ADL) is a paradigm where agents balance rapid, intuitive actions with slower, deliberate reasoning to optimize decisions under uncertainty.
• Key frameworks include dual-system reinforcement learning, adaptive decision trees, and Decision Transformers that dynamically adjust complexity based on task proficiency.
• Empirical results demonstrate that ADL methods enhance efficiency, generalization, and interpretability, reducing query complexity and improving performance in applications from clinical diagnostics to group testing.

Adaptive Decision Learning (ADL) is a paradigm in which agents dynamically balance fast, intuitive decision policies with slow, deliberative reasoning to optimize sequential decision-making. Recent instantiations of ADL range from dual-system reinforcement learning frameworks using vision-LLMs (VLMs), to adaptive decision tree policy extraction in clinical domains, to adaptive query selection in group testing, and exact learning of decision trees from membership queries. These approaches share the common attribute that adaptivity—modulating decision rules or learning steps based on accumulating information or proficiency—enables either higher efficiency, improved generalization, or greater interpretability across diverse settings.

1. Foundational Formulations and Objectives

In formal terms, ADL models decision-making as sequential optimization under uncertainty, often cast as a Markov decision process (MDP): $(\mathcal{S},\mathcal{A},P,R,\gamma)$, where $\mathcal{S}$ denotes the (possibly high-dimensional) state space, $\mathcal{A}$ a discrete action space, $P(s'|s,a)$ the transition dynamics, $R(s,a)$ an immediate reward function, and $\gamma\in[0,1)$ a discount factor. Typical objectives are to learn (or imitate) policies $\pi_\theta$ maximizing the expected cumulative reward $$J(\theta) = \mathbb{E}_{\tau\sim\pi_\theta}\left[ \sum_{t=0}^T \gamma^tR(s_t,a_t) \right]$$, extended under ADL to incorporate adaptive switching or corrections from a secondary (slow) policy module (Dou et al., 13 May 2025).

Alternative formalizations include learning an interpretable policy $\pi_\Theta(a_t|z_t,h_t)$ as a probabilistic tree (POETREE), minimizing a joint action-matching and next-observation prediction loss. In adaptive group testing, the goal is to minimize the expected query cost required to uniquely identify a sparse vector via adaptively chosen subset-sum queries, framed as an offline RL/inverse modeling problem with return-to-go objectives (Soleymani et al., 1 Sep 2025). For exact decision tree learning from membership queries, the objective is to identify the target function with the minimal number of adaptively chosen queries, approaching information-theoretic lower bounds (Bshouty et al., 2019).

2. Architectures and Algorithmic Structures

ADL frameworks operationalize adaptivity through architecture modularity or dynamic policy complexity. In DSADF (Dou et al., 13 May 2025), "System 1" is a reinforcement learning agent with an internal memory of sub-task proficiency; "System 2" is a VLM-based planner and performer employing chain-of-thought and self-reflection to decompose and reason about goals. A proficiency-based gating mechanism partitions sub-goals between systems: System 1 executes tasks it already masters (proficiency $p\geq T$), while System 2 handles out-of-distribution or novel sub-goals.

POETREE (Pace et al., 2022) implements a fully differentiable adaptive decision tree, where each node’s gate is parameterized and optimized by gradient descent. The tree grows incrementally during learning: suboptimal leaves are split and re-optimized only when validation loss indicates benefit, enforcing an adaptive Occam’s razor. Recurrent representations at leaves encode history, conferring memory over partially observed environments.

In adaptive group testing (Soleymani et al., 1 Sep 2025), the architecture uses Decision Transformers—a causal-transformer that ingests state, action, and return-to-go tokens and learns a query-selection policy over a sequence of stages. This policy adaptively exploits feedback in the query process, outperforming fixed non-adaptive strategies.

Adaptive exact learning of decision trees (Bshouty et al., 2019) uses random projections, combinatorially designed query sets, and two-stage or two-round algorithms to find all relevant variables and reconstruct the tree’s sparse polynomial representation with reduced query complexity.

3. Adaptive Switching and Complexity Control

ADL centralizes around adaptively modulating policy complexity or switching between decision modules in real-time. In DSADF, gating depends on a memory-based proficiency score, using a hard switch (indicator function) or optionally a smooth logistic function $\alpha(s) = \sigma(p(s) - T)$, but with a practical hard threshold. This mechanism enables the agent to prioritize well-learned behaviors and delegate uncertainty or novelty to a slower, more deliberative process.

POETREE’s structure is dynamically adapted during learning: leaves are split only when justified by validation performance, and low-probability branches are pruned, resulting in trees of depth adapted to the task—on ADNI, the average depth was $3.3\pm0.7$, demonstrating more compact representations than fixed-depth counterparts.

In group testing, adaptivity manifests as the dynamic selection of queries guided by prior subset-sum responses, yielding order-wise reductions in the number of queries required—down to information-theoretic lower bounds for small $k$ or at least below the best non-adaptive bounds for larger $k\leq 8$ (Soleymani et al., 1 Sep 2025).

4. Training, Optimization, and Theoretical Guarantees

Training ADL systems involves both classic and novel techniques. DSADF utilizes actor-critic policy-gradient learning with hierarchical and progressive rewards: combining target achievement, sub-goal completions, and proximity signals. The RL agent is updated via advantage estimates, while the VLM module remains fixed but is steered by prompt engineering.

POETREE is optimized end-to-end by backpropagating the combined action and next-observation losses, with additional splitting-entropy and $L_1$ penalties to regularize complexity and sparsity. The decision tree structure is optimized via a cycle of local-reoptimization and validation-gated splitting.

In group testing, Decision Transformers are trained offline on large sets of expert or random trajectories, minimizing cross-entropy losses on predicted versus ground-truth queries. The resultant inference procedure is $5$–$10\times$ faster at test time than classic entropy/covariance maximization heuristics.

Theoretical analysis across these settings highlights the advantages of adaptivity: DSADF’s sub-goal decomposition mitigates credit assignment, accelerates RL convergence, and enables coverage of out-of-distribution scenarios without loss of guarantees on the mastered subset (Dou et al., 13 May 2025); POETREE’s adaptive depth yields lower-capacity models with comparable prediction accuracy; in group testing, adaptivity is shown to halve the query cost in favorable regimes. The deterministic decision tree learning algorithm achieves $2^{5.83d}+2^{2d+o(d)}\log n$ queries, a substantial reduction from the $2^{18d+o(d)}\log n$ cost of previous methods (Bshouty et al., 2019).

5. Empirical Evaluations and Benchmarks

Empirical findings consistently substantiate the benefit of adaptivity:

• In DSADF (Dou et al., 13 May 2025), on the Crafter (2D sandbox) and HouseKeep (robotic rearrangement) environments, DSADF achieves higher task success rates (TSR $\approx 88\%$, average completion time $1524$s on Crafter) and significantly outperforms both RL- and VLM-only agents on out-of-distribution tasks (TSR $68$–$83\%$ vs. $<25\%$ for other methods). On HouseKeep, average object success rate (AOSR) reaches $92$–$96\%$ compared to $70$–$89\%$ for single-module approaches.
• POETREE (Pace et al., 2022) achieves higher interpretability and action-matching metrics on clinical datasets (ADNI, MIMIC) compared to black-box LSTM or simplex-boundary baselines, with clinician preference for its flow-chart explanations and adaptive topology.
• In group testing, DT–Entropy policies match the adaptive lower bound for $k=2$ and surpass non-adaptive baselines for $k\leq 8$, delivering up to a twofold reduction in the number of queries relative to the non-adaptive information-theoretic baseline (Soleymani et al., 1 Sep 2025).
• For adaptive exact decision tree learning (Bshouty et al., 2019), the two-round randomized algorithm and deterministic algorithm both demonstrably reduce query counts, thus lowering resource requirements in high-stakes applications such as drug screening.

6. Interpretability, Generalization, and Open Challenges

Interpretability and generalization are recurrent advantages of ADL techniques. POETREE policies are directly human-interpretable and adapt their complexity dynamically, avoiding overfitting on small data. DSADF leverages VLMs for semantic goal decomposition and handles previously unseen (out-of-distribution) circumstances by routing such tasks to a self-reflecting planner. Adaptive group testing exploits sequential feedback to generalize more efficiently in sparsity-structured problems.

Open challenges remain: optimality of adaptive algorithms (e.g., achieving $O(2^d\log n)$ queries for decision trees in polynomial time), transfer of ADL methods to broader concept classes, and practical isolation of adaptivity’s effect in more complex, continuous, or partially observable domains (Bshouty et al., 2019). Further empirical study of decision-making in high-throughput pipelines, as well as architectural innovations in scalability and latency for System 2 modules in large environments, remain open research directions.

---

References:

• (Dou et al., 13 May 2025) DSADF: Thinking Fast and Slow for Decision Making
• (Pace et al., 2022) POETREE: Interpretable Policy Learning with Adaptive Decision Trees
• (Soleymani et al., 1 Sep 2025) Learning to Ask: Decision Transformers for Adaptive Quantitative Group Testing
• (Bshouty et al., 2019) Adaptive Exact Learning of Decision Trees from Membership Queries