Platypoos: Scale-Free Planning Alg.

• Platypoos is a scale-free planning algorithm designed for deterministic-dynamics discounted MDPs with noisy rewards, identifying the best next action under a fixed simulation budget.
• It adapts simultaneously to unknown reward/noise scales and latent value function smoothness by running multiple candidate sampling schedules and validating optimal branches.
• The algorithm offers improved performance over OLOP with regime-specific guarantees, achieving exponential or polynomial simple regret rates in low-noise and high-noise settings respectively.

Searching arXiv for the primary paper and closely related prior work to ground the article. arXiv search: (Bartlett et al., 20 Apr 2026) Platypoos OLOP StroquOOL SequOOL OPD Platypoos is a scale-free planning algorithm for deterministic-dynamics, discounted-reward problems with a fixed simulation budget when rewards are noisy and the learner does not know the reward range or noise range in advance. It is introduced in "Scale-free adaptive planning for deterministic dynamics & discounted rewards" (Bartlett et al., 20 Apr 2026) as an adaptive replacement for OLOP in the regime where one has access to a generative model and seeks the best first action from a current root state rather than a globally optimal policy. Its central technical feature is simultaneous adaptation to unknown reward and noise scales and to latent smoothness of the value function, while preserving guarantees comparable to or better than prior planning methods and yielding markedly faster rates in favorable low-noise or noiseless regimes (Bartlett et al., 20 Apr 2026).

1. Problem class and formal objective

Platypoos is defined for an infinite-horizon discounted MDP with state space $X$, finite action space $A$ with branching factor $K = |A|$, deterministic dynamics,

$x_{t+1} = f(x_t,a_t),$

and stochastic rewards of the form

$r_t = r(x_t,a_t) + \varepsilon_t,$

where the noise is independent across rounds and bounded (Bartlett et al., 20 Apr 2026).

Two reward models are considered. In the deterministic-reward case,

$\varepsilon_t = 0, \qquad r_t = r(x_t,a_t).$

In the stochastic-reward case, the observed reward satisfies

$$\mathbb{E}[r_t \mid x_t] = r(x_t,a_t), \qquad |r_t - r(x_t,a_t)| \le b,$$

where the unknown quantity $b \in \mathbb{R}_+$ is the reward-noise range. Rewards are assumed bounded in $[0,R_{\max}]$, and the discount factor

$0 \le \gamma < 1$

is known (Bartlett et al., 20 Apr 2026).

The learner has access to a generative model and a total budget of $A$0 node openings or evaluations. The objective is to recommend the best next action from a given root state $A$1, minimizing simple regret rather than cumulative regret. For a policy $A$2, the discounted value is

$A$3

with $A$4 and $A$5, and

$A$6

The optimal quantities are

$A$7

with Bellman equations

$A$8

$A$9

If the algorithm recommends action $K = |A|$0 after budget $K = |A|$1, the simple regret is

$K = |A|$2

so the formal target is

$K = |A|$3

This formulation places Platypoos squarely in fixed-budget planning with deterministic transitions and stochastic rewards. A plausible implication is that its guarantees are tailored to simulator-based decision support rather than online reinforcement learning in the cumulative-regret sense.

2. Tree-based representation and discounted smoothness

The planning problem is rewritten as optimization over the infinite tree of action sequences. For an infinite sequence $K = |A|$4, its value is

$K = |A|$5

For a finite sequence $K = |A|$6,

$K = |A|$7

namely the value of the best continuation after prefix $K = |A|$8. The optimal root value is

$K = |A|$9

For a node $x_{t+1} = f(x_t,a_t),$0 of depth $x_{t+1} = f(x_t,a_t),$1, the partial discounted return is

$x_{t+1} = f(x_t,a_t),$2

and the trivial upper bound is

$x_{t+1} = f(x_t,a_t),$3

Hence

$x_{t+1} = f(x_t,a_t),$4

This discounted structure induces a baseline smoothness: even without extra regularity assumptions, the tail beyond depth $x_{t+1} = f(x_t,a_t),$5 is controlled by $x_{t+1} = f(x_t,a_t),$6 (Bartlett et al., 20 Apr 2026).

The key structural smoothness proposition states that there exist $x_{t+1} = f(x_t,a_t),$7 and $x_{t+1} = f(x_t,a_t),$8 such that for all depths $x_{t+1} = f(x_t,a_t),$9 and all nodes $r_t = r(x_t,a_t) + \varepsilon_t,$0,

$r_t = r(x_t,a_t) + \varepsilon_t,$1

This always holds with the trivial choice

$r_t = r(x_t,a_t) + \varepsilon_t,$2

but some problems may exhibit extra smoothness through smaller $r_t = r(x_t,a_t) + \varepsilon_t,$3 or $r_t = r(x_t,a_t) + \varepsilon_t,$4 (Bartlett et al., 20 Apr 2026). Platypoos does not require $r_t = r(x_t,a_t) + \varepsilon_t,$5 as input and is analyzed as adapting to them.

To quantify the effective size of the near-optimal set, the paper defines the near-optimal branching factor. For any $r_t = r(x_t,a_t) + \varepsilon_t,$6 and $r_t = r(x_t,a_t) + \varepsilon_t,$7,

$r_t = r(x_t,a_t) + \varepsilon_t,$8

where $r_t = r(x_t,a_t) + \varepsilon_t,$9 counts nodes $\varepsilon_t = 0, \qquad r_t = r(x_t,a_t).$0 such that

$\varepsilon_t = 0, \qquad r_t = r(x_t,a_t).$1

An analogous $\varepsilon_t = 0, \qquad r_t = r(x_t,a_t).$2 is introduced using $\varepsilon_t = 0, \qquad r_t = r(x_t,a_t).$3 instead of $\varepsilon_t = 0, \qquad r_t = r(x_t,a_t).$4, and Platypoos’ analysis is phrased in terms of $\varepsilon_t = 0, \qquad r_t = r(x_t,a_t).$5 (Bartlett et al., 20 Apr 2026).

The favorable case $\varepsilon_t = 0, \qquad r_t = r(x_t,a_t).$6 corresponds to essentially a unique near-optimal branch. This suggests that Platypoos’ strongest rates are expected when the planning tree rapidly narrows around a single optimal continuation.

3. Algorithmic design and scale-free adaptation

Platypoos is a budget allocation strategy over depths and sample counts. It combines open-loop planning over the action tree, as in OLOP, with a scale-free optimization strategy inspired by StroquOOL (Bartlett et al., 20 Apr 2026). The core issue is that the appropriate number of samples per node depends on unknown noise level and unknown gap structure: too few samples yield unreliable estimates, while too many prevent deep exploration. Platypoos resolves this by simultaneously trying many sample levels.

At each depth $\varepsilon_t = 0, \qquad r_t = r(x_t,a_t).$7, and for each sampling level indexed by $\varepsilon_t = 0, \qquad r_t = r(x_t,a_t).$8, the algorithm opens some of the best currently estimated nodes using roughly $\varepsilon_t = 0, \qquad r_t = r(x_t,a_t).$9-scaled samples. Large $$\mathbb{E}[r_t \mid x_t] = r(x_t,a_t), \qquad |r_t - r(x_t,a_t)| \le b,$$0 assigns many samples per node and explores fewer nodes; small $$\mathbb{E}[r_t \mid x_t] = r(x_t,a_t), \qquad |r_t - r(x_t,a_t)| \le b,$$1 allocates fewer samples per node and explores more broadly. Rather than relying on user-specified confidence radii, Platypoos implicitly runs many candidate exploration schedules in parallel and then performs a cross-validation or final evaluation phase among candidate branches corresponding to different $$\mathbb{E}[r_t \mid x_t] = r(x_t,a_t), \qquad |r_t - r(x_t,a_t)| \le b,$$2-values (Bartlett et al., 20 Apr 2026).

The paper characterizes the method as scale-free because it does not require the reward range $$\mathbb{E}[r_t \mid x_t] = r(x_t,a_t), \qquad |r_t - r(x_t,a_t)| \le b,$$3, the noise bound $$\mathbb{E}[r_t \mid x_t] = r(x_t,a_t), \qquad |r_t - r(x_t,a_t)| \le b,$$4, the smoothness parameters $$\mathbb{E}[r_t \mid x_t] = r(x_t,a_t), \qquad |r_t - r(x_t,a_t)| \le b,$$5, or the branching factor $$\mathbb{E}[r_t \mid x_t] = r(x_t,a_t), \qquad |r_t - r(x_t,a_t)| \le b,$$6. It does require the total budget $$\mathbb{E}[r_t \mid x_t] = r(x_t,a_t), \qquad |r_t - r(x_t,a_t)| \le b,$$7, the action set $$\mathbb{E}[r_t \mid x_t] = r(x_t,a_t), \qquad |r_t - r(x_t,a_t)| \le b,$$8, the discount factor $$\mathbb{E}[r_t \mid x_t] = r(x_t,a_t), \qquad |r_t - r(x_t,a_t)| \le b,$$9, and access to a generative model (Bartlett et al., 20 Apr 2026). This definition of scale-freeness is precise: the algorithm avoids upper confidence bounds that explicitly plug in reward or noise scales.

The reconstructed main procedure initializes

$b \in \mathbb{R}_+$0

then enters an exploration phase over depths and sample levels, followed by a cross-validation phase over candidate paths, and finally outputs

$b \in \mathbb{R}_+$1

(Bartlett et al., 20 Apr 2026). The source text is described as partially corrupted, but the supplied formulas retain the intended structure of the schedule.

A common misconception would be to treat Platypoos as merely an untuned UCB variant. The paper’s description is narrower and more specific: it is not based on a single confidence scale, but on a family of sample-allocation scales combined with final validation.

4. Technical mechanism: confidence control, opening conditions, and discount dependence

The analysis is organized around a high-probability confidence event $b \in \mathbb{R}_+$2 controlling empirical prefix-value estimates. If $b \in \mathbb{R}_+$3 is the set of sequences evaluated by Platypoos, then

$b \in \mathbb{R}_+$4

and

$b \in \mathbb{R}_+$5

The deviation radius therefore scales as

$b \in \mathbb{R}_+$6

so larger $b \in \mathbb{R}_+$7 yields tighter confidence through heavier sampling (Bartlett et al., 20 Apr 2026).

The key technical quantity $b \in \mathbb{R}_+$8 denotes the depth of the deepest opened node on an optimal branch under sample level $b \in \mathbb{R}_+$9. A principal lemma states that, on event $[0,R_{\max}]$0, for

$[0,R_{\max}]$1

one has $[0,R_{\max}]$2 if two conditions hold simultaneously. The first is discrimination by sampling,

$[0,R_{\max}]$3

meaning estimation error is small enough to resolve $[0,R_{\max}]$4-depth near-optimality. The second is sufficient opening relative to the near-optimal set, expressed generically as

$[0,R_{\max}]$5

The paper also gives more detailed case splits depending on whether $[0,R_{\max}]$6 or $[0,R_{\max}]$7, and whether $[0,R_{\max}]$8 or $[0,R_{\max}]$9 (Bartlett et al., 20 Apr 2026).

Discounting enters the algorithm and its proof in three distinct ways. First, it provides the baseline smoothness relation

$0 \le \gamma < 1$0

Second, the sampling schedule uses terms involving $0 \le \gamma < 1$1, reflecting the smaller importance of deeper rewards under discounting. Third, validation cost and confidence accumulation involve

$0 \le \gamma < 1$2

which arises from controlling discounted sums of bounded deviations through $0 \le \gamma < 1$3 (Bartlett et al., 20 Apr 2026).

These ingredients clarify that Platypoos is neither depth-uniform nor noise-uniform. Its schedule is explicitly structured around the interaction among depth, discounting, near-optimal branching, and empirical uncertainty.

5. Performance guarantees and regime structure

The guarantees separate deterministic and stochastic reward settings. For noiseless rewards, a simpler SequOOL-style planning method yields the following simple-regret rates for a planning problem with associated $0 \le \gamma < 1$4 and branching factor $0 \le \gamma < 1$5:

• if $0 \le \gamma < 1$6,

$0 \le \gamma < 1$7

• if $0 \le \gamma < 1$8,

$0 \le \gamma < 1$9

Under a reset constraint, the exponential case weakens to $A$00, while the polynomial case remains essentially the same (Bartlett et al., 20 Apr 2026).

For stochastic rewards, the analysis introduces

$A$01

up to notation corruption in the source, and defines special depths $A$02 via balances between noise discrimination and near-optimal coverage: $A$03

$A$04

These depths characterize how deeply Platypoos can certify the optimal branch in different regimes (Bartlett et al., 20 Apr 2026).

In the high-noise regime, if the noise is large enough to satisfy the paper’s two high-noise conditions, Platypoos satisfies

$A$05

In the low-noise regime, if the paper’s low-noise conditions hold, then

$A$06

The $A$07 low-noise case is especially notable because the regret decays exponentially (Bartlett et al., 20 Apr 2026).

The paper’s summary also highlights an intermediate deterministic-like regime for $A$08, where the rate can become

$A$09

depending on noise and reset assumptions. This suggests that the boundary between stochastic and deterministic behavior is governed not only by $A$10, but also by the geometry of the near-optimal tree and the discount factor.

6. Relation to prior methods, practical use, and caveats

The main benchmark is OLOP. OLOP uses upper confidence bounds of the form

$A$11

where $A$12 is a user-provided bound on noise. According to the supplied account, this works if $A$13, but if $A$14 is too large then exploration is overly conservative, while if $A$15 is too small then the confidence intervals are invalid (Bartlett et al., 20 Apr 2026). Platypoos avoids this dependence by not building a UCB from $A$16; instead it runs a family of sample-allocation scales in parallel and validates them afterward.

The paper emphasizes four improvements over OLOP: adaptation to unknown noise, adaptation to unknown reward scale, adaptation to extra smoothness through $A$17 rather than the baseline $A$18, and low-noise or noiseless speedups. In particular, when noise is absent or tiny, Platypoos can recover deterministic planning rates and, for $A$19, exponentially decaying regret, whereas OLOP remains tied to a fixed confidence scale (Bartlett et al., 20 Apr 2026).

Platypoos is presented as practically relevant when a simulator or generative model is available, the state space is too large for full dynamic programming, only a good next action is needed under a fixed budget, rewards may be noisy, and the reward range, variance or noise range, or effective smoothness are not known in advance. The experiments described in the paper use a simple deterministic-dynamics MDP plus additive bounded noise, and Platypoos reportedly systematically outperforms OLOP while being insensitive to incorrect user inputs that can severely degrade OLOP (Bartlett et al., 20 Apr 2026).

Several limitations are explicit. The theory assumes deterministic transitions; the stochasticity is only in rewards. The noise must be bounded because the analysis uses Hoeffding- or Azuma-style confidence bounds. The setting is fixed-budget simple regret rather than cumulative-regret reinforcement learning. The guarantees depend on multiple regime splits involving $A$20, high versus low noise, and reset assumptions. Reset constraints weaken exponential rates, with

$A$21

The appendix also notes that floor and ceiling operations in the practical schedule may underuse budget unless tuned carefully (Bartlett et al., 20 Apr 2026).

One point in the supplied material requires careful reading. The abstract states that the paper establishes a matching lower bound, but the detailed summary also states that the supplied text does not contain a separately stated formal lower-bound theorem in the same explicit style as the upper bounds. In that summary, the matching comparison is described primarily relative to existing upper bounds from OLOP and OPD rather than to an explicit minimax lower-bound statement (Bartlett et al., 20 Apr 2026). This suggests that any interpretation of “matching” should distinguish the abstract’s claim from the level of detail visible in the provided exposition.