--- title: Time Optimization Policy (TOP) url: https://www.emergentmind.com/topics/time-optimization-policy-top type: topic --- # Time Optimization Policy (TOP) Searching arXiv for the cited TOP-related papers and closely related work. Time Optimization Policy (TOP) is used in recent literature to denote policies that optimize a temporal degree of freedom under explicit structural constraints. In the cited works, the optimized variable is not uniform across domains: it can be a path-parameterization along a geometric path, the convergence time to an equilibrium under sampling-based Lyapunov stability, inter-clip timestamps for humanoid upper-body motion, the number of self-generated rollouts used for test-time adaptation, or dispatch timing in dynamic team orienteering [2306.05197] [2301.00521] [2508.00355] [2512.02882] [2601.11010]. This suggests that TOP is best understood as a family of constrained time-optimization policies rather than a single canonical algorithm. ## 1. Common formulation pattern Across the cited literature, TOP formulations share a recurrent structure: a temporal control variable is optimized jointly with a feasibility condition that is domain-specific. In robotics path tracking, the objective is to track a geometric path in minimal time subject to kinodynamic constraints and SSM safety. In optimal-time stability, the objective is to minimize discounted cumulative cost subject to mean-cost stability. In humanoid standing manipulation, TOP adjusts timestamps $\Delta t_t$ so as to keep the system within the balance margin while minimizing time. In test-time policy optimization, rollout sampling is halted once posterior confidence exceeds a specified threshold and the retained rollouts are used for an on-policy update. In dynamic team orienteering, the platform decides which requests to accept and how to route workers under hard time budgets [2306.05197] [2301.00521] [2508.00355] [2512.02882] [2601.11010]. | Domain | Temporal decision | Governing constraint | |---|---|---| | ISO-safe path tracking | Path traversal time | Kinodynamic constraints and SSM safety | | Optimal-time stability | Convergence time in a mean-cost sense | Sampling-based Lyapunov stability | | Humanoid standing manipulation | Inter-clip timestamps $\Delta t_t$ | Balance, precision, and time efficiency | | Test-time policy optimization | Rollout stopping time | Wald thresholds and minimum retained samples | | Dynamic team orienteering | Rolling-horizon dispatch timing | Worker time budgets and task time windows | This common pattern does not erase substantive differences among formulations. In some cases TOP is a control policy over physical motion; in others it is a sequential decision rule over compute allocation or online dispatch. The unifying element is that the temporal decision is itself the optimization target, rather than a fixed schedule imposed externally. ## 2. Reachability-based time-optimal path tracking and ISO safety A central robotics lineage for TOP is Time-Optimal Path Parameterization (TOPP). In the reachability-analysis formulation of Pham and Pham, a robot with configuration $q\in\mathbb R^n$ follows a geometric path $q(s)$, $s\in[0,s_{\rm end}]$, with time-parameterization $s(t)$ under second-order constraints of the form $$ a(s)\,\ddot s + b(s)\,\dot s^2 + c(s)\in C(s). $$ After discretization into $N$ segments with $x_i=\dot s_i^2$, $u_i=\ddot s_i$, and $\Delta_i=s_{i+1}-s_i$, the dynamics become $$ x_{i+1}=x_i + 2\,\Delta_i\,u_i, $$ and the algorithm computes reachable and controllable sets by solving small LPs before performing a greedy forward pass. TOPP-RA has two passes, each $O(N)$ steps with two LPs per step, returns a feasible solution whenever one exists, and is asymptotically optimal as the grid is refined [1707.07239]. The ISO-safe extension formulates a stricter problem. Given a geometric path $P(s)$ in configuration space, kinodynamic constraints, dynamic obstacles of known maximum Cartesian speed $v_{q,\max}$, and a protective separation distance $d_{\text{protective}}$, the objective is to track $P(s)$ in minimal time subject to kinodynamic constraints and the SSM requirement that, if a collision occurs, the robot must be in a stationary state at the time instant of collision [2306.05197]. The formulation inherits the TOPP-RA discretization, $$ x_{i+1}=x_i + 2\Delta_i\,u_i,\qquad \Omega_i=\{(u,x)\mid a_i u + b_i x + c_i\in\mathcal C_i\}, $$ and adds stoppable sets $$ K_{j,N}=\{0\},\qquad K_{j,i}=Q_i(K_{j,i+1}), $$ together with the separation-time quantity $$ \psi_\ell = \min_q \left[\frac{d_{\ell,q}-d_{\text{protective}}}{v_{q,\max}}\right]. $$ At execution time, with current stage-velocity index $(i,k)$ and $\sqrt{x_i}=k\cdot \delta v$, the policy selects $$ j_{\text{stop}}=\max\{j\ge i\mid \tau_{j,i,k}\le \psi_j\} $$ and applies the first control from the standard TOPP-RA forward pass on $[i\ldots j_{\text{stop}}]$ [2306.05197]. The theoretical claim is precise: for any robot motion that is strictly faster than the motion recommended by the policy, there exists a human motion that results in a collision with the robot in a non-stationary state. The proof uses two theorems. The first establishes nested stoppable sets, $K_{j,i}\subseteq K_{k,i}$ for any $j0$ [2306.05197]. This formulation also clarifies a frequent misconception in safety-critical robotics. In general, it is impossible to avoid all human-robot collisions; the guarantee is instead that if a collision ever occurs, then the robot must have zero joint velocities at that instant. The controller is therefore time-optimal under the stated safety semantics, not a collision-elimination policy. In simulation, it is strictly less conservative than state-of-the-art safe robot control methods. For the 6-DoF robot simulation with a path from RRT, smoothed, $N\approx 517$ stages, $M=30$, dynamic obstacles at $1.6\,\text{m/s}$, and protective distance $=0$, the reported pre-computation improves from $4.03\pm 1.01\,\text{s}$ to $0.40\pm 0.09\,\text{s}$ using CPU multi-core plus GPU LP, while cycle time is $6\,\text{ms}$ on CPU and $5\,\text{ms}$ on GPU; the robot tracks the path, stops exactly at collision, then restarts, and never violates joint limits [2306.05197]. ## 3. Optimal-time stability in reinforcement learning A different TOP formulation appears in model-free reinforcement learning as “optimal-time stability.” The problem is posed on an MDP $(\mathcal S,\mathcal A,P,c,\rho)$, where mean-cost stability requires $$ \lim_{t\to\infty}\mathbb E[c_\pi(s_t)]=0, $$ and the optimization target is $$ \min_\pi \; \mathbb E_{\rho,P,\pi}\Big[\sum_{t=0}^\infty \gamma^t\,c_\pi(s_t)\Big] \quad\text{s.t.}\quad \lim_{t\to\infty}\mathbb E[c_\pi(s_t)]=0. $$ The paper’s interpretation is that minimizing the discounted cumulative cost under the stability constraint forces the system to reach equilibrium in the smallest possible cost-weighted time [2301.00521]. The theoretical device is a sampling-based Lyapunov theorem. It assumes a function $\mathcal L:\mathcal S\to\mathbb R_{\ge 0}$ and constants $\alpha,\beta,\lambda,k>0$ such that $$ \alpha\,c_\pi(s)\le \mathcal L(s)\le \beta\,c_\pi(s), $$ $$ \mathcal L(s)\ge c_\pi(s)+\lambda\,\mathbb E_{s'\sim P_\pi(\cdot\mid s)}[\mathcal L(s')], $$ and, in stationarity, $$ \mathbb E_{s\sim \mathcal U_\pi}\!\Big[\mathbb E_{s'\sim P_\pi(\cdot\mid s)}[\mathcal L(s')] - \mathcal L(s)\Big] \le -k\, \mathbb E_{s\sim \mathcal U_\pi}\!\Big[\mathcal L(s)-\lambda\,\mathbb E_{s'\sim P_\pi(\cdot\mid s)}[\mathcal L(s')]\Big]. $$ Under these conditions, the policy satisfies mean-cost stability. The paper further connects the strictness of the condition to a discrete analogue of finite-time stability [2301.00521]. The practical algorithm is Adaptive Lyapunov-based Actor-Critic (ALAC). It parameterizes a Lyapunov critic $\mathcal L_\theta(s,a)\ge 0$, uses a TD-style update toward $$ \mathcal L_\pi(s)=c_\pi(s)+\gamma\,\mathbb E_{s'\sim P_\pi}[\mathcal L_\pi(s')], $$ and defines a constraint-violation measure $$ \Delta\mathcal L_{\pi_\phi}(s,a) = \mathcal L_\theta(s',\pi_\phi(\cdot\mid s')) - \mathcal L_\theta(s,a) + k\Big[\mathcal L_\theta(s,a)-\lambda\,\mathcal L_\theta(s',\pi_\phi(\cdot\mid s'))\Big]. $$ A single Lagrange multiplier $\ell\ge 0$ enforces $\mathbb E[\Delta\mathcal L_{\pi_\phi}(s,a)]\le 0$, with adaptive updates $\lambda\leftarrow \min(\ell,\gamma)$ and $k\leftarrow 1-\ell$ [2301.00521]. Empirically, ALAC is evaluated on ten robotic control tasks, including Cartpole, Point-Circle, Swimmer, HalfCheetah, Ant, Humanoid, PyBullet Minitaur, and three free-floating space-robot planning tasks. Against SAC-cost, SPPO, LAC/LAC*, POLYC, LBPO, and TNLF, it attains the lowest cost and near-zero stability violation on all ten tasks. Under persistent action disturbance up to $0.5\times$ control range, its performance degrades far less than baselines, and under desired-goal shifts of $\pm 20\%$, only ALAC with error in state maintains near-optimal cost [2301.00521]. In this setting, TOP does not mean unconstrained minimum-time motion; it means policy optimization toward the fastest admissible stabilization compatible with the Lyapunov condition. ## 4. Timestamp optimization for humanoid standing manipulation In humanoid standing manipulation, TOP is introduced as a three-part framework consisting of a motion prior, a decoupled upper-/lower-body controller, and a Time Optimization Policy. The motion prior is a $\beta$-VAE trained on short upper-body motion clips $$ m_t = \{r_t\in\mathbb R^3,\;\theta_t\in SO(3)\to \mathbb R^6,\;q_t^{\text{upper}}\in\mathbb R^{n_j},\;\dot q_t^{\text{upper}}\in\mathbb R^{n_j}\}, $$ with $n_j=15$, stacked as $M_t=\{m_{t-W},\ldots,m_t,\ldots,m_{t+W}\}$ and $W=30$. The encoder maps $M_t$ to $(\mu_t,\sigma_t)$, samples $z_t\sim\mathcal N(\mu_t,\operatorname{diag}(\sigma_t^2))$, and the decoder reconstructs $M_t'$. The loss is $$ L_{\text{VAE}} = L_{\text{rec}} + \beta\cdot D_{KL}[\mathcal N(\mu_t,\sigma_t^2)\,\|\,\mathcal N(0,I)], $$ with $\beta=0.002$, four 1D-convolution layers with LayerNorm and ReLU, latent dimension $d_z=64$, and training for $30\,000$ epochs with batch $512$, learning rate $3\mathrm e{-3}$, and cyclical KL schedule [2508.00355]. Control is then decoupled. The upper body uses a PD controller, $$ \tau_t^{\text{up}} = k_p(q_t^{\text{ref}}-q_t)+k_d\cdot \dot q_t, $$ while the lower body uses a goal-conditioned policy $\pi_\phi(a_t\mid s_t,g_t)$ with $g_t=(m_t,z_t)$ and $$ s_t = \{q_t\in\mathbb R^{27},\;\dot q_t\in\mathbb R^{27},\;\text{base orientation }\theta_t,\;\text{base angular vel }\omega_t,\;\text{last action }a_{t-1},\;g_t\}. $$ The lower-body policy outputs $a_t\in\mathbb R^{12}$ and is trained with PPO using reward terms that penalize base linear and angular velocities, penalize COM drift through projected gravity, encourage “standing still” leg posture, impose foot contact and slip penalties, and regularize action rate and torque; convergence takes approximately $5$ hours on RTX4090 [2508.00355]. TOP acts on timing rather than torque. At each step $t$, given $(m_t,z_t,s_t)$ and a history embedding $h_t$, the TOP network $\pi_\theta$ outputs a horizon $$ \Delta t_t^{\text{seq}}=\{\Delta t_t,\Delta t_{t+1},\ldots,\Delta t_{t+n}\},\qquad n=10, $$ which is smoothed by exponential weighting $$ w_i=\frac{\exp(-k\cdot i)}{\sum_j \exp(-k\cdot j)},\qquad k=0.5, $$ to yield the actual $\Delta t$ applied at step $t$. The clip is then replaced by $$ m_t'=\text{LinearInterpolate}(m_{t-1},m_{t+(\Delta t_t)}), $$ with $\Delta t\in[0.01\,\text{s},0.1\,\text{s}]$. Training combines supervision and PPO, $$ L_{\text{sup}}=-\sum_i \log \pi_\theta(\Delta t_i^*\mid s_i),\qquad L_{\text{RL}}=-\mathbb E_t[\min(r_tA_t,\operatorname{clip}(r_t,1-\epsilon,1+\epsilon)A_t)], $$ $$ L_{\text{total}}=\lambda_{\text{sup}}L_{\text{sup}}+\lambda_{\text{RL}}L_{\text{RL}}, $$ with $\lambda_{\text{sup}}=0.1$ and $\lambda_{\text{RL}}=0.5$ [2508.00355]. At run time, the loop is $$ m_t \to z_t=E_\phi(M_t)\to \pi_\theta\Rightarrow \Delta t_t \Rightarrow m_t',\qquad m_t' \ \&\ z_t \to \text{upper PD + lower RL} \Rightarrow \tau_t. $$ The paper states that this closed-loop guarantees whole-body consistency and that TOP actively keeps $\Delta t$ in a regime where the Zero-Moment Point or Zero-Moment Line remains inside the support polygon; empirically, the ZMP-distance satisfies $d\le 0.32\,\text{m}$ at all times [2508.00355]. On $10\,000$ clips in simulation, the reported comparison is: Exbody, whole-body RL, TimeCost $15.0\,\text{s}$, success rate $92.5\%$, JPE $0.0376$, EEPE $0.0741$, ZMPProj $3.432$; Mobile-TV, decoupled, TimeCost $15.0\,\text{s}$, success rate $85.8\%$, JPE $0.0354$, EEPE $0.0513$, ZMPProj $3.831$; Ours (TOP), TimeCost $40.5\,\text{s}$, success rate $95.3\%$, JPE $0.0269$, EEPE $0.0270$, ZMPProj $2.729$ [2508.00355]. Inference overhead is reported as VAE $1\,\text{ms}$, RL policy $1.7\,\text{ms}$, and TOP policy $1.9\,\text{ms}$, for a total of approximately $5\,\text{ms}$ at $100\,\text{Hz}$ on RTX4090 or embedded CPU. ## 5. Adaptive rollout allocation in test-time policy optimization OptPO instantiates TOP in a non-physical setting: test-time adaptation of large language models. The setup assumes a prompt $x$, a policy $\pi_\theta(\cdot\mid x)$, and an extractor $\mathcal E$ mapping each sampled completion $Y$ to one of $m$ discrete answer tokens $\mathcal A=\{1,\dots,m\}$. The goal is twofold: efficiently decide on a consensus pseudo-label by sequentially sampling rollouts $D_t=\mathcal E(Y_t)$, and use the retained samples for a single on-policy update via PPO, GRPO, or related methods, without requiring ground-truth labels [2512.02882]. The core mechanism is a Bayesian sequential probability ratio test between the current leader $L(t)$ and runner-up $S(t)$. Under hypothesis $H_j$, each vote equals $j$ with probability $p_0$ and other classes absorb total probability $1-p_0$. With vote counts $v_j(t)$, the likelihood under $H_j$ is $$ p(D_{1:t}\mid H_j)=p_0^{\,v_j(t)}\prod_{c\ne j}p_c^{\,v_c(t)}, $$ and the posterior follows by Bayes’ rule. The Bayes-factor is $$ \Lambda_t=\frac{P(D_{1:t}\mid H_L)}{P(D_{1:t}\mid H_S)}, \qquad \log\Lambda_t=\Delta_t\log\kappa, $$ with $$ \Delta_t=v_L(t)-v_S(t),\qquad \kappa=\frac{p_0(m-1)}{1-p_0}>1. $$ Given type-I and type-II error budgets $\alpha,\beta\in(0,1)$, Wald thresholds are $$ A=\frac{1-\beta}{\alpha},\qquad B=\frac{\beta}{1-\alpha}, $$ and the equivalent stopping gaps are $$ \Delta_A=\Bigl\lceil \frac{\log A}{\log\kappa}\Bigr\rceil,\qquad \Delta_B=\Bigl\lfloor \frac{\log B}{\log\kappa}\Bigr\rfloor. $$ Sampling stops as soon as $\Delta_t\ge \Delta_A$ or $\Delta_t\le \Delta_B$, otherwise continuing until a hard cap $M$ [2512.02882]. After stopping at $t=\tau$, the method retains at least $N\le \tau$ samples and assigns them the consensus pseudo-label $a^\star=L(\tau)$. A PPO-style update uses $$ L^{\mathrm{PPO}}(\theta)=\mathbb E_{i=1,\dots,N}\Big[\min(r_i(\theta)\hat A_i,\operatorname{clip}(r_i(\theta),1-\varepsilon,1+\varepsilon)\hat A_i)\Big], $$ where $$ r_i(\theta)=\frac{\pi_\theta(Y_i\mid x)}{\pi_{\text{ref}}(Y_i\mid x)},\qquad \hat A_i=R(Y_i;a^\star)-b_i, $$ and $R(Y_i;a^\star)=\mathbf 1\{\mathcal E(Y_i)=a^\star\}$ is the $0/1$ reward or a smoothed version [2512.02882]. The efficiency claim is classical and specific. Under simple two-point models, SPRT minimizes the average number of rollouts for fixed type-I and type-II errors, and the expected stopping time is approximated by $$ E[\tau\mid H_1]\approx \frac{\log A}{D(P(\cdot\mid H_1)\,\|\,P(\cdot\mid H_0))},\qquad E[\tau\mid H_0]\approx \frac{\log(1/B)}{D(P(\cdot\mid H_0)\,\|\,P(\cdot\mid H_1))}. $$ Empirically, on AIME, AMC, MATH-500, and GPQA with Qwen-Math-1.5B, Qwen-7B, and Llama-3.2-1B backbones, OptPO achieves $30$–$50\%$ token savings versus fixed-budget TTRL baselines at equal or better mean@16 accuracy. The reported examples include GPQA with Qwen-Math-1.5B+PPO, where TTRL-PPO uses $262\,\text{M}$ tokens with mean@16 $=25.9\%$ while OptPO-PPO uses $144\,\text{M}$ tokens with mean@16 $=26.6\%$, and MATH-500 with Qwen-7B+GRPO, where mean@16 is approximately $79.5\%$ while tokens drop from $111\,\text{M}$ to $56\,\text{M}$ [2512.02882]. Here, TOP refers to optimal rollout allocation, not to trajectory retiming in physical space. ## 6. Rolling-horizon dispatch, virtual lookahead, and interpretive boundaries In dynamic team orienteering for spatial crowdsourcing, TOP appears as an event-driven rolling-horizon policy for the Dynamic Team Orienteering Problem in Spatial Crowdsourcing (DTOP-SC). Workers travel along fixed origin-destination trips under hard time budgets, tasks arrive online with release times and time windows, and at each decision epoch the dispatcher solves a deterministic static subproblem over currently available tasks and idle workers. The static snapshot is an HT-TOPTW with binary routing variables $x_{ij}^w$, service indicators $y_i^w$, and service start times $a_i^w$, maximizing $$ \max \sum_{w\in W_{\mathrm{idle}}(t)}\sum_{i\in A(t)} p_i\, y_i^w $$ subject to route-continuity and time-window constraints [2601.11010]. The scenario-sampling rolling-horizon framework mitigates myopic bias by augmenting each planning epoch with sampled virtual tasks. At each epoch, $S$ independent futures are generated: virtual task locations are sampled in the bounding box of current task and worker locations, profits and durations are drawn from empirical ranges, and time-window bounds are drawn from $[t,H]$. For each scenario, the augmented task set $\tilde A^{(s)}(t)=A(t)\cup V^{(s)}(t)$ is solved by ALNS, candidate real tasks are extracted from the resulting routes, and conflict-free assignments are selected from the union of candidates using frequencies $f_t(w,i)$ and the threshold $\theta_{\min}=\lfloor \alpha S\rfloor$ with $\alpha=0.2$ [2601.11010]. The static subproblems are solved with an ALNS à la Ropke–Pisinger using random removal, worst-cost removal, and Shaw removal; greedy insertion, regret-2 insertion, and regret-3 insertion; local search by intra-route 2-opt, inter-route relocate, and inter-route swap; adaptive operator weighting; and simulated annealing acceptance. The reported parameter settings are $I_{\text{init}}=1000$, $I_{\text{scen}}=100$, $S=15$, $N^{\text{vir}}=5$, and $\alpha=0.2$ [2601.11010]. Computationally, the reported results separate benchmark and map-based settings. On the DTOP benchmark with high dynamism, Scen-RH-ALNS obtains mean profit $74.33$ versus $74.92$ for MPAd and $76.81$ for MPAc, with gaps $0.8\%$ versus MPAd and $3.2\%$ versus MPAc, average decision time approximately $0.14\,\text{s}$, instance-wise best on $316/387$ high-dynamism instances and $708/1161$ overall, and new BKs on $115$ instances. On map-based DTOP-SC, the Base family reports a gap to the $600\,\text{s}$ MIP incumbent of $5.6\%$ on average with standard deviation $4.8\%$ and time approximately $19\,\text{s}$; scaling from $(5,50)$ to $(15,150)$ workers/tasks, the gap decreases from $5.2\%$ to $-23.6\%$ while time grows from approximately $4\,\text{s}$ to approximately $60\,\text{s}$ [2601.11010]. These formulations delineate the interpretive boundaries of TOP. First, “time-optimal” does not mean “always move or decide as fast as possible”: in ISO-safe tracking, any faster motion than the recommended one admits a human motion leading to a non-stationary collision [2306.05197]. Second, TOP does not necessarily assume perfect foresight: the ISO-safe controller requires only current minimum distance and obstacle $v_{\max}$, not detailed motion prediction; the DTOP-SC policy uses virtual tasks that are never executed and only influence routing; OptPO uses posterior confidence rather than labels from an oracle [2306.05197] [2601.11010] [2512.02882]. Third, the acronym is domain-specific. Some papers present TOP as a named method, while others contribute key building blocks for what one summary calls the emerging field of Time Optimization Policy [2301.00521]. The common denominator is therefore methodological rather than taxonomic: time is elevated to a controlled resource, and admissibility is encoded through safety, stability, balance, statistical confidence, or routing feasibility constraints. Source: https://www.emergentmind.com/topics/time-optimization-policy-top