Optimal Restocking Policy Overview

• Optimal restocking policy is a family of decision rules that determine when to replenish inventory, how much to order, and when to reset to optimize costs or profits.
• These policies integrate complex dynamics including network flows, cash constraints, perishability, and route recourse, often using threshold-based rules or reinforcement learning.
• Advanced computational methods like binary dynamic search, MILP formulations, and hybrid A3C-DPPO algorithms efficiently solve these non-convex, high-dimensional control problems.

Optimal restocking policy denotes a decision rule that determines when replenishment occurs, how much is ordered, and, in richer formulations, whether inventory is reset, redistributed, routed through a network, or financed externally, so as to optimize an objective such as expected total cost, discounted cost, long-run average cost, or profit. The state on which the policy acts may be a scalar inventory position, an augmented state $(x,t)$ with time since reset, a cash–inventory pair $(I,y)$, a route-execution state with residual vehicle capacity, or an age-structured perishable-inventory vector; correspondingly, optimal policies range from classical $(s,S)$ and $(R,s,S)$ rules to four-threshold reset policies, trigger sets, barrier policies, large-scale mixed-integer formulations, and continuous-action MDP policies learned by deep reinforcement learning (Visentin et al., 2020, Lee et al., 2022, Kaur et al., 4 Jun 2026).

1. Formal scope and canonical formulations

The term covers several mathematically distinct control problems. In single-location inventory control, the policy usually maps current inventory to an order quantity or to an order-up-to level. In reset-control models, the action additionally includes a binary reset decision. In networked settings, the policy specifies shipment, transshipment, or package-allocation flows across facilities. In routing models, restocking is a recourse rule embedded in route execution. In multi-echelon pharmaceutical models, the policy is an MDP over inventories, lead times, and age buckets. In joint replenishment, it is a sequence of item-specific and joint order times minimizing long-run average cost.

Model family	State/action structure	Optimal-policy representation
Periodic-review single item	inventory $x$, order-up-to $y$ or order $Q_t$	non-stationary $(s_t,S_t)$, or $(R,s,S)$
Reset-control inventory	$(x_n,t_n)$, actions $(I,y)$0	four-threshold reset/restock rule
Two-echelon retail network	flows $(I,y)$1, final stocks $(I,y)$2	MILP solution under CR, DR, or GR
VRPSD recourse	route position, last served customer, residual load	optimal preventive restocking $(I,y)$3, switch policy $(I,y)$4
Pharmaceutical supply chain	age-structured inventories, lead times, continuous actions	hybrid A3C DPPO policy
Continuous/discrete JRP	sequences of order times and quantities	$(I,y)$5-approximate dynamic policy

Representative formulations include the single-period transferring problem for retail redistribution, periodic barrier replenishment under spectrally positive Lévy demand, cash-flow–based inventory control with loans and deposits, route-recouse DPs for the VRPSD, age-structured MDPs for pharmaceutical supply chains, and continuous/discrete joint replenishment models with joint setup cost $(I,y)$6 (González-Díaz et al., 2024, Pérez et al., 2018, Katehakis et al., 2015, Florio et al., 2022, Segev, 23 Jun 2025).

2. Threshold-based single-location policies

In the finite-horizon, periodic-review, single-item inventory problem with non-stationary random demand and fixed ordering cost $(I,y)$7, the exact dynamic program is

$(I,y)$8

and optimal non-stationary $(I,y)$9 parameters satisfy

$(s,S)$0

The recursion-free approximation

$(s,S)$1

preserves this structure and yields average optimality gaps of $(s,S)$2 under moderate uncertainty and $(s,S)$3 under high uncertainty (Kilic et al., 2020).

When review itself is a decision, the policy generalizes to a non-stationary $(s,S)$4 policy over a finite horizon. With review indicators $(s,S)$5, review cost $(s,S)$6, fixed ordering cost $(s,S)$7, zero lead time, and stochastic non-stationary demand, the optimal review schedule and the optimal $(s,S)$8 parameters are computed jointly. Operationally, at each review instant, inventory is checked and, if it is $(s,S)$9, ordered up to $(R,s,S)$0; otherwise, no order is placed. This adds a timing layer to the classical order/no-order threshold structure (Visentin et al., 2020).

In cash-flow–based dynamic inventory management, the relevant state is $(R,s,S)$1, where $(R,s,S)$2 is inventory and $(R,s,S)$3 is capital position measured in purchasable units, with net worth $(R,s,S)$4. The optimal ordering policy is characterized by two thresholds $(R,s,S)$5 and takes the form

$(R,s,S)$6

The three regions correspond to under-utilization, full-utilization, and over-utilization of internal cash and debt capacity (Katehakis et al., 2015).

A different threshold variable appears in finite-horizon purchasing under a mean-reverting price process. There, one must buy a fixed quantity by a deadline, and the optimal policy is to buy at time $(R,s,S)$7 iff the observed price satisfies $(R,s,S)$8. The threshold sequence is monotonically increasing in time, and at the last decision epoch

$(R,s,S)$9

Here the threshold is not an inventory level but a price boundary induced by the option value of waiting under mean reversion (Dourban et al., 2017).

3. Augmented-state policies: resets, trigger sets, and peak control

Inventory systems with periodic and controllable resets augment the state by the age since last reset. With state $x$0, continuous action $x$1, binary reset action $x$2, and a hard constraint $x$3, the Bellman recursion compares a reset branch $x$4 against a no-reset branch $x$5. Because $x$6 is concave in $x$7 while the no-reset continuation is convex under the paper’s assumptions, the overall value function is non-convex, and standard convexity or $x$8-convexity arguments do not directly apply. Under the stated sufficient conditions, the optimal policy has the four-threshold structure

$x$9

which generalizes the classical $y$0 rule by adding low-inventory and high-inventory reset regions (Lee et al., 2022).

For industrial vending machines, the replenishment timing problem is formulated as an average-cost optimal stopping problem over the multi-item state $y$1. The key index is

$y$2

and the optimal policy is a trigger-set rule: $y$3 The same setting also admits an optimal fixed-cycle policy with unique cycle length $y$4 characterized by

$y$5

In numerical studies, optimal fixed cycle replenishment reduces costs by $y$6 to $y$7 compared to current practice, and the online control framework delivers an additional $y$8 to $y$9 improvement (Sindermann et al., 17 Mar 2025).

Peak-aware inventory control augments the state with

$Q_t$0

so that the objective includes an $Q_t$1 term $Q_t$2. The auxiliary state converts peak inventory into a terminal cost and yields a classical HJB formulation for the smoothed problem. The resulting control is a continuous-time feedback law in $Q_t$3, not a simple reorder-point rule. Numerical results show that peak inventory can be minimized with negligible revenue loss under $Q_t$4, whereas without peak-control the peak levels are significantly higher (Dhiman et al., 2024).

4. Networked, routing, and joint replenishment formulations

In a two-echelon retail network, optimal restocking is a single-period redistribution problem over warehouses $Q_t$5, outlets $Q_t$6, SKUs $Q_t$7, package types $Q_t$8, and allowed arcs $Q_t$9. The MILP minimizes transportation cost, penalty for unmet variable demand, and a tie-breaker on item transfers, with decision variables $(s_t,S_t)$0 for item flows, $(s_t,S_t)$1 for package flows, and $(s_t,S_t)$2 for final inventories. Changing $(s_t,S_t)$3 yields centralized redistribution (CR), decentralized redistribution (DR), and general redistribution (GR) within the same formulation. In the numerical study, GR is always best by construction; when warehouse-related shipping is cheap, CR performs as well as GR, while when warehouse legs are expensive, DR dominates. The tipping point is around a $(s_t,S_t)$4 discount on warehouse-involving postage rates (González-Díaz et al., 2024).

In the vehicle routing problem with stochastic demands, optimal restocking is a recourse policy embedded in route execution. Under $(s_t,S_t)$5, for a given next customer $(s_t,S_t)$6, current node $(s_t,S_t)$7, residual load $(s_t,S_t)$8, and downstream value function $(s_t,S_t)$9, the elementary recourse decision is

$(R,s,S)$0

where $(R,s,S)$1 is the direct-visit cost and $(R,s,S)$2 is the cost of preventive restocking before visiting $(R,s,S)$3. The switch policy $(R,s,S)$4 enlarges the state to $(R,s,S)$5 and permits adjacent customer swaps. On fixed planned routes, moving from detour-to-depot to optimal restocking yields about $(R,s,S)$6–$(R,s,S)$7 savings on average, while moving from optimal restocking to the switch policy yields about $(R,s,S)$8–$(R,s,S)$9 extra savings on average; when both policies are optimized a priori, the average improvement of the switch policy is only about $(x_n,t_n)$0 (Florio et al., 2022).

For the VRPSD under optimal restocking (OR), the recourse function satisfies superadditivity under concatenation: $(x_n,t_n)$1 This property is the necessary and sufficient condition for validity of the disaggregated integer L-shaped method. It supports a DL-shaped algorithm with new dynamic-programming-based lower bounds and E-cuts that generalize P-cuts and S-cuts. Computationally, the new valid inequalities speed up computations by an order of magnitude, and the algorithm solves several open instances to optimality, including $(x_n,t_n)$2 single-vehicle instances (Legault et al., 7 Aug 2025).

In continuous/discrete-time joint replenishment, the policy is a collection of order-time sequences for $(x_n,t_n)$3 items sharing a joint setup cost $(x_n,t_n)$4. The central result is an EPTAS: every continuous-time infinite-horizon instance can be reduced to a corresponding discrete-time $(x_n,t_n)$5-period instance while incurring a multiplicative optimality loss of at most $(x_n,t_n)$6, and the resulting discrete problem admits a $(x_n,t_n)$7-approximation in time

$(x_n,t_n)$8

The output is a compactly-encoded replenishment policy within factor $(x_n,t_n)$9 of the dynamic optimum (Segev, 23 Jun 2025).

5. Algorithmic solution methods

The non-convex reset-control model admits an exact computational scheme through the Binary Dynamic Search (BiDS) algorithm. Rather than iterating on an entire value function, BiDS searches for the scalar reset-state value $(I,y)$00: for a candidate $(I,y)$01, it computes $(I,y)$02 by backward induction and evaluates the fixed-point map $(I,y)$03; binary search on $(I,y)$04 continues until $(I,y)$05 to tolerance. The method is much less sensitive to discount factor than standard value iteration and exploits the fact that all resets return the system to a single state $(I,y)$06 (Lee et al., 2022).

For non-stationary $(I,y)$07 policies, the hybrid branch-and-bound plus stochastic dynamic programming method searches over review plans $(I,y)$08 and solves an SDP for each surviving suffix. Dynamic-programming lower bounds support pruning, and up to $(I,y)$09 of the search tree is pruned in some configurations. In numerical experiments, the method solves almost twice as many periods as exhaustive SDP enumeration within similar time budgets (Visentin et al., 2020).

For the non-stationary $(I,y)$10 problem without review decisions, the recursion-free approximation replaces the stochastic DP by convex cycle-cost functions and a deterministic shortest-path computation over replenishment cycles. This avoids direct backward recursion on the value function and is effective across horizons of $(I,y)$11–$(I,y)$12 periods and both moderate and high demand uncertainty (Kilic et al., 2020).

In the retail-network redistribution problem, exact solution of the transfer MILP is called T–P, while the scalable approximation uses Relaxed Transferring–Rounding–Packing (RT–R–P). RT relaxes item-flow integrality, R solves SKU-wise minimum-cost flow rounding problems with totally unimodular constraints, and P solves arc-wise bin-packing MILPs. In extra-large tests with a $(I,y)$13-minute limit, RT handles up to $(I,y)$14 million variables with all $(I,y)$15 instances solved per group, whereas direct T solves reliably only up to $(I,y)$16 million variables (González-Díaz et al., 2024).

In pharmaceutical supply chains, the hybrid A3C-DPPO algorithm addresses continuous action spaces and high-dimensional multi-echelon states. The method combines asynchronous local actor-critic learning with global PPO-style clipped updates and gradient aggregation. In synthetic experiments it converges to near-optimal rewards in about $(I,y)$17 episodes, whereas PPO and DQN need $(I,y)$18 episodes. Under a Gamma demand distribution with $(I,y)$19, its average rewards are $(I,y)$20 higher than PPO and $(I,y)$21 higher than DQN (Kaur et al., 4 Jun 2026).

6. Coupled decisions, limitations, and recurring misconceptions

Optimal restocking is often coupled to decisions that are not inventory quantities. In dynamic product assembly and inventory control, the purchasing rule

$(I,y)$22

is solved jointly with a pricing decision that maximizes, for each product $(I,y)$23,

$(I,y)$24

The resulting JPP policy is $(I,y)$25-optimal in time-average profit with $(I,y)$26 inventory bounds and is robust to non-ergodic dynamics (Neely et al., 2010).

A common misconception is that optimal restocking must be expressible through convexity arguments of the standard $(I,y)$27 type. The reset-control model shows the opposite: once the Bellman recursion becomes

$(I,y)$28

with a concave reset branch and a convex continuation branch, the minimum is generally non-convex and classical $(I,y)$29-convexity is no longer directly applicable (Lee et al., 2022).

Another misconception is that route-based recourse validity follows from monotonicity alone. In the VRPSD, validity of the DL-shaped method requires superadditivity under concatenation; monotonicity over subsequences is insufficient, and the paper explicitly rectifies an incorrect argument from earlier work (Legault et al., 7 Aug 2025).

Average-cost formulations can also obscure operational constraints tied to peaks rather than averages. Peak-aware inventory control shows that optimizing weighted averages of holding, shortage, or revenue terms need not control the maximum inventory level; adding an $(I,y)$30 term changes the state space and the optimal control law, and numerical results show peak inventory can be reduced with negligible revenue loss under $(I,y)$31 (Dhiman et al., 2024).

Finally, dynamic joint replenishment still exhibits what the paper calls “profound gaps in our structural understanding of optimal such policies.” The new EPTAS closes the approximation gap, not the structural one: it proves that arbitrarily accurate near-optimal dynamic policies can be computed efficiently, but it does not reduce the exact dynamic optimum to a simple closed-form rule (Segev, 23 Jun 2025).

This suggests that “optimal restocking policy” is best understood not as a single canonical rule but as a family of state-dependent control laws whose form is determined by the modeling primitives: review timing, reset options, network topology, route recourse, perishability, cash, pricing, and the distinction between average and peak objectives.