Stochastic Cyclic Inventory Routing with Supply Uncertainty: A Case in Green-Hydrogen Logistics (2212.10532v3)

Abstract: Hydrogen can be produced from water, using electricity. The hydrogen can subsequently be kept in inventory in large quantities, unlike the electricity itself. This enables solar and wind energy generation to occur asynchronously from its usage. For this reason, hydrogen is expected to be a key ingredient for reaching a climate-neutral economy. However, the logistics for hydrogen are complex. Inventory policies must be determined for multiple locations in the network, and transportation of hydrogen from the production location to customers must be scheduled. At the same time, production patterns of hydrogen are intermittent, which affects the possibilities to realize the planned transportation and inventory levels. To provide policies for efficient transportation and storage of hydrogen, this paper proposes a parameterized cost function approximation approach to the stochastic cyclic inventory routing problem. Firstly, our approach includes a parameterized mixed integer programming (MIP) model which yields fixed and repetitive schedules for vehicle transportation of hydrogen. Secondly, buying and selling decisions in case of underproduction or overproduction are optimized further via a Markov decision process (MDP) model, taking into account the uncertainties in production and demand quantities. To jointly optimize the parameterized MIP and the MDP model, our approach includes an algorithm that searches the parameter space by iteratively solving the MIP and MDP models. We conduct computational experiments to validate our model in various problem settings and show that it provides near-optimal solutions. Moreover, we test our approach on an expert-reviewed case study at two hydrogen production locations in the Netherlands. We offer insights for the stakeholders in the region and analyze the impact of various problem elements in these case studies.

• The paper introduces a novel cyclic inventory routing model that integrates a tactical MIP for transportation scheduling with an operational MDP for dynamic purchasing.
• It demonstrates how parameterized cost adjustments can smooth daily demand variations and reduce total logistics costs under intermittent renewable supply.
• Computational experiments show significant cost savings and practical insights into managing supply and demand uncertainties in green-hydrogen logistics.

This paper, "Stochastic Cyclic Inventory Routing with Supply Uncertainty: A Case in Green-Hydrogen Logistics" (2212.10532), addresses the complex problem of planning hydrogen distribution from a producer with stochastic supply (due to intermittent renewable energy sources) to geographically dispersed customers with stochastic demand. The core challenge lies in jointly optimizing two distinct decision levels: a static, tactical-level transportation delivery schedule that repeats cyclically, and dynamic, operational-level purchasing decisions at the producer to manage inventory and buffer supply/demand uncertainties.

The problem, termed the Stochastic Cyclic Inventory Routing Problem (SCIRP) with supply uncertainty, is motivated by the emerging green hydrogen economy where hydrogen acts as an energy storage medium. Efficient logistics are crucial, but intermittent renewable energy production introduces significant supply uncertainty, which is often overlooked in traditional Inventory Routing Problems (IRPs).

The paper proposes a novel solution approach based on iteratively solving a parameterized Mixed Integer Programming (MIP) model for the tactical transportation schedule and a Markov Decision Process (MDP) model for the dynamic purchasing decisions.

1. Tactical-Level MIP: This model uses a set-partitioning formulation where candidate "clusters" of customers are generated. Each cluster defines a set of customers, a fixed vehicle route, a set of delivery periods within a cycle (e.g., a week), and base-stock levels for each customer on delivery days. The base-stock levels are set to achieve a target service level ($\alpha$) at customers. The MIP selects a subset of these clusters that partition all customers while minimizing the total expected cyclic cost. This cost includes fixed and variable transportation costs, expected inventory holding costs at customers, and expected emergency shipment costs (when vehicle capacity is insufficient). Chance constraints ensure customer service levels and limit the probability of exceeding vehicle capacity.
2. Operational-Level MDP: Given a fixed transportation schedule from the MIP, the producer faces a dynamic problem of managing inventory day-by-day. At the start of each period, the producer observes current inventory and the total stochastic demand imposed by the selected clusters' delivery schedule for that period. They also observe the stochastic supply realization. Based on this, the producer decides how much hydrogen to buy from or sell to an external market at fixed prices, subject to storage capacity limits. An emergency purchase cost is incurred if inventory is negative before deliveries. This dynamic decision problem is modeled as an MDP, and the optimal policy is found using value iteration. The expected cycle cost of this dynamic policy is then calculated.
3. Joint Optimization via Parameterized CFA: The key innovation is linking the static MIP and dynamic MDP. The tactical MIP's objective function is augmented with terms penalizing undesirable characteristics of the resulting demand patterns faced by the producer over the cycle. Specifically, parameters $\eta_1$ and $\eta_2$ scale penalties on the deviation of the mean and variance of the total demand faced by the producer on each day from the average mean/variance across the cycle. By adjusting these parameters, the MIP is steered towards transportation schedules that lead to lower costs in the subsequent dynamic purchasing problem (e.g., by smoothing demand over the week).
4. Iterative Solution Algorithm: The paper proposes a Line Search algorithm to find effective values for parameters $(\eta_1, \eta_2)$. Starting from small values, the algorithm iteratively adjusts one parameter at a time (either $\eta_1$ or $\eta_2$). In each step, it solves the parameterized MIP, takes the resulting tactical solution, solves the corresponding MDP via value iteration, simulates the MDP policy to get the total cost, and updates the parameters if a lower total cost is found. The search continues until no improvement is observed or bounds are reached.

The paper makes several contributions:

• It introduces and solves the SCIRP with supply uncertainty, specifically for green hydrogen logistics.
• It generalizes existing SCIRP literature by including cyclic schedules and stochastic supply simultaneously.
• It proposes a generic solution approach for jointly optimizing static (MIP) and dynamic (MDP) decisions using parameterized CFA, applicable potentially beyond IRPs.
• It demonstrates significant cost savings (up to 191.5% in experiments) compared to a step-by-step approach that optimizes the tactical level assuming infinite supply.
• It evaluates the approach on a real-world case paper in the Northern Netherlands, providing insights into hydrogen distribution under various future scenarios and analyzing the impact of supply/demand uncertainty.

Computational experiments validate the Line Search algorithm, showing it finds near-optimal solutions quickly compared to a full grid search over the parameters. The case paper analyzes scenarios with different numbers of customers, demand/supply volumes, capacities, and uncertainty levels. Key findings from the case paper highlight:

• Increased vehicle capacity and customer density reduce transportation costs per unit.
• Reduced supply and demand uncertainty significantly decrease dynamic purchasing costs.
• Imbalances between mean supply and demand lead to higher purchasing costs, especially if supply is lower than demand.
• Supply uncertainty has a greater impact on purchasing costs than demand uncertainty in the scenarios studied.

The paper concludes that jointly optimizing tactical and operational decisions is crucial for efficient hydrogen logistics, particularly in the early stages of market development with high uncertainty. The parameterized CFA approach provides a flexible framework for balancing these decisions. Future work could explore heterogeneous fleets, non-stationary stochastic processes, and integrating weather forecasting for dynamic supply prediction.