Industrial Benchmark (IB): Synthetic RL Testbed

• Industrial Benchmark (IB) is an open-source, synthetic reinforcement learning environment that mimics industrial control challenges with continuous actions, partial observability, delayed effects, and heteroscedastic noise.
• IB supports diverse research applications including system identification, forecasting, transfer learning, and offline RL by exposing hidden state dynamics and complex latency in benchmark tasks.
• Studies using IB have highlighted the need for methods leveraging observation histories to capture delayed consequences, as optimal policies often require periodic, non-static control strategies.

The Industrial Benchmark (IB) is an open-source, synthetic reinforcement-learning benchmark designed to reproduce the hardness profile of industrial control and optimization problems without approximating any single real plant. It was introduced to bridge the gap between freely available academic benchmarks and the continuous, partially observed, delayed, stochastic, and multi-objective conditions encountered in industrial applications. Across its introductory and detailed descriptions, IB is presented as a benchmark for industrially motivated control, system identification, forecasting, transfer learning, and batch/offline RL, with public Java and Python implementations and an OpenAI Gym wrapper (Hein et al., 2016, Hein et al., 2017).

1. Origin and intended role

IB was motivated by a recurring methodological problem: real-world industrial testing is expensive, risky, and slow, whereas many standard RL benchmarks are low-dimensional, fully observed, weakly stochastic, or otherwise too simple to capture industrial difficulty. The benchmark was therefore designed to include what its authors describe as aspects “vital in industrial applications,” while remaining generic rather than tied to a single physical process (Hein et al., 2016, Hein et al., 2017).

The environment is explicitly not a faithful model of any specific plant. Instead, it is meant to reproduce the kinds of hardness seen in applications such as gas turbines, wind turbines, power plants, and process-industry systems: continuous state and action spaces, partial observability, latent internal state, delayed and smeared action effects, heteroscedastic stochasticity, exogenous operating conditions, and conflicting optimization criteria (Hein et al., 2017). This positioning is central to the benchmark’s identity. A common misconception is that IB is a plant simulator; the papers state the opposite. It is a deliberately synthetic environment whose value lies in its difficulty structure rather than in physical fidelity (Hein et al., 2016).

The introductory paper also emphasizes that IB is useful beyond policy learning. The same environment is presented as relevant for regression, forecasting, system identification, transfer learning, active learning, feature selection, and change detection (Hein et al., 2016). This broader scope follows directly from the fact that the benchmark exposes hidden-state dynamics, delayed responses, and structured noise rather than only a reward-maximization problem.

2. Interface, state variables, and reward formulation

At each time step, the agent chooses a continuous three-dimensional action in $[-1,1]^3$, interpreted as changes to three steerings: velocity $v$, gain $g$, and shift. The introductory description denotes the third steering by $s$, whereas the later detailed paper writes it as $h$; both versions clip the updated steerings to $[0,100]$ and use unequal actuation scales, with $d^{\mathrm v}=1$, $d^{\mathrm g}=10$, and $d^{\mathrm h}\approx 5.75$ in the later formulation (Hein et al., 2016, Hein et al., 2017).

The observable vector has six components,

$o_t = (v_t,g_t,h_t,p_t,c_t,f_t)$

in the later notation, or equivalently $v$0 in the introductory paper. Here $v$1 is an observable but uncontrollable set point or external driver, $v$2 is consumption, and $v$3 is fatigue (Hein et al., 2016, Hein et al., 2017). The exogenous set point is important because IB is not an autonomous regulation problem with a fixed optimum; operating regime depends on $v$4.

The reward formulation differs across the two benchmark descriptions. The introductory paper defines reward as

$v$5

whereas the later detailed paper specifies

$v$6

Both versions frame IB as a cost-minimization problem built from consumption and fatigue, but the later paper weights fatigue three times more strongly (Hein et al., 2016, Hein et al., 2017). This discrepancy is best understood as a documentation/version difference rather than a conceptual disagreement about the benchmark’s purpose.

The benchmark is naturally represented as transition tuples of the form

$v$7

which made it a natural testbed for early batch/offline RL studies (Hein et al., 2016).

3. Internal dynamics and sources of difficulty

IB is built from three interacting sub-dynamics: operational cost, mis-calibration, and fatigue (Hein et al., 2016). The operational-cost component depends on set point, velocity, and gain through an exponential term. In the introductory formulation,

$v$8

and this signal is not observed directly; instead, it is delayed and smeared by a finite convolution over values from $v$9 to $g$0, with the largest weight at $g$1 (Hein et al., 2016). The later paper uses the same structure with $g$2 in place of $g$3 (Hein et al., 2017). This delay-smearing mechanism is one of the benchmark’s main credit-assignment obstacles.

Mis-calibration couples the external set point to the shift steering. The introductory paper defines an effective shift

$g$4

and states that mis-calibration depends on three latent variables $g$5, but it does not provide the explicit latent update equations (Hein et al., 2016). The later detailed paper replaces this abstract presentation with a fully specified cyclic latent system involving domain, response, and direction-index variables $g$6, together with a linearly biased Goldstone potential (Hein et al., 2017). That later paper explicitly states that the benchmark is designed so that the optimal policy does not correspond to a fixed point in the steerings, and that the optimal policy in the mis-calibration subsystem is approximately sinusoidal (Hein et al., 2017).

Fatigue is the second reward-relevant term and creates the benchmark’s multi-criteria tradeoff. In both descriptions it depends on set point, velocity, and gain, with a basic fatigue term of the form

$g$7

followed by an amplification mechanism driven by transformed controls, stochastic variables, and latent fatigue state (Hein et al., 2016, Hein et al., 2017). The later paper formalizes two latent fatigue variables, $g$8 and $g$9, whose thresholded dynamics can enter a high-fatigue regime once a threshold of $s$0 is crossed (Hein et al., 2017). This creates a hidden bifurcation-like effect resembling industrial stress or wear escalation.

Observed consumption adds heteroscedastic Gaussian noise to the noiseless cost: $s$1 The observation noise standard deviation therefore increases with the signal magnitude (Hein et al., 2016). Together, delayed operational cost, latent mis-calibration, latent fatigue accumulation, and state-dependent noise make IB difficult in precisely the way it was intended to be difficult: the agent observes only a compressed projection of a nonlinear, history-dependent process.

4. Partial observability, regimes, and standard protocols

IB is only Markovian in an augmented internal state. The introductory paper describes a preferred minimal Markovian state with 20 variables: the 6 observable variables, 9 delayed operational-cost values, 3 mis-calibration latents, and 2 fatigue latents (Hein et al., 2016). The later paper gives the same 20-variable count in updated notation, with hidden operational-cost history, mis-calibration variables, and fatigue variables augmenting the observation vector (Hein et al., 2017). The direct implication is that the observable state is not Markovian, so history windows, recurrent policies, or latent-state estimators are natural methodological choices.

The set point $s$2 defines operating regimes. In the Constant Set Point Setting, the introductory paper uses

$s$3

and notes that constant-set-point experiments should be viewed not as one single POMDP but as a family of POMDPs parameterized by set point (Hein et al., 2016). A Variable Set Point Setting is also described, in which changes in $s$4 are externally driven rather than action-driven (Hein et al., 2016, Hein et al., 2017). This makes transfer across operating regimes an intrinsic part of the benchmark’s conceptual design.

For initial offline evaluation, the introductory paper recommends a maximum entropy policy, uniformly random over the action domain. The standard initialization is three steerings at $s$5, default latent variables, and ten set points $s$6; running the random policy for 1000 time steps per set point yields 10,000 data points (Hein et al., 2016). That same paper reports early predictive-modeling baselines using recurrent neural networks, with MRABD of approximately $s$7 for reward, approximately $s$8 for consumption, and approximately $s$9 for fatigue when $h$0 (Hein et al., 2016).

This experimental framing explains why IB became closely associated with batch/offline RL. The benchmark was designed under the industrially motivated assumption that exploration may be unsafe or expensive, while logged data remain available. That assumption later became central to published control studies on the environment (Hein et al., 2017).

5. Batch RL studies and characteristic control behavior

An early extensive study of batch/off-policy RL on IB compared three approaches: Particle Swarm Optimization Policy (PSO-P), Recurrent Control Neural Network (RCNN), and Neural Fitted Q-Iteration (NFQ) (Hein et al., 2017). In that study, the offline dataset contained 100,000 transitions, generated from ten set points, ten initializations per set point, and trajectories of length 1000 (Hein et al., 2017). A learned recurrent system model with separate networks for consumption and fatigue was used to support planning and, in NFQ’s case, policy selection.

The reported mean average rewards per step were $h$1 for PSO-P, $h$2 for RCNN, and $h$3 for NFQ, against an approximate upper bound of $h$4 obtained by optimizing directly on the benchmark with frozen stochasticity (Hein et al., 2017). PSO-P was best on 8 of the 10 set points and was described as more robust than the alternatives, though it required substantial online compute: under the reported setup, computing one action took less than 8 seconds using 96 CPUs (Hein et al., 2017).

This study is important because it exposed a characteristic feature of IB’s optimum: the best shift strategy is periodic rather than fixed. The paper reports an optimal shift trajectory with amplitude 20, centered around 70, and cycle duration 24 (Hein et al., 2017). NFQ tended to hold shift near 70 instead of learning the periodic pattern, while PSO-P was the only method reported to follow the optimal shift strategy closely (Hein et al., 2017). A plausible implication is that IB strongly favors methods that can reason over delayed consequences and represent non-equilibrium control laws, rather than merely stabilize at a local operating point.

The same batch-RL paper also reinforces a methodological point already present in the benchmark design: methods that rely only on current observation are structurally disadvantaged. PSO-P used an observation-history approximation to hidden state, RCNN used the internal recurrent state of the learned model, and NFQ used only current observation plus discretized actions, a mismatch that likely contributed to its weaker performance (Hein et al., 2017).

6. Scope, limitations, and later uses of the term

IB sits deliberately between toy benchmarks and real industrial systems. The introductory paper emphasizes that it remains simpler than true industrial plants because the reward is known explicitly, all reward-relevant variables are observable, and the simulator equations are fixed and available (Hein et al., 2016). It also states that safety constraints, actuator failures, maintenance, economics, and plant-specific hard constraints are not modeled in realistic detail (Hein et al., 2016). The later detailed paper preserves this overall positioning while strengthening the formal description and public software availability (Hein et al., 2017).

There are also documentary caveats. The 2016 text is explicitly described as a deprecated documentation version, and it notes that some formulas—especially the steering clipping notation and the fatigue amplification term—appear to suffer from typesetting errors (Hein et al., 2016). This matters for historical reading of the benchmark literature because some notational differences between the 2016 and 2017 papers are substantive documentation differences rather than conceptual changes.

In later literature, the phrase “industrial benchmark” and even the abbreviation “IB” have been used for unrelated benchmarks in industrial procurement QA, industrial optimization agents, industrial telemetry reasoning, visual SLAM, deformable linear object manipulation, and industrial anomaly detection (Bai et al., 11 May 2026, Zhou et al., 27 May 2026, Merzouki et al., 8 May 2026, Hana et al., 17 Mar 2026, Zhu et al., 16 Jun 2026, Xie et al., 2023). The classical Industrial Benchmark, however, refers specifically to the reinforcement-learning environment introduced in 2016 and elaborated in 2017 (Hein et al., 2016, Hein et al., 2017).

Taken in that classical sense, IB remains notable for a precise combination of properties rarely co-located in a public benchmark: continuous controls, partial observability, delayed and smeared effects, latent variables, heteroscedastic noise, exogenous operating regimes, and competing reward components. This combination, rather than any claim of plant realism, is what established the Industrial Benchmark as a reference environment for industrially motivated RL research (Hein et al., 2017, Hein et al., 2017).