---
title: 'Industrial Benchmark (IB): Synthetic RL Testbed'
url: https://www.emergentmind.com/topics/industrial-benchmark-ib
type: topic
---

# Industrial Benchmark (IB): Synthetic RL Testbed

The **Industrial Benchmark (IB)** is an open-source, synthetic reinforcement-learning benchmark designed to reproduce the hardness profile of industrial control and optimization problems without approximating any single real plant. It was introduced to bridge the gap between freely available academic benchmarks and the continuous, partially observed, delayed, stochastic, and multi-objective conditions encountered in industrial applications. Across its introductory and detailed descriptions, IB is presented as a benchmark for industrially motivated control, system identification, forecasting, transfer learning, and batch/offline RL, with public Java and Python implementations and an OpenAI Gym wrapper [1610.03793, 1709.09480].

## 1. Origin and intended role

IB was motivated by a recurring methodological problem: real-world industrial testing is expensive, risky, and slow, whereas many standard RL benchmarks are low-dimensional, fully observed, weakly stochastic, or otherwise too simple to capture industrial difficulty. The benchmark was therefore designed to include what its authors describe as aspects “vital in industrial applications,” while remaining generic rather than tied to a single physical process [1610.03793, 1709.09480].

The environment is explicitly **not** a faithful model of any specific plant. Instead, it is meant to reproduce the kinds of hardness seen in applications such as gas turbines, wind turbines, power plants, and process-industry systems: continuous state and action spaces, partial observability, latent internal state, delayed and smeared action effects, heteroscedastic stochasticity, exogenous operating conditions, and conflicting optimization criteria [1709.09480]. This positioning is central to the benchmark’s identity. A common misconception is that IB is a plant simulator; the papers state the opposite. It is a deliberately synthetic environment whose value lies in its difficulty structure rather than in physical fidelity [1610.03793].

The introductory paper also emphasizes that IB is useful beyond policy learning. The same environment is presented as relevant for regression, forecasting, system identification, transfer learning, active learning, feature selection, and change detection [1610.03793]. This broader scope follows directly from the fact that the benchmark exposes hidden-state dynamics, delayed responses, and structured noise rather than only a reward-maximization problem.

## 2. Interface, state variables, and reward formulation

At each time step, the agent chooses a continuous three-dimensional action in \([-1,1]^3\), interpreted as changes to three steerings: **velocity** \(v\), **gain** \(g\), and **shift**. The introductory description denotes the third steering by \(s\), whereas the later detailed paper writes it as \(h\); both versions clip the updated steerings to \([0,100]\) and use unequal actuation scales, with \(d^{\mathrm v}=1\), \(d^{\mathrm g}=10\), and \(d^{\mathrm h}\approx 5.75\) in the later formulation [1610.03793, 1709.09480].

The observable vector has six components,
\[
o_t = (v_t,g_t,h_t,p_t,c_t,f_t)
\]
in the later notation, or equivalently \((v,g,s,p,c,f)\) in the introductory paper. Here \(p\) is an observable but uncontrollable **set point** or external driver, \(c\) is **consumption**, and \(f\) is **fatigue** [1610.03793, 1709.09480]. The exogenous set point is important because IB is not an autonomous regulation problem with a fixed optimum; operating regime depends on \(p\).

The reward formulation differs across the two benchmark descriptions. The introductory paper defines reward as
\[
r(t)=-(c(t+1)+f(t+1)),
\]
whereas the later detailed paper specifies
\[
r_{t+1}=-c_{t+1}-3f_{t+1}.
\]
Both versions frame IB as a cost-minimization problem built from consumption and fatigue, but the later paper weights fatigue three times more strongly [1610.03793, 1709.09480]. This discrepancy is best understood as a documentation/version difference rather than a conceptual disagreement about the benchmark’s purpose.

The benchmark is naturally represented as transition tuples of the form
\[
(\vec O,\vec a,\vec O',r),
\]
which made it a natural testbed for early batch/offline RL studies [1610.03793].

## 3. Internal dynamics and sources of difficulty

IB is built from three interacting sub-dynamics: **operational cost**, **mis-calibration**, and **fatigue** [1610.03793]. The operational-cost component depends on set point, velocity, and gain through an exponential term. In the introductory formulation,
\[
o(t)=\exp\left(\frac{2p(t)+4v(t)+2.5g(t)}{100}\right),
\]
and this signal is not observed directly; instead, it is delayed and smeared by a finite convolution over values from \(t-5\) to \(t-9\), with the largest weight at \(t-7\) [1610.03793]. The later paper uses the same structure with \(\theta_t\) in place of \(o(t)\) [1709.09480]. This delay-smearing mechanism is one of the benchmark’s main credit-assignment obstacles.

Mis-calibration couples the external set point to the shift steering. The introductory paper defines an effective shift
\[
s^e=\min(1.5,\max(-1.5,s/20-p/50-1.5)),
\]
and states that mis-calibration depends on three latent variables \(m_1^l,m_2^l,m_3^l\), but it does not provide the explicit latent update equations [1610.03793]. The later detailed paper replaces this abstract presentation with a fully specified cyclic latent system involving domain, response, and direction-index variables \((\delta,\psi,\phi)\), together with a linearly biased Goldstone potential [1709.09480]. That later paper explicitly states that the benchmark is designed so that the optimal policy does not correspond to a fixed point in the steerings, and that the optimal policy in the mis-calibration subsystem is approximately sinusoidal [1709.09480].

Fatigue is the second reward-relevant term and creates the benchmark’s multi-criteria tradeoff. In both descriptions it depends on set point, velocity, and gain, with a basic fatigue term of the form
\[
f_b=\max\left(0,\frac{30000}{5v+100}-0.01\,g^2\right)
\]
followed by an amplification mechanism driven by transformed controls, stochastic variables, and latent fatigue state [1610.03793, 1709.09480]. The later paper formalizes two latent fatigue variables, \(\mu_t^{\mathrm v}\) and \(\mu_t^{\mathrm g}\), whose thresholded dynamics can enter a high-fatigue regime once a threshold of \(1.2\) is crossed [1709.09480]. This creates a hidden bifurcation-like effect resembling industrial stress or wear escalation.

Observed consumption adds heteroscedastic Gaussian noise to the noiseless cost:
\[
c=\hat c+\mathrm{gauss}(0,1+0.02\,\hat c).
\]
The observation noise standard deviation therefore increases with the signal magnitude [1610.03793]. Together, delayed operational cost, latent mis-calibration, latent fatigue accumulation, and state-dependent noise make IB difficult in precisely the way it was intended to be difficult: the agent observes only a compressed projection of a nonlinear, history-dependent process.

## 4. Partial observability, regimes, and standard protocols

IB is only Markovian in an augmented internal state. The introductory paper describes a preferred minimal Markovian state with **20 variables**: the 6 observable variables, 9 delayed operational-cost values, 3 mis-calibration latents, and 2 fatigue latents [1610.03793]. The later paper gives the same 20-variable count in updated notation, with hidden operational-cost history, mis-calibration variables, and fatigue variables augmenting the observation vector [1709.09480]. The direct implication is that the observable state is **not Markovian**, so history windows, recurrent policies, or latent-state estimators are natural methodological choices.

The set point \(p\) defines operating regimes. In the **Constant Set Point Setting**, the introductory paper uses
\[
p\in\{10,20,\ldots,100\},
\]
and notes that constant-set-point experiments should be viewed not as one single POMDP but as a family of POMDPs parameterized by set point [1610.03793]. A **Variable Set Point Setting** is also described, in which changes in \(p\) are externally driven rather than action-driven [1610.03793, 1709.09480]. This makes transfer across operating regimes an intrinsic part of the benchmark’s conceptual design.

For initial offline evaluation, the introductory paper recommends a **maximum entropy policy**, uniformly random over the action domain. The standard initialization is three steerings at \(50\), default latent variables, and ten set points \(p\in\{10,20,\ldots,100\}\); running the random policy for 1000 time steps per set point yields **10,000 data points** [1610.03793]. That same paper reports early predictive-modeling baselines using recurrent neural networks, with MRABD of approximately \(10\%\) for reward, approximately \(3.6\%\) for consumption, and approximately \(24\%\) for fatigue when \(f>1\) [1610.03793].

This experimental framing explains why IB became closely associated with **batch/offline RL**. The benchmark was designed under the industrially motivated assumption that exploration may be unsafe or expensive, while logged data remain available. That assumption later became central to published control studies on the environment [1705.07262].

## 5. Batch RL studies and characteristic control behavior

An early extensive study of batch/off-policy RL on IB compared three approaches: **Particle Swarm Optimization Policy (PSO-P)**, **Recurrent Control Neural Network (RCNN)**, and **Neural Fitted Q-Iteration (NFQ)** [1705.07262]. In that study, the offline dataset contained **100,000** transitions, generated from ten set points, ten initializations per set point, and trajectories of length 1000 [1705.07262]. A learned recurrent system model with separate networks for consumption and fatigue was used to support planning and, in NFQ’s case, policy selection.

The reported mean average rewards per step were **\(-159.84\)** for PSO-P, **\(-167.01\)** for RCNN, and **\(-170.66\)** for NFQ, against an approximate upper bound of **\(-145.49\)** obtained by optimizing directly on the benchmark with frozen stochasticity [1705.07262]. PSO-P was best on **8 of the 10 set points** and was described as more robust than the alternatives, though it required substantial online compute: under the reported setup, computing one action took **less than 8 seconds** using **96 CPUs** [1705.07262].

This study is important because it exposed a characteristic feature of IB’s optimum: the best shift strategy is **periodic** rather than fixed. The paper reports an optimal shift trajectory with **amplitude 20**, **centered around 70**, and **cycle duration 24** [1705.07262]. NFQ tended to hold shift near 70 instead of learning the periodic pattern, while PSO-P was the only method reported to follow the optimal shift strategy closely [1705.07262]. A plausible implication is that IB strongly favors methods that can reason over delayed consequences and represent non-equilibrium control laws, rather than merely stabilize at a local operating point.

The same batch-RL paper also reinforces a methodological point already present in the benchmark design: methods that rely only on current observation are structurally disadvantaged. PSO-P used an observation-history approximation to hidden state, RCNN used the internal recurrent state of the learned model, and NFQ used only current observation plus discretized actions, a mismatch that likely contributed to its weaker performance [1705.07262].

## 6. Scope, limitations, and later uses of the term

IB sits deliberately between toy benchmarks and real industrial systems. The introductory paper emphasizes that it remains simpler than true industrial plants because the reward is known explicitly, all reward-relevant variables are observable, and the simulator equations are fixed and available [1610.03793]. It also states that safety constraints, actuator failures, maintenance, economics, and plant-specific hard constraints are not modeled in realistic detail [1610.03793]. The later detailed paper preserves this overall positioning while strengthening the formal description and public software availability [1709.09480].

There are also documentary caveats. The 2016 text is explicitly described as a **deprecated** documentation version, and it notes that some formulas—especially the steering clipping notation and the fatigue amplification term—appear to suffer from typesetting errors [1610.03793]. This matters for historical reading of the benchmark literature because some notational differences between the 2016 and 2017 papers are substantive documentation differences rather than conceptual changes.

In later literature, the phrase **“industrial benchmark”** and even the abbreviation **“IB”** have been used for unrelated benchmarks in industrial procurement QA, industrial optimization agents, industrial telemetry reasoning, visual SLAM, deformable linear object manipulation, and industrial anomaly detection [2605.10267, 2605.28158, 2605.07675, 2603.16240, 2606.18097, 2301.13359]. The classical **Industrial Benchmark**, however, refers specifically to the reinforcement-learning environment introduced in 2016 and elaborated in 2017 [1610.03793, 1709.09480].

Taken in that classical sense, IB remains notable for a precise combination of properties rarely co-located in a public benchmark: continuous controls, partial observability, delayed and smeared effects, latent variables, heteroscedastic noise, exogenous operating regimes, and competing reward components. This combination, rather than any claim of plant realism, is what established the Industrial Benchmark as a reference environment for industrially motivated RL research [1709.09480, 1705.07262].

Source: https://www.emergentmind.com/topics/industrial-benchmark-ib