BoreaRL: Climate-Adaptive Boreal Forest RL

• BoreaRL is a reinforcement learning environment that accurately simulates coupled energy, water, and carbon fluxes for boreal forest management.
• It features a physically grounded simulator with a dual-objective MDP framework, enabling analysis of site-specific and generalist policy training paradigms.
• Empirical findings reveal a significant learning asymmetry where carbon objectives are more tractable than the noisy, delayed permafrost thaw signals.

Searching arXiv for the specified paper and closely related work on multi-objective RL and climate/forest management. arxiv_search(query="BoreaRL multi-objective reinforcement learning climate-adaptive boreal forest management", max_results=5, sort_by="submittedDate") BoreaRL is a multi-objective reinforcement learning environment for climate-adaptive boreal forest management, introduced in "BoreaRL: A Multi-Objective Reinforcement Learning Environment for Climate-Adaptive Boreal Forest Management" (Dsouza et al., 24 Sep 2025). It is presented as the first multi-objective reinforcement learning environment for this domain and is designed around a physically grounded simulator of coupled energy, carbon, and water fluxes. The motivating problem is that boreal forests store 30–40% of terrestrial carbon, much of it in climate-vulnerable permafrost soils, so management decisions involve nontrivial trade-offs between carbon sequestration and permafrost preservation that current tools cannot adequately address (Dsouza et al., 24 Sep 2025).

1. Scientific scope and problem setting

BoreaRL is framed around climate-adaptive management in boreal ecosystems where management interventions affect both aboveground carbon accumulation and belowground thermal dynamics. The central premise is that optimizing for carbon sequestration and optimizing for thaw avoidance are not interchangeable goals. In the formulation reported for BoreaRL, these objectives are explicitly separated and studied as a multi-objective decision problem rather than being collapsed into a single proxy target from the outset (Dsouza et al., 24 Sep 2025).

The environment is intended to support two complementary research settings. In site-specific mode, a single partially observable Markov decision process is fixed, enabling controlled study of repeatable, location-targeted strategies. In generalist mode, environmental conditions are sampled episode by episode over realistic site distributions, so the learning problem becomes one of robustness under environmental stochasticity. This division makes BoreaRL suitable both for algorithmic diagnosis and for studying policy transfer across heterogeneous boreal conditions (Dsouza et al., 24 Sep 2025).

A plausible implication is that BoreaRL occupies an intermediate position between purely stylized RL benchmarks and operational forest simulators: it preserves explicit process-based biophysics while exposing an RL interface with annual management actions and long-horizon credit assignment.

2. Simulator architecture and physical formulation

At the core of the environment is BoreaRL-Sim, described as a process-based, sub-annual simulator that resolves coupled energy, water, and carbon fluxes on $n$-minute time-steps over a full 365-day year for each annual RL decision (Dsouza et al., 24 Sep 2025). The simulator spans canopy, trunk, snowpack, surface soil, and deep soil layers and solves a five-node energy balance,

$$C_i\,\frac{dT_i}{dt} = \sum F_{in,i}-\sum F_{out,i}\,.$$

For the canopy, the reported balance includes net radiation, sensible heat, latent heat, a photosynthesis-linked energy term, melt, and canopy–trunk coupling. The component expressions include

$$R_{net,can}=Q_{\rm solar}(1-\alpha_{can})+\epsilon_{can}(L_{down,atm}+L_{up,ground})-2\epsilon_{can}\sigma T_{can}^4,$$

$H_{can}=h_{can}(T_{can}-T_{air}),$

$$LE_{can}=\alpha_{PT}\tfrac{\Delta}{\Delta+\gamma}R_{net,can}\,f_{VPD}\,f_{SWC},$$

and

$G_{photo}=GPP\cdot J_{\rm per\,gC}.$

The last term couples energy to the carbon cycle via light-use efficiency. Similar balances govern trunk, snow, and soil nodes. Conductive flux across the deep soil boundary is then used as the permafrost thaw proxy (Dsouza et al., 24 Sep 2025).

This design is significant because the thaw objective is not represented as a direct static label or heuristic penalty. Instead, it arises from the simulator’s sub-annual thermodynamic evolution. That modeling choice helps explain why the thaw objective is reported to be substantially harder for current multi-objective RL methods: the signal is noisy, delayed, and mediated through coupled physical processes rather than immediate stand-level inventory changes.

3. Markov decision process and reward design

BoreaRL-Env wraps BoreaRL-Sim as a partially observable Markov decision process,

$(\mathcal S,\mathcal A,\mathcal O,P,R,\gamma).$

The observation space depends on the training paradigm. In site-specific mode, the observation is $o_t\in\mathcal O\subset\mathbb R^{43}$. In generalist mode, it is $\mathbb R^{105}$. The observation includes year, stem density, conifer fraction, carbon stocks, age-class histograms, and disturbance history; in generalist mode it additionally includes sampled site parameters $(\phi)$ and a scalar preference weight $$C_i\,\frac{dT_i}{dt} = \sum F_{in,i}-\sum F_{out,i}\,.$$0 (Dsouza et al., 24 Sep 2025).

The action space is discrete:

$$C_i\,\frac{dT_i}{dt} = \sum F_{in,i}-\sum F_{out,i}\,.$$1

Each action encodes one of five density changes $$C_i\,\frac{dT_i}{dt} = \sum F_{in,i}-\sum F_{out,i}\,.$$2 stems ha$$C_i\,\frac{dT_i}{dt} = \sum F_{in,i}-\sum F_{out,i}\,.$$3 and one of five conifer-fraction targets $$C_i\,\frac{dT_i}{dt} = \sum F_{in,i}-\sum F_{out,i}\,.$$4, yielding $$C_i\,\frac{dT_i}{dt} = \sum F_{in,i}-\sum F_{out,i}\,.$$5 maneuvers. The transition operator $$C_i\,\frac{dT_i}{dt} = \sum F_{in,i}-\sum F_{out,i}\,.$$6 is implemented implicitly by updating stand density and species mix and then running the sub-annual physics loop together with stochastic disturbance checks for fire, insects, recruitment, and natural mortality (Dsouza et al., 24 Sep 2025).

The reward is two-dimensional:

$$C_i\,\frac{dT_i}{dt} = \sum F_{in,i}-\sum F_{out,i}\,.$$7

The carbon component is

$$C_i\,\frac{dT_i}{dt} = \sum F_{in,i}-\sum F_{out,i}\,.$$8

where $$C_i\,\frac{dT_i}{dt} = \sum F_{in,i}-\sum F_{out,i}\,.$$9 normalizes net ecosystem carbon change, including harvested wood products; $$R_{net,can}=Q_{\rm solar}(1-\alpha_{can})+\epsilon_{can}(L_{down,atm}+L_{up,ground})-2\epsilon_{can}\sigma T_{can}^4,$$0 and $$R_{net,can}=Q_{\rm solar}(1-\alpha_{can})+\epsilon_{can}(L_{down,atm}+L_{up,ground})-2\epsilon_{can}\sigma T_{can}^4,$$1 are currently zero bonuses on total stock and HWP sales; $$R_{net,can}=Q_{\rm solar}(1-\alpha_{can})+\epsilon_{can}(L_{down,atm}+L_{up,ground})-2\epsilon_{can}\sigma T_{can}^4,$$2 and $$R_{net,can}=Q_{\rm solar}(1-\alpha_{can})+\epsilon_{can}(L_{down,atm}+L_{up,ground})-2\epsilon_{can}\sigma T_{can}^4,$$3 penalize exceedance of biomass and soil carbon limits; $$R_{net,can}=Q_{\rm solar}(1-\alpha_{can})+\epsilon_{can}(L_{down,atm}+L_{up,ground})-2\epsilon_{can}\sigma T_{can}^4,$$4 if density $$R_{net,can}=Q_{\rm solar}(1-\alpha_{can})+\epsilon_{can}(L_{down,atm}+L_{up,ground})-2\epsilon_{can}\sigma T_{can}^4,$$5, else $$R_{net,can}=Q_{\rm solar}(1-\alpha_{can})+\epsilon_{can}(L_{down,atm}+L_{up,ground})-2\epsilon_{can}\sigma T_{can}^4,$$6; and $$R_{net,can}=Q_{\rm solar}(1-\alpha_{can})+\epsilon_{can}(L_{down,atm}+L_{up,ground})-2\epsilon_{can}\sigma T_{can}^4,$$7 penalizes ineffective thinning or planting (Dsouza et al., 24 Sep 2025).

The thaw component is

$$R_{net,can}=Q_{\rm solar}(1-\alpha_{can})+\epsilon_{can}(L_{down,atm}+L_{up,ground})-2\epsilon_{can}\sigma T_{can}^4,$$8

where $$R_{net,can}=Q_{\rm solar}(1-\alpha_{can})+\epsilon_{can}(L_{down,atm}+L_{up,ground})-2\epsilon_{can}\sigma T_{can}^4,$$9 and $H_{can}=h_{can}(T_{can}-T_{air}),$0 are annual positive and negative heat fluxes through the deep soil boundary, with a 5:1 penalty ratio for warming versus cooling (Dsouza et al., 24 Sep 2025).

Users express preferences through

$H_{can}=h_{can}(T_{can}-T_{air}),$1

and train either fixed-$H_{can}=h_{can}(T_{can}-T_{air}),$2 agents or preference-conditioned policies using linear scalarization,

$H_{can}=h_{can}(T_{can}-T_{air}),$3

This formulation makes the carbon–thaw trade-off explicit and measurable. It also clarifies a potential misconception: maximizing carbon gain is not treated as a sufficient proxy for permafrost preservation. The reward design separates the two objectives formally, and the reported results indicate that the resulting optimization landscape is strongly asymmetric (Dsouza et al., 24 Sep 2025).

4. Training paradigms and algorithmic baselines

The two principal training paradigms differ in whether site parameters are fixed or sampled. In site-specific mode, $H_{can}=h_{can}(T_{can}-T_{air}),$4 is fixed, with deterministic weather seed, zero temperature noise, and fixed soil and disturbance priors; agents learn over $H_{can}=h_{can}(T_{can}-T_{air}),$5 annual steps. In generalist mode, each episode samples $H_{can}=h_{can}(T_{can}-T_{air}),$6 over realistic ranges including latitude, mean annual temperature, seasonal amplitude, and soil conductivity, and the objective becomes

$H_{can}=h_{can}(T_{can}-T_{air}),$7

The reported observation dimensionalities are 43 and 105, respectively (Dsouza et al., 24 Sep 2025).

Paradigm	Site specification	Observation dimensionality
Site-specific mode	$H_{can}=h_{can}(T_{can}-T_{air}),$8 fixed	43
Generalist mode	$H_{can}=h_{can}(T_{can}-T_{air}),$9 sampled each episode	105

Four representative multi-objective RL approaches were evaluated (Dsouza et al., 24 Sep 2025).

Fixed-$$LE_{can}=\alpha_{PT}\tfrac{\Delta}{\Delta+\gamma}R_{net,can}\,f_{VPD}\,f_{SWC},$$0 EUPG trains a single policy $$LE_{can}=\alpha_{PT}\tfrac{\Delta}{\Delta+\gamma}R_{net,can}\,f_{VPD}\,f_{SWC},$$1 on a constant scalarization $$LE_{can}=\alpha_{PT}\tfrac{\Delta}{\Delta+\gamma}R_{net,can}\,f_{VPD}\,f_{SWC},$$2 using Expected Utility Policy Gradient. The policy input is augmented with cumulative returns per objective for ESR-consistency, and the objective is

$$LE_{can}=\alpha_{PT}\tfrac{\Delta}{\Delta+\gamma}R_{net,can}\,f_{VPD}\,f_{SWC},$$3

Variable-$$LE_{can}=\alpha_{PT}\tfrac{\Delta}{\Delta+\gamma}R_{net,can}\,f_{VPD}\,f_{SWC},$$4 EUPG is preference-conditioned. At the start of each episode, $$LE_{can}=\alpha_{PT}\tfrac{\Delta}{\Delta+\gamma}R_{net,can}\,f_{VPD}\,f_{SWC},$$5 is sampled and $$LE_{can}=\alpha_{PT}\tfrac{\Delta}{\Delta+\gamma}R_{net,can}\,f_{VPD}\,f_{SWC},$$6 is fed into the observation, producing the training objective

$$LE_{can}=\alpha_{PT}\tfrac{\Delta}{\Delta+\gamma}R_{net,can}\,f_{VPD}\,f_{SWC},$$7

PPO Gated uses Proximal Policy Optimization with separate gated network heads for planting versus thinning actions, intended to prevent illegal moves, and is trained on preference input with standard PPO objective, clip 0.2 and GAE 0.95.

Curriculum PPO uses a two-network scheme in which a curriculum selector $$LE_{can}=\alpha_{PT}\tfrac{\Delta}{\Delta+\gamma}R_{net,can}\,f_{VPD}\,f_{SWC},$$8 learns which episodes to train on; episodes are accepted when $$LE_{can}=\alpha_{PT}\tfrac{\Delta}{\Delta+\gamma}R_{net,can}\,f_{VPD}\,f_{SWC},$$9 with threshold 0.5. Its objective is

$G_{photo}=GPP\cdot J_{\rm per\,gC}.$0

Generalist agents used $G_{photo}=GPP\cdot J_{\rm per\,gC}.$1 timesteps, while site-specific agents used $G_{photo}=GPP\cdot J_{\rm per\,gC}.$2 (Dsouza et al., 24 Sep 2025).

5. Empirical asymmetry between carbon and thaw objectives

A central result reported for BoreaRL is a pronounced asymmetry in learning difficulty. Across both training paradigms, carbon-focused agents with $G_{photo}=GPP\cdot J_{\rm per\,gC}.$3 learn rapidly, whereas thaw-focused agents with $G_{photo}=GPP\cdot J_{\rm per\,gC}.$4 remain near baseline (Dsouza et al., 24 Sep 2025).

In site-specific mode, carbon policies increase scalarized reward by 80–100% above baseline within $G_{photo}=GPP\cdot J_{\rm per\,gC}.$5 steps, while thaw policies show less than 5% improvement even after $G_{photo}=GPP\cdot J_{\rm per\,gC}.$6 steps. In generalist mode, the same gap persists, which the paper attributes to the noisy, delayed character of the thaw signal (Dsouza et al., 24 Sep 2025).

The learned stand structures also diverge. Carbon policies ramp density to approximately $G_{photo}=GPP\cdot J_{\rm per\,gC}.$7 stems ha$G_{photo}=GPP\cdot J_{\rm per\,gC}.$8, whereas thaw policies hold near approximately $G_{photo}=GPP\cdot J_{\rm per\,gC}.$9–$(\mathcal S,\mathcal A,\mathcal O,P,R,\gamma).$0 stems ha$(\mathcal S,\mathcal A,\mathcal O,P,R,\gamma).$1. Species strategies differ as well: purely carbon agents maintain current conifer fraction, thaw agents shift toward deciduous, and mixed policies raise conifer fraction only under intermediate preference weights (Dsouza et al., 24 Sep 2025).

These findings directly bear on benchmark interpretation. A superficial reading might treat the environment as a standard forest carbon maximization task with an added ecological regularizer. The reported outcomes contradict that simplification. The thaw objective behaves as a distinct and substantially harder control target, and this suggests that performance on carbon-centered forestry tasks does not transfer straightforwardly to permafrost-sensitive management.

6. Curriculum learning, emergent strategies, and benchmark role

In the generalist, preference-conditioned benchmark, both Variable-$(\mathcal S,\mathcal A,\mathcal O,P,R,\gamma).$2 EUPG and PPO Gated are reported to collapse to near-zero performance for most weights. Curriculum PPO, by contrast, converges quickly to stable performance across the weight spectrum. Its evaluation Pareto front dominates the others, with 50% fewer $(\mathcal S,\mathcal A,\mathcal O,P,R,\gamma).$3-monotonicity violations—50% versus 75–100% for fixed/EUPG/PPO Gated—and superior front coverage (Dsouza et al., 24 Sep 2025).

Analysis of the final policies identifies three distinct management philosophies. PPO Gated pursues aggressive carbon farming, with high early planting, densities greater than $(\mathcal S,\mathcal A,\mathcal O,P,R,\gamma).$4 stems ha$(\mathcal S,\mathcal A,\mathcal O,P,R,\gamma).$5, and conifer fractions $(\mathcal S,\mathcal A,\mathcal O,P,R,\gamma).$6–$(\mathcal S,\mathcal A,\mathcal O,P,R,\gamma).$7, producing maximal carbon but little permafrost benefit. Variable-$(\mathcal S,\mathcal A,\mathcal O,P,R,\gamma).$8 EUPG remains conservative, with linear density growth and conifer fraction around $(\mathcal S,\mathcal A,\mathcal O,P,R,\gamma).$9, achieving mediocre outcomes on both objectives. Curriculum PPO adopts moderate densities around $o_t\in\mathcal O\subset\mathbb R^{43}$0 stems ha$o_t\in\mathcal O\subset\mathbb R^{43}$1 and conifer fractions around $o_t\in\mathcal O\subset\mathbb R^{43}$2, maximizing thaw protection $o_t\in\mathcal O\subset\mathbb R^{43}$3 without sacrificing much carbon (Dsouza et al., 24 Sep 2025).

Across evaluation episodes, longer growing seasons under Curriculum PPO correlate strongly with higher thaw rewards, which the paper interprets as effective use of shading, evapotranspiration, and altered snowpack dynamics for permafrost insulation (Dsouza et al., 24 Sep 2025). This is scientifically important because it indicates that successful policies exploit coupled biophysical pathways rather than merely adjusting stand density in isolation.

The environment is open-sourced under a BSD-3 license at the reported repository URL, with code for BoreaRL-Sim, environment wrappers, training scripts, and experimental seeds. Installation requires Python 3.8+, PyTorch $o_t\in\mathcal O\subset\mathbb R^{43}$4, gymnasium, morl-baselines, NumPy, and optionally Numba for JIT acceleration. The commands python train.py --config=generalist and site-specific are reported to reproduce the published results after pip install -r requirements.txt (Dsouza et al., 24 Sep 2025).

Taken together, these characteristics establish BoreaRL as a benchmark for multi-objective RL in climate applications. The benchmark’s defining contribution is not only that it couples a rigorously parameterized forest simulator to an RL interface, but also that it exposes a specific failure mode of current MORL methods: carbon objectives are tractable under standard approaches, whereas thaw preservation remains difficult in both specialist and generalist settings. This suggests that future progress in the area will depend not only on better policy optimization, but also on methods that can cope with delayed, noisy, physically mediated objectives of the kind BoreaRL makes explicit.