Regime-Based Portfolio Allocation Using Hidden Markov Models and Reinforcement Learning

Abstract: This study develops a regime-aware portfolio allocation framework that integrates Markov switching models with Reinforcement Learning (RL) to dynamically allocate across equities (SPY), long-term Treasuries (TLT), and gold (GLD). Using daily ETF data from 2004-2025, we first characterize market behavior through a discrete Markov chain and then estimate a three-state Gaussian Hidden Markov Model (HMM) selected by the Bayesian Information Criterion (BIC). The estimated regimes-low-volatility, transitional, and high-volatility-exhibit strong persistence and state-dependent return dynamics consistent with recent findings on nonlinear market states (Ardia et al., 2024; Gupta & Pierdzioch, 2023). State-conditional analysis shows that SPY dominates in stable regimes, while TLT and GLD provide protection during stressed periods, motivating regime-conditioned allocation rules. We evaluate rule-based rotation and RL-driven strategies using a 30% out-of-sample test window with a one-day execution lag to avoid look-ahead bias. Both HMM-based allocations outperform a passive SPY benchmark, while the RL policy achieves the highest risk-adjusted performance, delivering the strongest Sharpe ratio and materially lower drawdowns, yet remains fully interpretable through discrete regime-dependent actions. Sensitivity analysis confirms the robustness of the three-state specification relative to two-state alternatives. Overall, the results demonstrate that RL can systematically enhance HMM-based regime detection, providing a transparent, adaptive, and empirically grounded framework for tactical asset allocation. The combined HMM-RL system provides a transparent, rules-based approach to tactical allocation that improves risk-adjusted performance relative to standard benchmark strategies.

• The paper introduces a hybrid method using Gaussian HMM for regime detection and tabular RL for dynamic portfolio allocation.
• It demonstrates that the RL-based policy achieves a Sharpe ratio of 0.83 and a maximum drawdown of –23.5% compared to conventional strategies.
• Empirical results validate the framework's transparency and adaptability in responding to market volatility and regime shifts.

Regime-Based Portfolio Allocation with HMMs and RL: A Technical Overview

Methodological Framework

This study presents a dynamic portfolio allocation architecture leveraging hybrid regime-detection and decision-making: it combines state segmentation using Gaussian Hidden Markov Models (HMM) with interpretable regime-aware policy derivation via discrete-state Reinforcement Learning (RL). The asset universe focuses on three highly liquid U.S.-listed ETFs: equities (SPY), long-duration Treasuries (TLT), and gold (GLD), using daily data from 2004–2025, with VIX serving as an observable volatility proxy.

Market regimes—critical to asset allocation given documented structural breaks and volatility clustering—are identified first through quantile-based discrete Markov Chain state assignment (on VIX) to illustrate transition persistence, and more formally via HMMs. Model selection (based on BIC/AIC) favors a three-state Gaussian HMM parameterized by EM, estimating filtered and smoothed state probabilities (Hamilton filter, Rauch-Tung-Striebel smoother). Inference reveals:

• Regime 0: Low-volatility with positive equity risk premium
• Regime 1: Transitional with elevated volatility and muted returns
• Regime 2: High-volatility/stress with strongly negative equity returns, positive mean for TLT/GLD

Crucially, state-dependent asset return characteristics directly motivate both rule-based and RL-based dynamic allocation.

Asset Allocation Strategies

Three baseline decision rules are evaluated:

1. Top-1 Rotation: Allocate fully to the asset with the highest expected return in the prevailing regime
2. 60/40 Rotation: 60% to the regime’s top asset, 40% to the next best
3. RL Policy: Discrete Markov Decision Process with the regime (from HMM) as state, and seven discrete portfolio allocations as the action space; optimal policy derived by tabular policy iteration maximizing expected risk-adjusted reward given HMM transition probabilities

All strategies are tested with a one-day execution lag to avoid look-ahead bias and are benchmarked against both equal-weight and buy-and-hold (SPY) strategies.

Numerical Results

Empirical evaluation (30% holdout window, 2022–2025) demonstrates significant regime-conditional variation in portfolio performance. Strong claims, numerically substantiated, include:

• The RL-based regime-aware policy achieves the highest Sharpe ratio (0.83) and materially reduced maximum drawdown (–23.5%) relative to all benchmarks.
• Top-1 and 60/40 rotation strategies outperform passive SPY in risk-adjusted returns and tail risk, but are less stable when regime boundaries are ambiguous or during rapid transitions, reflecting increased drawdown sensitivity due to concentrated positions.
• Notably, the RL agent assigns zero weight to TLT in periods classified as high-stress, prioritizing GLD’s favorable skew and regime resilience, consistent with evidence of diminished bond-hedging utility when stock-bond correlations turn positive (Baele et al., 2023).

The RL policy remains interpretable: it rotates fully into equities in stable regimes, partially or fully into gold during crises, and maintains moderate risk in transitional phases.

Out-of-sample (2022–2025 test window) performance comparison:

Strategy	Annualized Return	Volatility	Sharpe	Max Drawdown
RL (optimal policy)	14.3%	17.3%	0.83	–23.5%
Top-1 rotation	12.6%	16.0%	0.79	–21.8%
60/40 rotation	10.5%	12.8%	0.82	–29.3%
Equal-weight	9.1%	11.0%	0.83	–24.0%
Buy & hold SPY	13.2%	20.5%	0.65	–35.7%

RL’s ability to optimally exploit regime-dependent reward asymmetry, particularly by concentrating risk in favorable states and rotating defensively during stress, is directly attributable to its use of regime forecasts in policy computation.

Practical Implications and Theoretical Significance

The HMM+RL hybrid approach advances regime-switching portfolio techniques in several respects:

• Transparency and Interpretability: Unlike high-dimensional black-box RL, the tabular MDP with discrete regime-state mapping yields allocation rules that are tractable for portfolio oversight and regulatory review.
• Adapts to Regime Duration and Transition Probability: Explicit transition matrices ensure that both regime persistence and abrupt changes in market volatility are dynamically accounted for—crucial for mitigating drawdowns without sacrificing upside in benign states.
• Empirical Stability: Sensitivity analysis shows that a three-state specification is robust, with additional states inducing overfitting and interpretability loss.
• Operational Realism: Execution lag and strict out-of-sample validation support practical relevance and address widespread concerns of backtest over-optimization in financial ML (Bailey et al., 2014).

Importantly, the RL agent’s disregard for TLT during high-volatility episodes while favoring gold reflects a contradictory position on bond-hedging effectiveness during turbulent regimes compared to classical asset allocation theory, aligning with new empirical findings on time-varying stock-bond correlations.

Model Limitations and Future Research Directions

The study acknowledges several constraints:

• Distributional Misspecification: The assumption of Gaussian emissions in HMM may underrepresent heavy tails and jump dynamics in asset returns, leading to understated regime-change risk.
• Discrete Regime Boundaries: While the interpretation is clear, real market transitions may be smoother, and hard state assignments can misclassify boundary periods.
• Tabular RL Limitation: The finite, discrete action/state spaces provide interpretability at the expense of modeling capacity for high-dimensional or continuous action contexts, suggesting benefit from Deep RL or function approximation methods.
• Lack of Trading Friction Modeling: Zero transaction costs, bid-ask spreads, and slippage assumptions may overstate achievable performance, especially during regime transitions.
• Markov Property Assumption: Memory effects and persistent autocorrelation in financial markets might not be fully captured, potentially biasing regime inference or action selection.

Future extensions proposed in the paper include:

• Incorporation of Deep RL architectures (DQN, actor-critic) for richer state representation
• Non-Gaussian HMMs (Student-t, nonparametric) for superior tail modeling
• Macro-driven regime indicators combining statistical regime modeling with economic cycle proxies
• Explicit transaction costs and execution constraint modeling
• Expansion to multi-asset and global portfolios, exploiting cross-market regime transmission

Conclusion

This research establishes a comprehensive, interpretable regime-aware allocation framework by integrating discrete-state HMM-based regime detection and RL-derived adaptive portfolio rotation. The empirical results reinforce that further advances in regime detection and dynamic asset allocation can be achieved via model transparency and grounded probabilistic inference. This hybrid paradigm, balancing interpretability and adaptivity, addresses both the need for dynamic risk management in practice and the methodological rigor required for robust portfolio science.

The approach invites extensions that fuse richer statistical models and advanced learning architectures, with clear prospects for both empirical finance practitioners and theoretical researchers in sequential decision-making under uncertainty.

---

Reference: "Regime-Based Portfolio Allocation Using Hidden Markov Models and Reinforcement Learning" (2605.27848)