DeepAries: Adaptive Portfolio Management

• DeepAries is a deep reinforcement learning framework for dynamic portfolio management that jointly decides when to rebalance and how to allocate capital.
• It integrates a Transformer-based market encoder with Proximal Policy Optimization to process high-dimensional market data and optimize risk-adjusted returns.
• The approach outperforms fixed-frequency strategies by significantly improving metrics like CAGR, Sharpe ratio, and drawdown across major equity markets.

Searching arXiv for DeepAries and closely related reinforcement-learning portfolio management papers. DeepAries is a deep reinforcement learning framework for dynamic portfolio management that treats portfolio control as a joint decision problem over when to rebalance and how to allocate capital across assets. Introduced in "DeepAries: Adaptive Rebalancing Interval Selection for Enhanced Portfolio Selection" (Kim et al., 11 Sep 2025), it departs from fixed-frequency portfolio agents by allowing the rebalancing interval itself to be selected adaptively from a candidate set while portfolio weights are generated simultaneously. The framework combines a Transformer-based market encoder with Proximal Policy Optimization (PPO) over a mixed discrete-continuous action space, with the stated objective of reducing unnecessary transaction costs and improving risk-adjusted performance across multiple equity markets.

1. Conceptual basis and problem setting

DeepAries is motivated by the claim that practical portfolio management involves two tightly coupled questions: the next rebalancing time and the portfolio weights to be held until that time (Kim et al., 11 Sep 2025). Prior reinforcement-learning systems named in the paper, including PGPortfolio, FinRL, DeepTrader, HADAPS, and DeepClair, are described as typically assuming a fixed decision interval and often effectively re-optimizing at every step. DeepAries identifies this assumption as financially restrictive because stable markets and rapidly changing markets imply different optimal trading frequencies.

The framework therefore formulates multi-asset portfolio selection as a sequential decision problem in which the action determines not only the new portfolio composition but also the next decision epoch. This alters the temporal structure of control. Instead of an exogenous daily or weekly clock, the policy selects a rebalancing horizon $h_m$ from a candidate set

$\mathcal{H}=\{h_1,h_2,\dots,h_L\},$

with examples such as $\{1,5,20\}$ days, together with a long-only allocation vector

$\mathbf{w}(T_m)\in\mathbb{R}^N.$

The next decision time is then endogenous:

$T_{m+1}=T_m+h_m.$

This formulation makes timing and allocation inseparable. A portfolio intended for a one-day hold need not coincide with a portfolio intended for a twenty-day hold. A plausible implication is that the method should be interpreted not as a standard fixed-step trading policy with an auxiliary scheduling heuristic, but as a policy over variable-duration commitments.

2. Reinforcement-learning formulation

At each decision epoch $T_m$, the observed state is a tensor

$$\mathbf{s}(T_m)\in\mathbb{R}^{N\times \tau \times F},$$

where $N$ is the number of assets, $\tau$ is the look-back window length, and $F$ is the number of features per asset, with OHLC features given as an example (Kim et al., 11 Sep 2025). The action has two parts. The first is a discrete interval-selection action,

$\mathcal{H}=\{h_1,h_2,\dots,h_L\},$0

and the second is a continuous portfolio-allocation action,

$\mathcal{H}=\{h_1,h_2,\dots,h_L\},$1

subject to

$\mathcal{H}=\{h_1,h_2,\dots,h_L\},$2

For the interval $\mathcal{H}=\{h_1,h_2,\dots,h_L\},$3, the paper defines a price-relative vector $\mathcal{H}=\{h_1,h_2,\dots,h_L\},$4 and the corresponding portfolio return

$\mathcal{H}=\{h_1,h_2,\dots,h_L\},$5

Before a new rebalance, the previous portfolio drifts according to market movement, producing pre-rebalancing weights

$\mathcal{H}=\{h_1,h_2,\dots,h_L\},$6

Transaction cost is modeled as proportional to turnover:

$\mathcal{H}=\{h_1,h_2,\dots,h_L\},$7

Portfolio value then evolves as

$\mathcal{H}=\{h_1,h_2,\dots,h_L\},$8

The optimization target is final portfolio value,

$\mathcal{H}=\{h_1,h_2,\dots,h_L\},$9

This gives DeepAries a semi-endogenous event schedule: actions advance the environment by variable temporal increments rather than one fixed timestep.

3. Architecture and mixed action-space policy

DeepAries consists of three principal components: a Transformer-based state encoder, an adaptive interval-selection head, and a portfolio-allocation head, together with value networks for PPO (Kim et al., 11 Sep 2025). At decision epoch $\{1,5,20\}$0, the input

$\{1,5,20\}$1

is processed by a Transformer encoder $\{1,5,20\}$2, producing hidden states $\{1,5,20\}$3. A temporal attention block and a feed-forward block then produce a shared asset-level latent embedding

$\{1,5,20\}$4

The paper motivates Transformer use by the high dimensionality, non-stationarity, temporal dependence, and cross-sectional dependence of market data. It explicitly compares LSTM, TCN, vanilla Transformer, Reformer, Informer, Autoformer, Flashformer, FEDformer, Crossformer, and iTransformer, and reports that iTransformer performs best within the adaptive rebalancing framework. The manuscript does not provide low-level architectural details such as layer count, attention heads, positional encoding, dropout, or feed-forward widths; those details are therefore not specified in the available text.

The interval-selection head maps the latent state to logits over $\{1,5,20\}$5, yielding probabilities

$\{1,5,20\}$6

The portfolio-allocation head outputs Gaussian parameters

$\{1,5,20\}$7

samples a latent continuous action from a diagonal Gaussian, applies $\{1,5,20\}$8, and then normalizes the result to obtain portfolio weights. The manuscript states the intent to enforce long-only simplex constraints but does not spell out the exact normalization formula.

The joint policy is factorized as

$\{1,5,20\}$9

This factorization is the mechanism by which PPO is extended to the mixed action space.

A further modeling choice is the use of a separate value estimator for each interval,

$\mathbf{w}(T_m)\in\mathbb{R}^N.$0

with overall state value

$\mathbf{w}(T_m)\in\mathbb{R}^N.$1

This suggests that the critic is intended to encode interval-specific risk-return structure rather than a single value surface shared across all holding periods.

4. Training objective, reward shaping, and rollout mechanics

DeepAries is trained with PPO using a total loss

$\mathbf{w}(T_m)\in\mathbb{R}^N.$2

where $\mathbf{w}(T_m)\in\mathbb{R}^N.$3 is the clipped policy objective and $\mathbf{w}(T_m)\in\mathbb{R}^N.$4 is an MSE-style value loss (Kim et al., 11 Sep 2025). The algorithm listing names PPO hyperparameters $\mathbf{w}(T_m)\in\mathbb{R}^N.$5, but the numerical settings are not provided in the text excerpt.

Rollout generation is variable-interval. The procedure described in the paper is: observe $\mathbf{w}(T_m)\in\mathbb{R}^N.$6, sample interval $\mathbf{w}(T_m)\in\mathbb{R}^N.$7, sample weights $\mathbf{w}(T_m)\in\mathbb{R}^N.$8, hold the allocation over $\mathbf{w}(T_m)\in\mathbb{R}^N.$9, compute return and reward, store the transition $T_{m+1}=T_m+h_m.$0, and set $T_{m+1}=T_m+h_m.$1. Episodes therefore contain irregularly spaced decisions, with the spacing determined by the policy itself.

A distinctive design choice is reward shaping for interval selection:

$T_{m+1}=T_m+h_m.$2

where $T_{m+1}=T_m+h_m.$3 is the ex-post best-performing interval among the candidate horizons and $T_{m+1}=T_m+h_m.$4 is a bonus coefficient, given as $T_{m+1}=T_m+h_m.$5 in the formulation section. The paper presents this as a mechanism to stabilize and accelerate learning of the timing policy. At the same time, it explicitly means that training uses hindsight information for supervision of interval choice, although inference does not.

The paper also notes an unresolved inconsistency in transaction-cost settings: the formulation section gives an example cost rate of $T_{m+1}=T_m+h_m.$6, whereas the transaction-cost sensitivity experiment uses a baseline fee of $T_{m+1}=T_m+h_m.$7. This indicates that the cost parameter should be treated as experiment-specific rather than globally fixed.

5. Empirical evaluation and reported performance

Experiments are conducted over 20 years on four equity markets: DJ 30, FTSE 100, KOSPI, and CSI 300 (Kim et al., 11 Sep 2025). The paper reports 10 independent trials with different random seeds and states that test results correspond to the run with the lowest validation loss. Evaluation uses CAGR, SR, SoR, CR, and MDD.

In the main comparison, DeepAries is reported to win 17 out of 20 evaluation metrics across the four markets. The paper gives the following DeepAries results:

Market	Reported DeepAries metrics	Salient comparison
DJ 30	CAGR 7.90%, SR 0.130, SoR 0.211, CR 0.577, MDD 13.67%	DeepTrader has higher CAGR at 8.88%, but DeepAries has higher Sharpe and Sortino and lower drawdown
FTSE 100	CAGR 9.95%, SR 0.086, SoR 0.123, CR 0.546, MDD 18.23%	Slightly exceeds EIIE and DeepTrader in CAGR and improves risk-adjusted metrics
KOSPI	CAGR 14.27%, SR 0.180, SoR 0.372, CR 0.627, MDD 22.77%	Presented as one of the clearest gains
CSI 300	CAGR 5.09%, SR 0.070, SoR 0.109, CR 0.465, MDD 10.95%	The paper highlights that most baselines are negative in this market

The paper also notes a small inconsistency in its own narrative: a sentence below Table 1 says the best SR in CSI 300 is 0.109, but the table shows 0.109 as the Sortino ratio and 0.070 as the Sharpe ratio. The table values are the safer source.

The most direct test of the adaptive-timing thesis is the comparison against fixed schedules. DeepAries is stated to outperform Fixed Daily in all reported metrics across all four markets. For example, on DJ 30, Fixed Daily gives CAGR 2.31%, SR 0.048, and MDD 21.81%, whereas DeepAries gives CAGR 7.90%, SR 0.130, and MDD 13.67%. On KOSPI, Fixed Daily gives CAGR -4.47%, SR -0.029, and MDD 42.45%, compared with DeepAries at CAGR 14.27%, SR 0.180, and MDD 22.77%. The paper also emphasizes that Fixed Monthly is often a strong baseline, which is an important qualification: the argument is for adaptive timing, not for maximal trading frequency.

A separate ablation compares Adaptive (20), which selects from $T_{m+1}=T_m+h_m.$8, with DeepAries, which uses $T_{m+1}=T_m+h_m.$9. The more granular action space performs inconsistently and poorly in some markets, especially CSI 300. This suggests that a smaller interval set may be easier to learn robustly.

6. Backbone effects, operational interpretation, and limitations

The backbone ablation shows that adaptive timing usually improves a given sequence encoder, but iTransformer + adaptive timing performs best overall (Kim et al., 11 Sep 2025). For iTransformer, the gains from fixed to adaptive rebalancing are reported as follows: DJ 30, CAGR from 2.31% to 7.90% and SR from 0.048 to 0.130; FTSE 100, CAGR from 2.51% to 9.95%; KOSPI, CAGR from -4.47% to 14.27% and SR from -0.029 to 0.180; CSI 300, CAGR from 1.33% to 5.09%. The paper interprets these results as evidence that encoder quality matters materially for adaptive interval selection.

Transaction-cost sensitivity experiments on FTSE 100 and KOSPI increase the fee from a baseline of 0.01\% to 5x and 10x that level. The qualitative result is that both DeepAries and Fixed Daily deteriorate as fees increase, but Fixed Daily deteriorates more sharply. This is presented as evidence that the framework’s principal benefit is not only return enhancement but also lower sensitivity to fee drag through reduced unnecessary rebalancing.

Operationally, the policy behaves as follows: at time $T_m$0, it reads recent multi-asset history, encodes the state, predicts whether the next review should occur tomorrow, next week, or next month for $T_m$1, and outputs the weights to be held until then. The system then commits to those weights for the chosen horizon and reevaluates only at $T_m$2. This implies that inactivity is itself a learned action.

The paper also identifies or implies several limitations. The transaction-cost model is simplified and proportional, with no detailed treatment of slippage or market impact. The setting is long-only, with no leverage or shorting. Many details required for exact reproducibility, including exact splits and hyperparameters, are deferred to the demo or code. The reward shaping uses the ex-post best interval $T_m$3 during training, which is acceptable as a training signal but constitutes strong hindsight supervision rather than purely model-free RL. The paper does not provide an in-depth discussion of leakage concerns, and it does not present a formal regime-shift or out-of-distribution generalization analysis beyond cross-market evaluation.

Within the literature of reinforcement-learning portfolio management, DeepAries is therefore best understood as a framework that redefines the control variable from portfolio weights alone to timing-plus-allocation. Its main technical identity lies in the combination of a Transformer encoder, a categorical interval policy, a Gaussian allocation policy, and PPO training over variable-duration decisions. Its main empirical claim is that adaptive interval selection, especially with an iTransformer backbone, improves risk-adjusted performance and robustness to transaction costs across DJ 30, FTSE 100, KOSPI, and CSI 300 (Kim et al., 11 Sep 2025).