PM-Agent: Adaptive Metaheuristics & Portfolio

• PM-Agent is an autonomous decision-making architecture combining dynamic metaheuristic search with modular reinforcement learning for portfolio management.
• It orchestrates heterogeneous algorithms (e.g., GA, PSO, DQN, PPO) to achieve rapid convergence and superior performance compared to traditional methods.
• Its modular and adaptive design enables real-time switching and robust optimization in dynamic systems, proving effective in financial asset allocation.

A PM-Agent (Portfolio Management Agent or Polymorphic Metaheuristic Agent, depending on context) refers to an autonomous agent architecture for decision-making and optimization tasks spanning two advanced paradigms: dynamic metaheuristic search in general complex systems and modular reinforcement learning for financial portfolio management. The defining features across both lines are agent modularity, real-time performance monitoring, and self-adaptive policy switching, whether through metaheuristic control or reinforcement learning-based asset allocation. Notably, the Polymorphic Metaheuristic Agent (PMA) specializes in orchestrating dynamic switching among multiple metaheuristics based on feedback-driven selection, while the Portfolio Management Agent (as in MSPM) encapsulates multi-agent DQN/PPO-driven portfolio control. The following sections structure the theoretical underpinnings, architectural principles, learning algorithms, mathematical models, and empirical performance of PM-Agents in these two domains.

1. Architectures of PM-Agents

1.1. Polymorphic Metaheuristic Agent (PMA)

The PMA is a core component of the Polymorphic Metaheuristic Framework (PMF), tasked with driving iterative search by applying and dynamically switching between heterogeneous metaheuristic algorithms such as Genetic Algorithms (GA), Particle Swarm Optimization (PSO), Differential Evolution (DE), Ant Colony Optimization (ACO), Simulated Annealing (SA), Tabu Search (TS), and Covariance Matrix Adaptation Evolution Strategy (CMA-ES). At each iteration, the PMA evaluates key metrics—fitness improvement, stagnation, exploration-exploitation balance, and computational cost—and relays these to the Polymorphic Metaheuristic Selection Agent (PMSA), which determines the agent’s policy regarding algorithm switching or continuation (Esfahani et al., 20 May 2025).

1.2. Portfolio Management Agent (PM-Agent) in MSPM

In the MSPM system, the PM-Agent architecture is realized through a modular assembly of per-asset Evolving Agent Modules (EAMs) and a portfolio-coordinating Strategic Agent Module (SAM). Each EAM is a Deep Q-Network (DQN)-based unit tasked with transforming heterogeneous asset-specific data streams (OHLCV, news sentiment, buzz) into low-dimensional “signal-comprised” tensors, which are then aggregated by the SAM—a PPO-based agent—into continuous portfolio reallocation decisions across multiple assets and cash (Huang et al., 2021).

2. Mathematical Formalization

2.1. PMA Decision Process

Let $A = \{A_1, ..., A_N\}$ be candidate metaheuristics. At iteration $t$, the PMA maintains:

• $i(t)\in\{1,\ldots,N\}$: current algorithm index.
• $X(t)$: population of $M$ solutions.
• $f_i(t)$: feedback for each $A_i$, aggregated as

$$f_i(t) = w_1 \cdot \Delta F_i(t) + w_2 \cdot S_i(t) + w_3 \cdot E_i(t) + w_4 \cdot C_i(t),$$

where $\Delta F_i(t)$ is mean fitness improvement, $S_i(t)$ is stagnation count, $E_i(t)$ is an exploration–exploitation metric, and $C_i(t)$ is normalized computational cost over a sliding window $W$ with user-defined $w_1\ldots w_4$.

Selection probability uses a feedback-weighted softmax:

$$P_i(t) = \frac{\exp\left(-\lambda f_i(t)\right)}{\sum_{j=1}^N \exp\left(-\lambda f_j(t)\right)}$$

with $\lambda>0$ controlling sensitivity.

Switching is performed if $P_{i(t)}(t) < \theta$, for threshold $\theta\in[0,1]$:

$$i(t+1) = \begin{cases} \arg\max_j P_j(t), & \text{if } P_{i(t)}(t) < \theta \ i(t), & \text{otherwise} \end{cases}$$

All population handover leverages elite preservation or structural mapping mechanisms (Esfahani et al., 20 May 2025).

2.2. MSPM PM-Agent Learning

EAM (DQN-based)

• State at $t$: $v_t = (s_t, \rho_t)$ with price $s_t \in \mathbb{R}^{n\times 5}$ and sentiment $\rho_t \in \mathbb{R}^{n\times 2}$
• Action: $a_t \in \{\text{buy}, \text{close}, \text{skip}\}$
• Reward:

$r_t = \begin{cases} 100\cdot \left( \frac{p_t^{\text{close}}}{p_{t-1}^{\text{close}}} - 1 - \beta \right), & \text{if open at $t$} \ 0, & \text{otherwise} \end{cases}$

where $\beta = 0.0025$. DQN uses Double-DQN, dueling architectures, and $n$-step targets.

SAM (PPO-based)

• State: $V^+_t \in \mathbb{R}^{f\times m^*\times n}$, stacking $f$ features across $m^*$ assets+cash in a rolling window.
• Action: $a_t$ with $a_t\in\mathbb{R}^{m^*}$, $\sum_i a_{i,t}=1$, $a_{i,t}\geq 0$.
• Reward:

$$r^*_t = \ln(a_t\cdot y_t - \beta \sum_{i} |a_{i,t} - w_{i,t}| - \phi \sigma_t^2)$$

with risk-penalty $\phi=0.001$, price-volatility $\sigma_t^2$.

• PPO Objective:

$$L^{\text{PPO}}(\theta) = E_t \left[ \min\big(r_t(\theta) \hat{A}_t,\, \text{clip}(r_t(\theta),1-\epsilon,1+\epsilon)\hat{A}_t\big) \right]$$

(Huang et al., 2021).

3. Algorithmic Workflow and Internal Loops

3.1. PMA Operation

The PMA operates through a structured loop:

• Apply the current metaheuristic $A_{i(t)}$ on $X(t-1)$
• Compute $\Delta F_{i(t)}(t)$, $S_{i(t)}(t)$, $E_{i(t)}(t)$, and $C_{i(t)}(t)$ over the feedback window
• Update $f_j(t)$ for all $j$, propagate feedback via softmax to probabilities $P_j(t)$
• Query PMSA or built-in logic for switching decision; transfer population if switching occurs, ensuring elite/persistent candidates migrate This framework embodies a nonstationary multi-armed bandit, exploiting statistical signals for adaptive search while mitigating stagnation (Esfahani et al., 20 May 2025).

3.2. Modular Training in PM-Agent (MSPM)

• EAMs are trained/fine-tuned asynchronously using Double DQN; each receives time series + news data and emits “signal-comprised” outputs.
• Once EAMs are ready, the SAM constructs episodes by stacking per-asset EAM signals and learns a continuous action control policy via PPO.
• Periodic retraining of SAM occurs as EAMs or the asset universe change (Huang et al., 2021).

4. Empirical Performance

Table: PMA and MSPM Benchmarks

Agent/System	Final Fitness or ARR	Baseline Comparison	Key Metrics
PMA (CEC2022 F12, 10-D)	10 954.30	PSO: 14 254.40, GA: 14 007.14	$\approx9$ iters to 90% opt.
MSPM (Portfolio a)	ARR ≈131.5%	CRP: 45.9%, ARL: 88.1%	DRR 0.404%, SR 2.86
MSPM (Portfolio b)	ARR ≈566.6%	CRP: 120.6%, ARL: 107.6%	DRR 0.938%, SR 4.18

In PMA’s evaluation, convergence to 90% optimum required only 9 iterations, compared to 25 and 22 for GA and PSO, respectively. The average number of switches per run was 15, with early emphasis on exploration and a late shift to exploitative algorithms (e.g., ACO, SA). These properties directly correlate with PMA's ability to dynamically configure optimization trajectories, blending strengths of diverse heuristics and avoiding local optima entrapment (Esfahani et al., 20 May 2025).

Empirical results for MSPM on U.S. equity data show pronounced gains: for a portfolio comprising AAPL, AMD, and GOOGL, ARR of 131.5% is achieved compared to 45.9% for CRP and 88.1% for ARL. With GOOGL, NVDA, and TSLA, ARR is 566.6% versus 120.6% (CRP). Ablation studies confirm the criticality of EAMs, with ARR dropping to ≈–5.9…+9.8% in EAM-disabled configurations (Huang et al., 2021).

5. Feedback Loops, Adaptivity, and Theoretical Interpretation

The feedback loop in PMA comprises immediate extraction of performance deltas, stagnation measures, and exploitation–exploration indicators, which are aggregated into a composite score for softmax-based selection. This real-time, closed-loop adaptivity is analogous to reinforcement learning multi-armed bandits with nonstationary reward streams. Although formal convergence proofs are not presented, the system heuristically approaches the optimal allocation of search effort by dynamically increasing exposure to high-performing heuristics. PMSA can exist as a lightweight statistical filter or, for advanced context-awareness, as a downstream Retrieval-Augmented Generation (RAG)/LLM module processing historical data and reasoning for switching advice (Esfahani et al., 20 May 2025).

In MSPM, asynchronous updates and decoupling between asset-level and portfolio-level agents ensure that the system adapts efficiently to changes in market structure or data heterogeneity. Transfer learning among EAMs supports reusability and scalability (Huang et al., 2021).

6. Scalability and Integration Patterns

PMA and PMSA collectively embody a modular, actor–critic paradigm: the PMA enacts low-level search, while PMSA provides a switching policy. This separation enables linear scaling with numbers of metaheuristics, problem dimensionality, and distributed computation, as each instance can execute independently with periodic exchange of summary statistics. Population transfer strategies involve elite preservation, candidate mapping, and diversity restarts to efficiently manage state as algorithms switch (Esfahani et al., 20 May 2025).

In the MSPM paradigm, EAMs are plug-and-play modules for arbitrary assets. Portfolio expansion does not require retraining all components—only new EAMs for new assets, and if desired, fine-tuning of SAM. Asynchronous, parallel retraining further increases practical scalability in high-frequency or highly volatile environments (Huang et al., 2021).

7. Key Application Domains and Extensions

PMA is agnostic to the underlying problem, finding application in engineering optimization, logistics, and dynamic decision-making systems that require mitigation of algorithmic stagnation and real-time adaptation. The agent's modular feedback and switching logic is suitable for both centralized and distributed deployments and compatible with future integration of AI-driven advisors such as LLMs for context-aware reasoning (Esfahani et al., 20 May 2025).

The PM-Agent in MSPM is tailored for financial portfolio control, leveraging heterogeneous, multi-modal data sources for robust asset-level signal extraction and flexible, risk-penalized allocation at the portfolio level. Empirical evidence suggests pronounced improvement in accumulative and daily return rates, Sortino ratios, and risk-adjusted performance over classical and advanced RL-driven baselines in back-tested financial markets (Huang et al., 2021).

A plausible implication is the deployment of such architectures in other domains requiring robust, scalable, and adaptive multi-agent decision-making under uncertainty.