Actor-Curator: Co-Adaptive Systems in RL & Archives

• Actor-Curator is a dual-role paradigm where an 'actor' performs tasks while a 'curator' dynamically selects and structures training or narrative content.
• In reinforcement learning, the framework uses adaptive sampling with neural utility estimates and online stochastic mirror descent to optimize policy improvement with proven regret bounds.
• In embodied archive exploration, users co-construct immersive narratives by physically interacting with digital archives, blending performance with real-time curatorial control.

The term Actor-Curator has emerged in recent technical literature to denote a class of co-adaptive systems in which an "actor" and a "curator" interact to drive adaptive learning or experiential narrative via iterative decision processes. The Actor-Curator paradigm appears in two prominent domains: (1) reinforcement learning (RL) for post-training LLMs, where the actor is the policy being trained and the curator dynamically selects training samples; and (2) embodied exploration of large audiovisual archives, where visitors become both performers and curators of multimedia narratives. Across these applications, the Actor-Curator concept operationalizes a duality: agents not only act within an environment or dataset but also select, structure, or remix the material that constitutes their learning or experiential trajectory.

1. Reinforcement Learning Post-Training: Actor-Curator as Co-Adaptive Curriculum (Gu et al., 24 Feb 2026)

In the RL post-training setting for LLMs, Actor-Curator refers to a fully automated curriculum learning architecture wherein a neural curator adaptively samples problems to maximize the actor's policy improvement. The framework assumes a pretrained LLM $\pi$, a reward model $R(y|x) \in [0,1]$, and a large problem bank $P$, with the objective to maximize expected reward:

$$J(\pi) = \mathbb{E}_{x \sim p_P,\, y \sim \pi(\cdot \mid x)} [R(y|x)]$$

The learning loop is as follows: at each RL iteration, the curator proposes a candidate subset $\tilde{X}^t \subset P$ (e.g., $|\tilde{X}^t|=2048$), scores these using neural parameters $\phi$, selects a training batch $X^t$, and gathers rollouts to update $\pi$. Empirical utilities $\tilde{u}^t_x$ are estimated for $x \in X^t$ based on the marginal policy improvement, providing bandit-style feedback to the curator. Iterative updates leverage online stochastic mirror descent (OSMD) over the curator's proposal distribution, balancing exploration and exploitation with theoretical regret guarantees under nonstationarity and partial (semi-bandit) observability.

2. Bandit-Based Curriculum Selection and Mirror Descent Loss

Problem selection is framed as a non-stationary stochastic multi-armed bandit problem: for each candidate $x\in P$, utility $u^t_x$ reflects the expected performance gain for the actor due to training on $x$. The curator maintains a distribution $p^t$ over problems (with $\alpha$-clipping to ensure exploration), and receives updates only for selected $x \in X^t$. The ideal update solves

$$p^{t+1} = \arg\min_{p\in\Delta_\alpha(P)} \left\{ -\eta \langle p, \tilde{u}^t \rangle + \mathrm{KL}(p\|p^t) \right\}$$

In practice, the scoring function is parameterized by a neural network $w_\phi(x)$, and the update is performed via a clipped PPO-style surrogate loss:

$$\mathcal{L}_\mathrm{PCO}(\phi) = -\eta \sum_{x\in\tilde{X}^t} \min( \rho_\phi(x) g^t(x), \operatorname{clip}(\rho_\phi(x),1-\epsilon,1+\epsilon) g^t(x) )$$

with $$\rho_\phi(x) = p_\phi(x | \tilde{X}^t)/p^t(x|\tilde{X}^t)$$ and $g^t(x)=p_P(x)\hat{A}^t(x)/q(x)$.

3. Theoretical Guarantees: Dynamic Regret Under Nonstationarity

A dynamic regret bound is established by comparing Actor-Curator to the best sequence of problem-distributions in hindsight:

$$\operatorname{Reg}_T = \sum_{t=1}^T \left[ f_t(p^{*t}) - f_t(p^t) \right]$$

where $f_t(p) = -\langle p, u^t \rangle$, and $$V_T = \sum_{t=2}^T \max_{x\in P} |u^t_x - u^{t-1}_x|$$ quantifies nonstationarity. The paper proves

$\operatorname{Reg}_T \leq O(T^{2/3} V_T^{1/3})$

under unbiasedness and boundedness conditions, matching the $\sqrt{T}$ stationary bound when $V_T = 0$. The proof employs conditional OSMD analysis, importance weighting for unbiasedness, and block-decomposition to handle nonstationarity (Gu et al., 24 Feb 2026).

4. Quantitative Performance, Ablations, and Implementation

Actor-Curator demonstrates consistent improvements over standard curricula on challenging reasoning benchmarks, achieving test accuracy gains up to 30.5% (ARC-1D) and 28.6% (AIME24) relative to strong baselines and showing up to 80% speedup in convergence. Stability is maintained over long RL runs, surpassing plateaus present in other approaches. Key ablations show performance degrades 5–10% when using alternative utility signals or simple regression, and practical deployments benefit from two-stage sampling, moving-average smoothing, and $\alpha$-clipping to prevent premature collapse.

Benchmark	Uniform	SEC	PCL	Actor-Curator	$\Delta$%
AIME24	23.33%	20.00%	23.33%	30.00%	+28.6%
ARC-1D	26.74%	27.87%	26.37%	36.37%	+30.5%
Zebra-hard	30.50%	27.50%	26.00%	34.50%	+13.1%

Practical guidelines include initialization (“warmup” steps with uniform sampling), hyperparameters (candidate batch 2048, train batch 256, learning rates $10^{-6}$), and infrastructure for scalable curation.

5. Actor-Curator in Embodied Archive Exploration (Alliata, 2023)

In interactive media and cultural informatics, Actor-Curator denotes a user role that fuses experiential “acting” within a digital archive with on-the-fly curation. Notably, Alliata’s immersive archive frameworks empower visitors to perform embodied interactions—walking, reaching, selecting—mapped isomorphically to traversal and selection within large digitized audiovisual collections. System architectures (e.g., Panorama+ with 360° projection and Linear Navigator with physical rail) provide real-time tracking and spatial metaphors. Visitors both enact movement through the archive ("actor") and sequence, remix, or chain video fragments ("curator"), thus co-constructing emergent narrative structures.

Formal mappings utilize timestamp-based polar coordinate layouts, with each clip $v_i$ positioned:

$$r_i = R_0 + \alpha(y_i - y_{\min}), \quad \theta_i = 2\pi (m_i/12) + \phi_i$$

and

$$\mathbf{p}_i = (r_i \cos\theta_i, 0, r_i \sin\theta_i)$$

Physical interaction (e.g., sliding a screen along $s \in [0, L]$) maps to traversing archive years; active regions $V_\mathrm{active}(s)$ are windowed subsets centered on the current year. More advanced retrieval could integrate tag- and visual-similarity-based scoring, though full retrieval algorithms are yet unpublished.

6. Design Principles and Evaluation Methodologies

Both technical implementations and user-experience frameworks extract design guidelines from the Actor-Curator duality:

1. Embodied 1:1 mapping: Physical movement is mapped near-isomorphically to archive traversal.
2. On-the-fly curation (“detach+chain”): Always enabling selection, recombination, and sequencing operations.
3. Social visibility: Supporting multi-user spaces with active and spectating participants; narrative emergence involves both personal and collective agency.
4. Content-anchored metaphors: E.g., time as orbit, heritage as constellation.
5. Mixed granularity: Enabling transitions from coarse navigation (e.g., decades) to fine-grained snippet editing.
6. Progressive disclosure: Curation boundaries are initially visible, with affordances for user-driven reconfiguration.

Planned evaluations (as of April 2022) include within-subjects studies measuring creativity (Sternberg’s 8P framework) and sense of agency (Likert-scale indices), plus behavioral logs of branching and remix activity. Expected outcomes are higher originality and agency relative to conventional interfaces.

7. Synthesis and Implications

Across computational learning and digital archive interaction, the Actor-Curator paradigm encapsulates co-adaptive selection and performance. In RL, it yields principled, efficient curricula with regret guarantees, outperforming prior sampling regimes. In media archives, it transforms passive browsing into embodied, generative engagement, architecting “playful performance” where every selection is a curatorial and narrative act. A plausible implication is the broader applicability of Actor-Curator frameworks for any domain requiring both continuous adaptation and active structuring of complex informational or experiential spaces (Gu et al., 24 Feb 2026, Alliata, 2023).