Multi-Agent Video Recommender Systems

• Multi-Agent Video Recommendation Systems (MAVRS) are architectures that decompose video recommendation into specialized modules such as perception, reasoning, memory, and explanation.
• They employ collaborative patterns like hierarchical orchestration and pipeline-based modularity to optimize for multi-objective metrics including fairness, safety, and long-term user satisfaction.
• Recent systems integrate reinforcement learning and large language models to enhance content understanding, negative feedback handling, and user control, yielding measurable improvements over traditional models.

Searching arXiv for papers on multi-agent video recommendation systems and the specific cited works. Multi-Agent Video Recommendation Systems (MAVRS) are recommender systems in which multiple specialized, semi-autonomous agents cooperate—and sometimes compete—to produce, refine, explain, or evaluate video recommendations. In the recent literature, MAVRS denotes a shift away from monolithic, single-objective architectures toward systems that decompose video recommendation into perception, reasoning, memory, search, simulation, explanation, and constraint-handling modules. The motivation is recurrent across the literature: traditional recommenders optimize static engagement metrics, struggle with multimodal video semantics, expose limited control and explainability, and are increasingly inadequate for fairness, safety, and long-term satisfaction requirements (Ranganathan et al., 2 Apr 2026).

1. Formalization and scope

A recent agentic perspective formalizes an individual LLM agent as

$$A_{\mathrm{LLM}} = \bigl(\mathcal{M}, \mathcal{I}, \mathcal{O}, \mathcal{F}, \Omega \bigr),$$

where $\mathcal{M}$ is the underlying LLM, $\mathcal{I}$ the input space, $\mathcal{O}$ the output space, $\mathcal{F}$ the tool/API set, and $\Omega$ the memory state. A multi-agent recommender is then written as

$\mathrm{MAS} = (\mathcal{A}, \mathcal{E}, \Pi),$

with agents $\mathcal{A}$, shared environment $\mathcal{E}$, and interaction protocol $\Pi = (\mathbf{C}, \Gamma)$, where $\mathcal{M}$0 is a communication matrix and $\mathcal{M}$1 a message schema set. In a video setting, $\mathcal{M}$2 naturally includes a video catalogue, watch logs, safety models, and policy knowledge bases, while $\mathcal{M}$3 can include candidate lists, safety reports, explanations, and simulated user actions (Maragheh et al., 2 Jul 2025).

Within this formalism, MAVRS encompasses both interactive and non-interactive recommendation. For interactive recommendation, the task is written as

$\mathcal{M}$4

where $\mathcal{M}$5 is the session transcript and $\mathcal{M}$6 is the recommended list. The same perspective explicitly connects MAVRS to multi-modal recommendation, user simulation, and explanation generation, which broadens the concept beyond ranking alone toward complete recommendation pipelines (Maragheh et al., 2 Jul 2025).

The survey literature treats MAVRS not as a single architecture but as a family of systems for short-form video, long-form storytelling, and educational video. Its defining properties are decomposition into roles, explicit coordination, and the increasing use of LLMs and multimodal foundation models as “cognitive cores” inside agents (Ranganathan et al., 2 Apr 2026).

2. Collaborative patterns and agent roles

The dominant taxonomy identifies four collaborative patterns: hierarchical orchestration, pipeline-based modular collaboration, user-agent collaboration, and user simulation agent ensembles. Hierarchical orchestration uses a coordinator that weighs subordinate outputs; pipeline-based collaboration chains specialized agents sequentially; user-agent collaboration exposes recommendation control through conversation; and simulation ensembles use agent societies to model users offline (Ranganathan et al., 2 Apr 2026).

The corresponding role taxonomy is stable across papers. Perception or video-understanding agents consume frames, captions, audio, or metadata and emit summaries or embeddings. Reasoning or ranking agents transform user history and item representations into recommendations. Auxiliary signal agents specialize in individual feedback dimensions such as likes, follows, or comments. Memory agents maintain long-term user state. Search agents access large corpora. Interpreter agents convert natural language into structured tasks. Reflection or critique agents evaluate intermediate outputs. User simulation agents model reactions for training or evaluation (Ranganathan et al., 2 Apr 2026).

MACRec is an early recommender-oriented LLM multi-agent framework rather than a simulation-only system. It defines a Manager, User/Item Analyst, Reflector, Searcher, and Task Interpreter, and supports rating prediction, sequential recommendation, conversational recommendation, and explanation generation. Its design shows a reusable orchestration pattern for video domains in which watch history, item attributes, external knowledge, and reflection are handled by distinct agents rather than a single monolithic model (Wang et al., 2024).

Industrial video systems instantiate the same pattern with domain-specific roles. MMRF uses a main WatchTime agent together with auxiliary agents for Click, Like, Follow, Comment, Hate, and LongView. VRAgent-R1 separates an Item Perception agent from a User Simulation agent. ENF separates a Profile Agent, Video Agent, and Reason Agent. These systems differ in learning algorithms and deployment targets, but they share the same principle of role specialization and inter-agent communication (Zhou et al., 2024, Chen et al., 3 Jul 2025, Chen et al., 24 Nov 2025).

3. Content understanding, memory, and structured retrieval

Video semantics are a central driver of MAVRS. The survey literature argues that video is high-dimensional, temporal, and multimodal, so a single model cannot ingest full histories at the pixel level and reason over them efficiently. A common solution is to let perception agents compress raw video into textual or embedding summaries that other agents can reason over (Ranganathan et al., 2 Apr 2026).

VRAgent-R1 provides a concrete content-understanding pipeline. Its Item Perception Agent performs Key Frame Retrieval, Collaborative Multimodal Perception, and Recommendation Relevant Analysis. On MicroLens-100K, the IP Agent improves NDCG@10 by 6.0% and HR@10 by 4.3% over the multimodal SASRec fusion baseline, and the improvement exceeds 10% on cold-start users in HR@10 and NDCG@10 (Chen et al., 3 Jul 2025). The paper also shows that directly using long MLLM-generated descriptions harms ranking, whereas short recommendation-oriented captions are beneficial, which indicates that abstraction granularity is itself an optimization variable.

MAVIS extends this content-side idea into a structured retrieval substrate. Each video is parsed into a concise caption and a structured semantic tuple

$\mathcal{M}$7

with scene, object, and action sets indexed in a Structured Semantic Library. A planner decomposes a query into role-specific sub-intents, specialized agents retrieve candidates per dimension, and a Logic-aware Debate mechanism with a strict veto protocol prunes mismatches before expensive fine-grained matching. On MSR-VTT, MAVIS reports R@1 of 78.6% with about 3,200G FLOPs, compared with 50.8% and about 100,000G FLOPs for brute-force ViCLIP, and its ablations show that “Union + Veto” yields the best trade-off between recall and candidate-set size (Zhang et al., 8 Jun 2026). This suggests that structured retrieval modules can function as content-side subsystems within MAVRS, especially when recommendation requires explicit control over scene, object, or action semantics.

Memory is the complementary substrate. The agentic recommender perspective formalizes working, episodic, semantic, and procedural memory, with explicit update, retrieval, and decay operators, including token-budgeted retrieval under a knapsack constraint (Maragheh et al., 2 Jul 2025). In video recommendation, this memory hierarchy maps naturally onto short-term session state, episodic watch sessions, long-term semantic tastes, and stored procedures for retrieval, safety checks, or explanation generation.

A related line models stable user values rather than transient interests. ZOOM uses summarization, evaluator agents, supervisor agents, and debate to infer Schwartz-style user values from historical interactions, then aligns user embeddings and value embeddings with a contrastive loss. On MovieLens-1M, ZOOM improves MoRec HR@10 from 0.2586 to 0.2725 and NDCG@10 from 0.0393 to 0.0415, while EasyRec improves HR@10 from 0.1957 to 0.2749 and NDCG@10 from 0.0224 to 0.0335 (Chen et al., 2 May 2025). This suggests that MAVRS can use agentic value mining as a long-term preference layer complementing session-level video behavior.

4. Learning, optimization, and multi-objective control

A major branch of MAVRS is multi-agent reinforcement learning. MMRF formulates a recommendation session as a multi-agent MDP in which the $\mathcal{M}$8-th agent maximizes WatchTime and the remaining $\mathcal{M}$9 agents optimize auxiliary feedback dimensions. The main objective is

$\mathcal{I}$0

Each auxiliary agent outputs ranking scores, attention-based collaboration aggregates beneficial information across agents, and a model-based user feedback simulator generates non-impression samples to alleviate sample selection bias (Zhou et al., 2024). On KuaiRand-1k, MMRF improves WatchTime GAUC by 7.3% and NCIS by 7.1% over baselines, and in online A/B testing it improves WatchTime by 0.55%, Depth by 0.54%, Follow by 1.45%, and Comment by 1.28% over LTR while serving over hundreds of millions of users (Zhou et al., 2024).

Reinforcement fine-tuning of language agents appears in more recent systems. VRAgent-R1 uses Group Relative Policy Optimization for a User Simulation Agent that reasons in CoT form over history and candidate captions. Its US Agent shows approximately 45.0% higher accuracy in user decision simulation compared to state-of-the-art baselines, reaching 0.715 accuracy and 0.727 F1 on MicroLens-100K preference judgment, and 0.641/0.602 accuracy for next-video selection with 3 and 4 negatives respectively (Chen et al., 3 Jul 2025).

ENF introduces Step Group Relative Policy Optimization for hierarchical negative-feedback reasoning. The method decomposes reward into judge, class, and reason rewards, with advantage normalization

$\mathcal{I}$1

and progressive reward propagation from easier to harder subtasks. The Video Agent uses a 3-step reward on explicit negative feedback, while the Reason Agent uses a 2-step reward on implicit fast-skip signals (Chen et al., 24 Nov 2025).

Another optimization line emphasizes hard constraints. DualAgent-Rec separates optimization into an Exploitation Agent using the Constraint Domination Principle and an Exploration Agent using unconstrained Pareto dominance, with an LLM coordinator adjusting resource allocation and adaptive $\mathcal{I}$2-relaxation tightening constraints over time. On Amazon Reviews 2023, DualAgent-Rec achieves 100% constraint satisfaction and improves Pareto hypervolume by 4–6% over strong baselines while maintaining competitive accuracy-diversity trade-offs (Zhang et al., 27 Jan 2026). Although the reported experiments are in e-commerce, the paper explicitly maps fairness, coverage, new-item exposure, and safety constraints to video recommendation, so its main contribution to MAVRS is architectural rather than domain-specific (Zhang et al., 27 Jan 2026).

5. Explainability, negative feedback, and user governance

A distinctive recent direction in MAVRS is the treatment of negative feedback as a first-class signal. ENF targets a Tencent News short-video/news-video environment with explicit dislikes, fast skips, and low play-rate interactions. It defines multi-granular negative feedback at binary, categorical, and textual levels, and operationalizes the task as a three-stage hierarchy: binary judgment, multi-choice reason classification, and detailed explanation generation (Chen et al., 24 Nov 2025).

Its agent structure is explicitly user–content–interaction centered. The Profile Agent extracts behavioral cues and psychographic tags from demographics and watch history; the Video Agent performs multimodal analysis over titles and 16 sampled frames using Qwen2.5-VL-7B; and the Reason Agent predicts fast skips or dislikes and explains them from the user’s perspective (Chen et al., 24 Nov 2025). On the TVNF benchmark, the Video Agent reaches Recall 0.808, F1 0.750, Class_Acc 0.654, and Reasoning 0.537, yielding an 8.6% improvement over GPT-4o in reason classification and +13.5% in explanation quality. In online deployment, Base RS + ENF raises Avg_Time from 47.6% to 53.8%, lowers Fast-skip Rate from 23.7% to 14.3%, and lowers Dislike Rate from 0.61% to 0.35% (Chen et al., 24 Nov 2025).

A complementary line focuses on user-governed filtering. MAP-V is a multimodal multi-agent filtering layer for discomforting recommendations in feeds containing text, images, and short videos. It separates Intent Parser, Visual Feature Extraction Agent, Rule Judge Agent, Dispute Agent, and a Reflection-based Adaptive Hierarchy agent. The core design principle is fact-grounded adjudication: visual agents describe only observable features, judge agents may trigger rules only from explicit text or visual facts, and appeals create a human–AI negotiation loop (Zhang et al., 19 Apr 2026).

On an adversarial dataset of 473 highly confusing samples, MAP-V reduces false positives by 74.3% relative to a text-only LLM baseline and reaches Precision 0.6061, Recall 0.8696, and F1 0.7143, compared with 0.2519, 0.7391, and 0.3757 for the text-only LLM (Zhang et al., 19 Apr 2026). In a 7-day field study with 19 participants and 66,603 exposures, it records Precision 0.9645, Recall 0.7259, F1 0.8283, an interception gain of +134.7%, and a decrease in manual cost per final block from 0.59 to 0.36 events per block (Zhang et al., 19 Apr 2026). Its dual-layer preference graph, with

$\mathcal{I}$3

and bias decay

$\mathcal{I}$4

adds an explicit user-control layer missing from most classical recommenders (Zhang et al., 19 Apr 2026).

Together, these systems show that explainability in MAVRS is not limited to post-hoc rationale generation. It increasingly includes reason taxonomy design, explicit negative-feedback modeling, user-editable rules, disagreement handling, and auditable traces.

6. Representative systems, empirical landscape, and open challenges

The current empirical landscape can be summarized through several representative systems.

System	Agent structure	Reported outcome
MMRF	Main WatchTime agent + auxiliary feedback agents	WatchTime +0.55% online; KuaiRand-1k WatchTime GAUC +7.3% (Zhou et al., 2024)
VRAgent-R1	Item Perception Agent + User Simulation Agent	IP Agent +6.0% NDCG@10; US Agent about 45.0% higher simulation accuracy (Chen et al., 3 Jul 2025)
ENF	Profile Agent + Video Agent + Reason Agent	Avg_Time 53.8% vs 47.6%; Fast-skip 14.3% vs 23.7% (Chen et al., 24 Nov 2025)
MAP-V	Intent Parser + Visual + Judge + Dispute + RAH	False positive rate reduced by 74.3%; F1 0.7143 (Zhang et al., 19 Apr 2026)
MAVIS	Planner + Scene/Object/Action agents + Debate Controller	R@1 78.6% with about 3,200G FLOPs (Zhang et al., 8 Jun 2026)

Despite this progress, the challenge set is consistent across the literature. The MAVRS survey isolates cost and scalability, multimodal grounding, evaluation, controllability and trustworthiness, and incentive alignment as the main unresolved problems (Ranganathan et al., 2 Apr 2026). The agentic recommender perspective adds protocol complexity, hallucination and error propagation, emergent misalignment including covert collusion, and brand compliance as formal challenge families (Maragheh et al., 2 Jul 2025).

These concerns are visible in system papers. ENF reports sparse explicit negative feedback, reliance on GPT-4o-assisted labeling, domain-specific taxonomies, and per-query cost from Qwen2.5-VL-7B and GPT-4o (Chen et al., 24 Nov 2025). MAP-V reports latency vulnerabilities, privacy concerns in cloud-edge collaboration, missing audio/full-video modeling, and residual false positives from strong trigger words (Zhang et al., 19 Apr 2026). MAVIS reports threshold sensitivity, limited semantic coverage beyond scene/object/action, and bias inheritance from underlying MLLMs and VLMs (Zhang et al., 8 Jun 2026). DualAgent-Rec reports that dual-population optimization plus LLM calls is too slow for real-time feeds and is therefore more naturally an offline or nearline orchestration layer (Zhang et al., 27 Jan 2026).

A plausible implication is that future MAVRS will continue to hybridize efficient recommendation backbones with agentic sidecars rather than replace ranking stacks outright. The survey explicitly identifies hybrid RL–LLM architectures, lifelong personalization, human-in-the-loop validation, and self-improving recommender systems as central research directions (Ranganathan et al., 2 Apr 2026). A second plausible implication is that distributed video-processing ideas such as DMVF’s consensus-based importance estimation and multi-budget strategy allocation may become relevant for resource-aware MAVRS at the edge, where recommendation, filtering, and prefetching must be coordinated under compute and bandwidth constraints (Lan et al., 2020).

In that sense, MAVRS is best understood not as a narrow algorithmic family but as an architectural reorganization of video recommendation. Its central claim is that perception, ranking, simulation, explanation, constraint satisfaction, and user control are sufficiently heterogeneous that they are better handled by coordinated specialist agents than by a single opaque model. The literature now contains industrial ranking systems, explainable negative-feedback systems, controllable filtering systems, training-free structured retrieval systems, and formal agentic frameworks that together define the field’s present shape and its most credible next steps (Zhou et al., 2024, Chen et al., 24 Nov 2025, Zhang et al., 19 Apr 2026, Zhang et al., 8 Jun 2026, Maragheh et al., 2 Jul 2025).