Multi-Agent Self-Supervision (MASS) Overview

• Multi-Agent Self-Supervision (MASS) involves methods where multiple interacting agents generate training or control signals from their observations and environmental outcomes, focusing on task execution and supervision.
• MASS systems use interaction-derived information for learning, including temporal transitions, peer responses, and cross-agent observations, to construct targets, evaluate behavior, and modify future agent behavior.
• Applications of MASS include multi-agent imitation learning, predictive representation learning, and runtime supervision, with notable improvements in tasks like multi-agent imitation, video behavior analysis, and LLM-based system design.

Multi-Agent Self-Supervision (MASS) is a family of methods in which multiple interacting agents generate training or control signals from their own observations, trajectories, responses, relationships, execution traces, or environmental outcomes. MASS does not denote a single algorithm. Its instances range from self-supervised multi-agent imitation learning and masked latent reconstruction to peer-based visual supervision, adaptive communication, runtime intervention, executable multi-agent-system synthesis, and recursive optimization of agent organizations. The defining property is that multi-agent structure is used not only for task execution but also for producing supervision that shapes representations, rewards, policies, communication topologies, workflows, or model parameters.

1. Conceptual scope and defining properties

MASS is distinguished from ordinary multi-agent coordination by the use of interaction-derived information as a learning signal. Coordination organizes agents to solve a task; MASS additionally uses the resulting interactions to construct targets, evaluate behavior, attribute failures, or modify future agent behavior and organization. Such signals may arise from temporal transitions, cross-agent observations, peer responses, occupancy relationships, execution traces, tool outcomes, or automatically verified task results.

A broad MASS process can be represented as:

$$\text{execute} \rightarrow \text{observe} \rightarrow \text{evaluate} \rightarrow \text{attribute} \rightarrow \text{update} \rightarrow \text{reorganize}.$$

The updated object may be an agent representation, policy, prompt, memory, role assignment, communication graph, workflow, model allocation, or complete multi-agent system. The supervision may be local or global, online or offline, parameter-free or parameter-updating, and internally generated or grounded by an external evaluator.

The term self-supervision requires qualification. Some MASS systems construct targets entirely from observations and interactions. Others use automatically generated labels, benchmark answers, compiler tests, simulators, or external evaluators. These latter systems are self-supervised with respect to intermediate behavior but not fully label-free. For example, SS-MAIL uses expert demonstrations as its central imitation target while generating additional labels through trajectory interpolation (Dharmavaram et al., 2021). MHGPO uses terminal answer scores derived from gold answers in its main experiments (Chen et al., 3 Jun 2025). MAS-GPT filters generated systems using executable correctness checks and ground-truth answers (Ye et al., 5 Mar 2025). MASS therefore includes a spectrum from intrinsic self-supervision to evaluator-assisted autonomous optimization.

Several recurring properties characterize the field:

• Self-generated targets: future observations, masked latent states, peer actions, interpolated trajectories, critiques, or execution outcomes serve as automatically available targets.
• Relational supervision: agent identity, role, interaction, influence, topology, or joint state determines the meaning of the signal.
• Trajectory-level learning: supervision may be derived from complete sequential behavior rather than isolated state-action pairs.
• Structural adaptation: the system may change roles, communication links, workflows, model assignments, or agent populations.
• Closed-loop improvement: execution produces data that affects subsequent execution or later training.
• Partial observability: many systems learn from local observations while using peer behavior or environmental consequences to infer unobserved state.

MASS should therefore be distinguished from unsupervised multi-agent clustering, ordinary hierarchical aggregation, and fixed-role collaboration. A system may contain multiple agents and exchange information without producing any learning signal from those interactions. Conversely, a system such as MA2CL is MASS-like because its masked-agent reconstruction target, negative keys, temporal replacement mask, and attention context are all defined by the team configuration (Song et al., 2023).

2. Principal methodological families

Trajectory and reward self-supervision

In self-supervised imitation learning, demonstrations remain important but additional supervision is induced from relationships between expert and learner behavior. SS-MAIL assigns discriminator values of $0$ to generated state-action pairs, $1$ to expert pairs, and an interpolated value $\alpha$ to synthetic intermediate pairs generated from:

$\tau_\alpha=\alpha\tau_G+(1-\alpha)\tau_E.$

The resulting squared-error objective shapes the discriminator into a graded reward model rather than a binary classifier. The policy then uses the discriminator output as a reward and optimizes it with SAC. SS-MAIL combines this self-supervised adversarial objective with a graph-based centralized actor-critic, a learned latent interaction graph, and Trajectory Forcing, which progressively increases generated rollout length (Dharmavaram et al., 2021).

The method illustrates a central MASS principle: self-supervision can regulate the geometry of a learning signal rather than merely improve a representation. Intermediate samples provide dense local gradients, global reward shaping, exploration guidance, and mode-specific feedback. The approach remains demonstration-dependent and assumes that state and action interpolation is meaningful and aligned across trajectories.

A different trajectory-based family uses system-level reinforcement learning. MHGPO trains a three-role search system comprising a Rewriter, Reranker, and Answerer. The final answer receives a shared reward, which is propagated backward through the dependency chain and combined with role-specific penalties. Group-relative advantages eliminate the need for a Critic:

$$\hat A_{k,i}= \frac{R_{k,i}-\mu_{m_{k,i}}} {\sigma_{m_{k,i}}+\delta}.$$

Independent Sampling, Fork-on-First, and Round-Robin sampling trade off within-group comparability, end-to-end coordination, and computational efficiency. MHGPO-RR obtains the strongest reported HotpotQA results among the evaluated methods, while MHGPO-FoF performs best on MuSiQue (Chen et al., 3 Jun 2025). Because the main experiments use gold answers and token-level F1, this is evaluator-assisted self-supervision rather than completely label-free learning.

Predictive and masked representation learning

Predictive MASS uses the temporal and relational structure of multi-agent observations to generate auxiliary targets. The Adaptive Multi-Agent Continuous Learning System performs next-input prediction and clustering. Lower-level agents encode local temporal patterns, upper-level agents aggregate cluster codes, and higher-level predictions are routed downward as feedback. The system uses longest common subsequence comparisons and maintains clusters online. Its reported video experiment shows improved adaptation when a third behavior type is introduced, while removing predictive feedback impairs convergence (Qian et al., 2022).

MA2CL formulates a more explicit representation-learning objective for cooperative MARL. A subset of agents is masked by replacing current observations and actions with previous-step values:

$$o_t^{i_m}\leftarrow o_{t-1}^{i_m}, \qquad a_t^{i_m}\leftarrow a_{t-1}^{i_m}.$$

An online encoder processes the masked team state, while an EMA target encoder processes the original observations. An attention reconstruction model predicts the current latent representation of the masked agent. InfoNCE treats the corresponding current representation as the positive key and other agents’ representations as negatives:

$$\mathcal L_{\mathrm{cl}} = \sum_{i=1}^{N} -M_i \log \frac{\exp(\omega(q_i,k_i))} {\sum_{j=1}^{N}\exp(\omega(q_i,k_j))}.$$

The objective is combined with the underlying MARL loss. MA2CL therefore provides temporal prediction, agent-level discrimination, identity awareness, and relational attention without requiring execution-time communication beyond the base policy (Song et al., 2023).

B-KinD-multi applies self-supervision to multi-agent video representation. A pretrained video segmentation system, DEVA, generates temporally propagated agent masks. Heatmaps are masked separately for each agent, yielding $NK$ geometry channels, and the model reconstructs a spatiotemporal difference target using perceptual features. Rotation-equivariance and keypoint-separation losses constrain the representation. The method improves normalized keypoint-regression error over B-KinD on CalMS21, Fly vs. Fly, and PAIR-R24M, with particularly large gains for visually similar interacting animals (Khalil et al., 2024). Its principal MASS contribution is per-agent factorization; it does not itself provide a rich relational model of social dynamics.

Peer supervision and response-driven organization

In LLM-based systems, agents can generate supervision through peer responses and semantic relationships. SelfOrg independently generates responses, embeds them, estimates contribution using cosine similarity to the response centroid, and constructs a directed acyclic communication graph. For response embeddings $\mathbf r_n$, the approximate contribution is:

$0$0

An edge from agent $0$1 to agent $0$2 is formed when their responses exceed a similarity threshold and the source has higher contribution. Cycles are removed by deleting edges associated with lower-contribution nodes. The graph is rebuilt after each collaboration round, and the final response is selected using a contribution-weighted centroid (Tastan et al., 1 Oct 2025).

SelfOrg does not train topology parameters or use an external LLM judge during graph formation. It is consequently an example of online structural self-supervision: responses supervise communication structure, which then determines later response formation. Its theoretical guarantees are conditional on dispersed errors, semantic clustering of correct answers, equal embedding norms, and other assumptions. Correlated errors or a semantically central incorrect response can cause the graph to amplify misinformation.

Runtime and execution supervision

Runtime MASS systems supervise ongoing execution rather than training representations directly. SupervisorAgent attaches a meta-level controller to an existing MAS without modifying the base agent architecture. Each ActionStep contains model output, tool calls, observations, and errors. An LLM-free adaptive filter triggers supervision only for selected conditions, including sub-agent completion, errors, inefficient repetition, and observations exceeding 3,000 characters. The supervisor may approve, provide guidance, correct an observation, or invoke a verification agent (Lin et al., 30 Oct 2025).

This framework is operationally self-supervisory because it derives signals from trajectories, tool calls, errors, and global traces. It does not, however, update supervisor parameters online or learn a trigger policy in the reported experiments. It is best characterized as a runtime supervision layer with execution-derived feedback. On GAIA, the detailed pass@1 table reports unchanged average accuracy and a 29.68% reduction in Smolagent token consumption, although the abstract states 29.45%.

3. Organization-level self-supervision

A distinctive MASS direction treats the multi-agent organization itself as the object of learning. This includes role composition, communication topology, tool allocation, workflow structure, model routing, and runtime repair.

MAS-GPT reframes system construction as conditional code generation:

$0$3

A trained model receives a query and emits executable Python implementing an adaptive multi-agent workflow. The synthetic dataset is built by pairing benchmark queries with candidate systems, executing them, filtering failures, selecting consistent systems for similar queries, and refining successful programs. Training then uses supervised fine-tuning on the resulting query–reasoning–MAS tuples (Ye et al., 5 Mar 2025).

MAS-GPT is automatically supervised but not an online self-training system. Execution outcomes filter the data before SFT; they are not used as a direct policy-learning signal. Its relevance to MASS lies in making the workflow executable and learnable, with agents, prompts, communication, tools, and aggregation represented in one program.

MAS-ZERO and SELF-MAS represent inference-time system redesign. Seed MASs are executed, their sub-question and agent outputs are inspected, and a meta-agent generates revised systems over a fixed number of iterations. Candidate answers are accumulated and finally self-verified. The supplied material reports an ablation in which removing decomposition or meta-feedback reduces the weighted average across AIME24, GPQA, and SWE (Ke et al., 21 May 2025). The algorithm still depends on seed systems, a meta-agent, fixed iteration counts, and a self-verification mechanism; “zero supervision” therefore does not mean absence of human-designed infrastructure.

MAS² extends system construction with a generator–implementer–rectifier architecture. The generator produces roles, tools, and communication protocols; the implementer assigns available LLM backbones to roles; and the rectifier modifies the system when resource use exceeds a threshold or an operational failure occurs. Collaborative Tree Optimization estimates trajectory values and constructs value-weighted preferences for the three meta-agent policies (Wang et al., 29 Sep 2025). MAS² consequently adds runtime structural repair and model allocation to system generation.

OptiMAS treats MAS design as continual textual optimization. Its configuration includes the agent set, DAG orchestration graph, backbone assignments, prompts, skills, tools, and delegation rules. Complete ReAct trajectories, correctness feedback, validation results, and failure cases are passed to an optimizer that generates textual diagnoses and edits the configuration. A dual-track memory stores short-term optimization plans and long-term hypotheses with proposal, enactment, validation, refutation, suspension, and dormancy states (Cheng et al., 22 Aug 2026).

The most recent formalization of MASS in the supplied material alternates evolutionary workflow optimization with SFT on self-generated multi-agent trajectories. For each task, workflows encode roles, instructions, output contracts, and an ordered call topology. Candidate workflows are executed and compared; retained workflows generate multiple trajectories; the strongest trajectories are selected using pairwise evaluator judgments and a Bradley–Terry model; and the resulting orchestrator and subagent conversations are used for SFT. The model is then reused as executor, optimizer, and evaluator in the next cycle (Lee et al., 8 Oct 2026).

This approach defines MASS as recursive co-evolution of a model and its computational organization:

$0$4

The reported two-cycle experiment with Qwen3.6-27B shows increased performance per output token on four open-ended public benchmarks and stronger bare-prompt delegation, code handoff, and revision behavior. Gains are not universal: Terminal-Bench and SWE-bench decline slightly, and the method remains dependent on evaluator reliability and workflow-execution stability.

4. Supervision sources, attribution, and credit assignment

MASS systems use several distinct supervision sources. Their reliability and implications differ.

Temporal structure

Future observations can supervise current representations or cluster assignments. MA2CL uses current unmasked latent states as targets for masked agents, while the Adaptive Multi-Agent Continuous Learning System uses naturally occurring next inputs to verify clustering. Temporal targets are automatically available but depend on alignment, synchronization, and a suitable observation representation.

Relational structure

Other agents provide context, negatives, peer actions, or interaction signals. MA2CL uses other agents’ latent representations as contrastive negatives. Social Influence predicts peer actions and rewards changes in predicted peer distributions, while SVO instead uses other agents’ rewards and is therefore not strictly locally self-supervised (Xiang et al., 2023). In SS-MAIL, learned interaction graphs organize information used by actor and critic networks.

Peer agreement and disagreement

Peer responses can provide weak quality signals through agreement, similarity, ranking, or voting. SelfOrg formalizes semantic alignment with a centroid-based contribution approximation. Debate, self-consistency, and group comparison can expose alternative hypotheses, but consensus may reinforce shared errors. A peer signal should therefore not automatically be treated as ground truth.

Environmental and executable outcomes

Compilers, unit tests, simulators, retrieval results, benchmark evaluators, and tool status provide externally grounded signals. These are often more objective than peer judgments but remain external supervision. They become part of MASS when they are used to generate internal labels, attribute responsibility, modify workflows, or update policies without human intervention.

Trajectories and process traces

A trajectory may contain actions, messages, tool calls, observations, intermediate artifacts, errors, and final outcomes. Process traces allow supervision to move beyond final-answer scoring. OptiMAS uses trajectories to generate textual hypotheses; SupervisorAgent uses them for runtime intervention; LIFE frames them as the central substrate linking collaboration, attribution, and evolution (Qi et al., 14 May 2026).

Failure attribution

A global failure label does not identify which agent, step, tool, or message caused the failure. Responsibility may involve:

• Origin: where an error was introduced.
• Propagation: how the error was transmitted or amplified.
• Exposure: where the error became visible.
• Repair responsibility: which component should be changed.

MAS-OPD addresses this through Privileged Attribution for Coordination. A verifier identifies which agent’s output is supported, but the attribution is supplied only to the teacher during training; students must learn the corresponding revision behavior without seeing the privileged judgment (Zhong et al., 28 Sep 2026).

LIFE organizes these ideas into a progression from capability foundation to collaboration, fault finding, and autonomous evolution. Its central claim is that collaboration determines what failures can be observed, attribution converts failures into targeted supervision, and evolution changes the organization that will generate future supervision (Tang et al., 10 Jun 2026).

5. Empirical applications and findings

MASS has been evaluated across imitation learning, robotics, MARL, video behavior analysis, LLM-based question answering, software engineering, scientific research, and runtime tool use.

In multi-agent imitation, SS-MAIL shows qualitative stability advantages over GAIL and WAIL in Y-junction experiments, supports distinct modes rather than averaged trajectories, and reduces compounding-error growth under Noisy Mocap perturbations (Dharmavaram et al., 2021). In physical swarm robotics, SwarmHawk uses peer visual localization when a drone loses its camera or position signal. A leader observes AprilTag markers on blind followers and the landing platform, while a centralized ground station computes commands. The system reports landing errors of 4.2 cm in homogeneous formation and 1.9 cm in leader-follower formation on a static platform, and 6.9 cm versus 4.7 cm on a moving platform (Gupta et al., 2022). This is MASS in a fault-tolerant sensing sense, but it is centralized control with adaptive hierarchical leader-follower supervision rather than fully distributed learning.

In cooperative MARL, MA2CL improves reward and sample efficiency across the evaluated MAQC, Multi-Agent MuJoCo, and SMAC settings, with benefits generally larger for MAPPO than MAT (Song et al., 2023). By contrast, the realistic-MARL comparison in Clean Up finds that centralized or reward-sharing methods outperform decentralized self-supervised methods. Social Influence and ICM provide only marginal gains over IPPO, while heterogeneous SVO produces stronger role specialization and population return. The gap shrinks in Harvest, where motivations are more aligned (Xiang et al., 2023). These results show that self-supervision does not automatically produce socially useful specialization.

In animal behavior analysis, B-KinD-multi reduces normalized keypoint-regression error relative to B-KinD by 58.8% on CalMS21, 90.2% on Fly vs. Fly, and 81.9% on PAIR-R24M. Its strongest gains occur when agents have similar appearance and interact closely (Khalil et al., 2024). The method still depends on segmentation quality and does not guarantee identity consistency under severe occlusion.

In LLM-based system design, MAS-GPT reports an average score of 65.47 across its main benchmarks, exceeding the second-best baseline by 3.89% (Ye et al., 5 Mar 2025). MHGPO-RR reaches 40.864 accuracy, 37.043 EM, and 49.724 F1 on HotpotQA, while FoF performs best on MuSiQue among the reported MHGPO variants (Chen et al., 3 Jun 2025). SelfOrg improves average accuracy over single-agent and other multi-agent baselines, especially for weak backbones; with Qwen-2.5-1.5B it reports 45.05 average accuracy across the listed tasks (Tastan et al., 1 Oct 2025).

MAS² reports gains across multi-hop search, deep research, code generation, and mathematics, including 97.0 on HumanEval and 71.3 on MATH in the supplied results (Wang et al., 29 Sep 2025). OptiMAS improves initial configurations across WorkBench, GAIA, BrowseComp, and SWE-Bench, although the magnitude varies by backbone and task (Cheng et al., 22 Aug 2026). Recursive MASS reports increased win rates against the base model on synthetic research tasks and improved scores on ScienceAgentBench, MLR-Bench, AstaBench, and DSBench, while reporting slight declines on Terminal-Bench and SWE-bench (Lee et al., 8 Oct 2026).

These results support several conclusions:

1. Interaction-derived supervision can improve representations and policies.
2. Structural adaptation can improve communication and role allocation.
3. Trajectory-level supervision can be denser than terminal team rewards.
4. Runtime intervention can reduce token use without necessarily improving accuracy.
5. Self-supervision does not guarantee collective alignment, factual correctness, or scientific validity.
6. The strongest benefits often occur when individual models are weak, tasks are structurally complex, or agent roles are complementary.

6. Limitations, controversies, and research directions

MASS systems face common technical and methodological limitations.

Dependence on evaluators and representations

Peer similarity is not correctness. A semantically central answer may be wrong, while a correct minority answer may be phrased unusually. LLM judges, verifiers, and benchmark evaluators may be biased, exploitable, or correlated with the systems they evaluate. Embedding geometry may be inappropriate for symbolic mathematics, code, negation, or multi-claim answers. MAS-OPD inherits the teacher’s token preferences, and OptiMAS and recursive MASS depend on evaluator judgments that may favor persuasive but scientifically invalid artifacts.

Credit-assignment ambiguity

Terminal rewards do not identify causal responsibility across agents, turns, tokens, tools, and messages. Backward reward propagation, group-relative advantages, textual diagnoses, and privileged attribution provide approximations rather than complete causal explanations. A downstream failure may be incorrectly attributed to an upstream prompt, or an upstream error may be hidden by a later correction.

Correlated errors and collusion

Many theoretical results assume independent or dispersed errors. Shared backbones, shared prompts, common evidence, and common training distributions can produce correlated mistakes. Consensus, centroid routing, and majority voting may amplify systematic errors. Multi-agent systems also create risks of evaluator collusion, strategic communication, and self-confirming feedback loops.

Computational cost and scalability

Attention over agents, pairwise similarity, repeated LLM calls, trajectory storage, verification, and meta-level optimization all scale with system size and interaction length. Dynamic topology can reduce unnecessary communication but may introduce graph-construction overhead. Centralized optimizers and supervisors become bottlenecks as agent populations and trace lengths grow. Recursive workflow search can be substantially more expensive than deployment.

Safety and structural control

Systems that can generate agents, modify tools, change permissions, or rewrite workflows require sandboxing, authorization boundaries, rollback, and programmatic validation. Runtime rectification and structural evolution may introduce unsafe tool access, data leakage, prompt-injection vulnerabilities, privacy violations, or reward-hacking behavior. A DAG restriction improves termination but excludes cyclic debate and recurrent coordination patterns.

Evaluation weaknesses

Many reported studies use small teams, controlled environments, limited seeds, selected checkpoints, or benchmark-style tasks. Exact numerical variance, confidence intervals, per-agent contribution, large-scale communication cost, and robustness to adversarial inputs are often incomplete. Improvements may reflect increased token allocation, better evaluator alignment, or benchmark-specific adaptation rather than general collective intelligence.

Future MASS research is likely to focus on the following directions:

• Causal credit assignment: combine counterfactual replay, responsibility graphs, Shapley-style analysis, and agent-specific interventions.
• Confidence-weighted peer supervision: calibrate peer feedback using reliability, evidence, uncertainty, and historical performance.
• Disagreement-aware coordination: preserve useful minority hypotheses rather than optimizing only for consensus.
• Learned runtime supervision: replace fixed trigger heuristics with policies that estimate the value of intervention.
• Verifier ensembles: combine symbolic checks, executable tests, simulators, retrieval grounding, and independent evaluators.
• Persistent organizational memory: retain validated workflows, failure schemas, communication patterns, and role assignments while controlling contamination.
• Multi-objective optimization: jointly optimize task success, cost, latency, robustness, safety, and intervention burden.
• Adaptive topology and role discovery: learn when to add, remove, merge, or reassign agents and communication links.
• Continual and recursive improvement: determine when repeated self-training improves capabilities rather than evaluator exploitation or distribution-specific performance.
• Scientific and real-world validation: evaluate whether improvements preserve the intended semantics of tasks rather than merely producing attractive reports or benchmark scores.

MASS is consequently best understood as an architectural and methodological paradigm rather than a single training objective. Its central research problem is to convert multi-agent interaction into reliable supervision while preserving diversity, causal accountability, safety, and generalization. The most complete systems combine capability learning, relational representation, trajectory monitoring, failure attribution, role specialization, structural adaptation, and outcome-based verification. The long-term objective is a closed-loop multi-agent system that can execute, evaluate, diagnose, reorganize, and improve itself without mistaking agreement for truth or local success for collective reliability.