Multi-User Gymnasium (MUG) Framework

• Multi-User Gymnasium (MUG) is a framework that deploys interactive, web-based multi-agent Grid and Gym experiments integrating both human and AI participants.
• It supports API-agnostic wrapping of Gymnasium and PettingZoo environments, low-latency interaction via rollback netcode, and a full experiment management pipeline.
• Its modular design and client–server architecture facilitate scalability from single-player to massive multiplayer studies, advancing human-AI interaction research.

Searching arXiv for the cited work and closely related papers on MUG, CoGrid, and Gym-wrapping antecedents. Multi-User Gymnasium (MUG) is a framework for deploying multi-agent Grid and Gym environments as interactive, browser-based experiments. Introduced together with CoGrid, it is designed to reduce barriers to running multi-agent experiments in which humans and AI act together, and it supports interactions with arbitrary numbers of humans and AI, utilizing either server-authoritative or peer-to-peer networking with rollback netcode to account for latency (McDonald et al., 16 Apr 2026). MUG is tightly coupled to CoGrid, but it is API-agnostic and can in principle wrap any Gymnasium- or PettingZoo-style environment. Its scope is broader than environment wrapping alone: it includes built-in support for landing pages, waiting rooms and matchmaking, condition assignment, surveys, bonus calculations, participation codes, and data logging.

1. Design goals and problem setting

The principal design goals of MUG are Accessibility, Scalability, Low-latency interaction, and a Full experiment pipeline. In the formulation given for the system, accessibility means that researchers should be able to turn any standard-API Python environment into a web-playable experiment without rewriting the environment logic in JavaScript or a game engine. Scalability is defined as support for arbitrary numbers of participants (human or AI), from single-player through massive multiplayer, with minimal per-study overhead. Low-latency interaction is stated as a requirement that real-time environments at 30–60 Hz remain playable even under non-negligible network delays. The full experiment pipeline goal adds built-in support for landing pages, waiting rooms and matchmaking, condition assignment, surveys, bonus calculations, participation codes, and data logging (McDonald et al., 16 Apr 2026).

These design goals place MUG at the intersection of RL environment tooling, browser-based experiment deployment, and human-AI interaction research. The emphasis on standard Python APIs, rather than bespoke game-engine implementations, suggests a deliberate attempt to make experimental deployment contiguous with simulation and training workflows. A plausible implication is that MUG is intended to shorten the path from a research environment used for offline multi-agent training to a live study involving human participants.

2. Core architecture and execution model

MUG realizes its design goals through a plugin-style architecture with three main components: CoGrid (simulation library), MUG Runtime (interactive_gym), and a Client Front-End. CoGrid is a Python library for multi-agent, partially observable Markov decision processes (POMDPs) on a 2D grid. In that formulation, environments define a state space $\mathcal{X}$, action space $\mathcal{A}$, observation space $\mathcal{O}$, next-state function

$$s_{t+1} = \mathcal{T}(s_t, a^1_t, a^2_t, \dots, a^N_t),$$

reward function $\mathcal{R}(s_t, \{a^i_t\}_{i=1}^N)$, and per-agent observation map $\Omega: \mathcal{X}\to \mathcal{O}^N$ (McDonald et al., 16 Apr 2026).

The MUG Runtime is a Python server that coordinates experiment flow and networking. Researchers assemble an experiment as a sequence of Scene objects such as StartScene, GymScene, SurveyScene, and EndScene, managed by a top-level Stager. The client front-end consists of a browser UI, described as Phaser-based, plus, optionally, Pyodide (Python³ → WebAssembly) on the client, which can execute the environment logic locally and render the state at 30–60 Hz. At runtime, the researcher writes one small Python configuration file specifying which policies map to which agents, how to build the Gym/PettingZoo environment, keybinding mappings, rendering routines or assets, matchmaking constraints, waiting-room text, experiment-flow logic, survey forms, and bonus-code generation. When app.run(config) is invoked, MUG spawns the HTTP/WebSocket server, serves static assets, handles session management via cookies or URL tokens, and steps through Scenes in lock-step as participants connect and proceed (McDonald et al., 16 Apr 2026).

This decomposition is technically significant because it separates simulation semantics, experiment orchestration, and client rendering while keeping them within a single Python-centered workflow. The architecture also makes clear that MUG is not restricted to CoGrid’s 2D grid environments, even though CoGrid provides the most fully integrated substrate.

3. Networking models, determinism, and rollback

MUG supports two interaction models: server-authoritative and peer-to-peer with rollback (“GGPO style”). In server-authoritative mode, all environment stepping, AI policy calls, and state updates happen on the server. Clients send their discrete action $a^i_t$ on each tick $t$; the server applies

$s_{t+1} = \mathcal{T}(s_t,\;a^1_t,\dots,a^N_t),$

then broadcasts the new state or observations $\{o^i_{t+1} = \Omega_i(s_{t+1})\}$ back to each client. This mode is described as trivial to implement, but it suffers from $\mathcal{A}$0 latency per tick. At 30 Hz, even 50 ms round-trip leads to noticeable lag (McDonald et al., 16 Apr 2026).

To achieve local responsiveness, MUG can instead push the environment logic down to each client using Pyodide. In this peer-to-peer model, all clients start from the same random seed and maintain an identical local copy of the environment. On each tick, the client immediately applies its own action, predicts remote input if it has not yet arrived, steps speculatively, renders the predicted next state, and later checks the received remote action against the prediction. The common default predictor is “last-known input” (zero-order hold):

$\mathcal{A}$1

If a mismatch is detected, the client performs rollback & replay from the last confirmed tick, recomputing the intervening trajectory with the correct actions (McDonald et al., 16 Apr 2026).

This mechanism imposes strict implementation requirements. Environments must be deterministic given a seed; they must support get_state() and set_state() methods to capture and restore the entire RNG and MDP state; and clients keep a small ring buffer of past states and actions of size $\mathcal{A}$2, where $\mathcal{A}$3 is the maximum predicted network delay, for example 5–10 ticks. In peer-to-peer mode, the effective local input-to-visual latency is exactly one frame, approximately 16 ms at 60 Hz, regardless of network RTT up to the replay buffer size, for example 200 ms. Empirically, participants reported no perceived lag up to 150 ms one-way latency. Because MUG uses deterministic replay, all clients remain exactly in sync modulo irreversible nondeterminism (no floating-point noise), and end-to-end state divergence is zero if seeds and replay buffers are correctly managed (McDonald et al., 16 Apr 2026).

4. API structure and experiment lifecycle

At the top level, an experiment configuration is built via the interactive_gym.Config object. Its methods are chainable and divided into functional groups. The .policies(...) group includes policy_mapping, which maps an agent to "Human", an AI policy object, or an ONNX file, and frame_skip, the number of frames the AI policy steps per real UI frame. The .environment(...) group includes env_creator, which may be lambda: gym.make("MyCoGridEnv-v0") or a CoGridEnv subclass, and backend, which may be "numpy" or "jax". The .rendering(...) group includes env_to_render_fn, canvas_size, fps, and a custom assets path. The .gameplay(...) group defines GymScene parameters such as key-bindings, tick rate, and maximum episode length. The .hosting(...) group specifies matchmaking rules, including min/max group size and RTT thresholds, together with waiting-room text templates. The .user_experience(...) group provides survey definitions, start/end scenes, randomization or counterbalancing, a bonus calculation formula bonus = α·score + β·accuracy + γ, and a completion-code template for Prolific/MTurk (McDonald et al., 16 Apr 2026).

The scene system is the main abstraction for experiment flow. MUG provides StartScene for instructions and a “Begin” button, GymScene for interactive play, SurveyScene for a qualtrics-style form, and EndScene for completion codes and debrief. Researchers can define custom scenes by subclassing Scene, and a Stager object holds the ordered list of scenes and can apply randomization or counterbalancing at the participant-ID level. In addition, every GymScene exposes hooks via GameCallback, including on_episode_start, on_step, and on_episode_end, which can be used to log trajectories locally or push them to external databases (McDonald et al., 16 Apr 2026).

This arrangement makes the experiment lifecycle explicit: recruitment and entry, synchronization in waiting rooms, interactive play, post-task instrumentation, and compensation or debrief. A plausible implication is that MUG is designed not merely as a runtime for games or demonstrations, but as experiment infrastructure with first-class support for behavioral study protocols.

5. Case studies and quantitative characteristics

The MUG paper presents two principal case studies. The first is CoGrid Overcooked (Cramped Room), an environment reconstructing Carroll & Gonzalez’s Overcooked in which two chefs prepare onion soup in a small kitchen on an ASCII map with a 5×4 grid. Objects are registered via decorators, rewards are built declaratively, and PPO via RLlib with self-play on the NumPy backend produced a policy delivering 7.5 dishes/episode on average. The interactive study had two conditions: Human–Human, in which 20 pairs completed 20 episodes each, and Human–AI, in which 29 participants each played 20 episodes against the fixed policy. The reported results were an AI–AI baseline of 7.5 dishes, Human–AI pairs at approximately 5 dishes stable over episodes, and Human–Human pairs improving from approximately 3 dishes to approximately 6 over 20 episodes. The contribution split between Human and AI drifted over time as humans took on more delivery tasks (McDonald et al., 16 Apr 2026).

The second case study is Slime Volleyball, a non-grid environment ported to the Gymnasium API with a 12-dim continuous observation and discrete actions left/right/jump/no-op. PPO with a self-play curriculum, updating a copy every time average reward exceeded 0.5, reached a max episode length of approximately 3000 steps. In the interactive study, participants completed 30 episodes each in Human–Human and Human–AI conditions. Episode length, interpreted as rally duration, was the metric of skill. Human–AI pairs sustained longer rallies, approximately 200 steps, than Human–Human pairs, approximately 80 steps, while AI–AI matches often hit the 3000 step cap (McDonald et al., 16 Apr 2026).

The system is also characterized by explicit performance benchmarks. For CoGrid throughput on an NVIDIA RTX 3090, the paper reports Overcooked-AI at approximately 3.4 k sps, JaxMARL from 4.3 k sps for 1 instance to 2.9 M sps for 1024 instances, CoGrid (JAX) from 4.5 k sps to 5.6 M sps, and CoGrid (NumPy) at approximately 450 sps. The JAX result is reported as a $\mathcal{A}$4 speedup over JaxMARL at 1024 instances, while the NumPy backend is described as sufficient for client-side but unsuitable for large-scale RL training. In both case studies, MUG required zero change to the environment code—only a short Config file and optional rendering tweaks (McDonald et al., 16 Apr 2026).

6. Relation to Gym-style wrappers and other multi-user systems

MUG belongs to a broader line of work that adapts simulation environments to standard RL interfaces, but its emphasis is distinct. An important antecedent is ABIDES-Gym, which proposed using the OpenAI Gym framework on discrete event time based Discrete Event Multi-Agent Simulation (DEMAS). ABIDES-Gym introduced a general technique to wrap a DEMAS simulator into the Gym framework, including an “interruptible runner” version of the kernel and a special “Gym agent” that periodically interrupts simulation to hand control back to the RL loop. Each ABIDES-Gym sub-environment then inherits from Gym-Core, defines its own MDP, and can register itself with Gym’s registry (Amrouni et al., 2021).

The comparison is instructive. ABIDES-Gym focuses on exposing a simulator through reset() and step() by means of interruptible execution and a Gym agent, and it applies that design to benchmark financial markets environments. MUG, by contrast, layers networking, browser rendering, session management, scene sequencing, surveys, bonus calculations, and waiting-room logic on top of Gymnasium- or PettingZoo-style environments. This suggests a shift from simulator wrapping for RL training toward full-stack deployment of human-AI experiments.

A different point of comparison is Gymcentral, a virtual fitness environment for older adults. Gymcentral is a client–server platform composed of two client applications and a central back-end, with a Trainee App, a Coach App, an Application Server, a Database, a Media Server, and a Notification Service. It provides multi-user interaction through avatar presence, text chat, invitations, and a virtual gym UI including Reception, Locker Room, Classroom, Agenda, Bulletin Board, Private Messaging, and Progress Report (Baez et al., 2016). A plausible implication is that the phrase “multi-user gym” can refer either to a literal virtual exercise environment or, in MUG, to a Gymnasium-derived experimentation framework. The two share client–server and multi-user coordination concerns, but they address different research problems and use different abstractions.

7. Significance, constraints, and research use

MUG is explicitly framed as tooling for studies of social decision making, psychology, cognition, decision making, and their relationship to human-AI interaction. Its technical significance lies in combining a concise Python API with a browser-playable front-end, deterministic replay, and experiment-management primitives. Because policies can map agents to humans or AI, and because environments can be standard Gymnasium or PettingZoo interfaces, the framework supports a continuum from pure simulation to mixed human-AI interaction (McDonald et al., 16 Apr 2026).

Its constraints are equally clear. The peer-to-peer rollback model depends on determinism given a seed, on complete get_state() / set_state() support, and on bounded replay buffers. Server-authoritative mode is simpler but incurs latency costs proportional to round-trip time. The strongest performance claims are associated with CoGrid’s JAX backend for large-scale simulation, whereas the NumPy backend is characterized as sufficient for client-side but unsuitable for large-scale RL training. These constraints indicate that MUG is best understood as a systems framework whose success depends on environment design discipline as much as on network engineering.

Within the Gym/Gymnasium ecosystem, MUG can therefore be situated as a framework for translating multi-agent simulation environments directly into interactive web-based experiments. In that role, it extends the established practice of environment wrapping into a broader experimental stack: one in which environment specification, human participation, AI control, rendering, networking, and post hoc data collection are treated as parts of a single research runtime (McDonald et al., 16 Apr 2026).