MusicSwarm: Decentralized Creative Intelligence

• MusicSwarm is a family of systems that use decentralized swarm intelligence to convert musical cues into creative outputs like visual paintings and musical compositions.
• It employs techniques such as music-to-behavior mapping, heterogeneous coverage control, and stigmergic coordination for agent-based art generation.
• Empirical evaluations show that scaling the swarm enhances pattern resolution and musical innovation, supporting applications in art, design, and collaborative creativity.

MusicSwarm encompasses a family of computational and robotic systems in which a distributed, decentralized swarm translates musical inputs or music-inspired coordination strategies into novel creative artifacts, ranging from visual paintings generated by coordinated mobile robots to long-form musical compositions orchestrated by agent-based foundation models. Across these instantiations, the "swarm" paradigm encodes collaborative intelligence, distributed memory, and emergent structure, typically leveraging mechanisms such as coverage control, peer-to-peer coordination, and stigmergy. This entry surveys the principal MusicSwarm concepts and realizations, with particular attention to end-to-end music-driven swarm painting (Cheng et al., 31 May 2025) and swarm-based compositional intelligence (Buehler, 15 Sep 2025).

1. System Architectures and Core Paradigms

MusicSwarm systems instantiate distributed creative processes via autonomous agents—either embodied robots (for painting) or virtual agents (for composition)—that are coordinated through direct or indirect peer interaction.

Music-driven Robot Swarm Painting: The system connects music and visual art through a pipeline comprising:

• Audio capture and feature extraction, yielding tempo and chord sequence.
• A music-to-painting mapper, assigning chords to emotions (via lookup tables), then to colors (Plutchik’s wheel), canvas positions (chord wheel mapping), and motional responsiveness (tempo-derived turning parameter).
• A swarm coverage controller, computing time-varying color densities as spatial Gaussians and moving each robot according to a heterogeneous coverage policy.
• Actuation and painting, either as simulated paint trails or physical RGB LED light-painting executed by mobile robots; the emergent spatial patterns express the dynamic structure of the musical stimulus (Cheng et al., 31 May 2025).

Swarm-based Music Composition: In this variant, a population of agents ("bar-wise" agents, one per musical bar) each wraps a frozen foundation model (e.g. GPT-5-mini). No weight updates are performed; instead, coordination arises via:

• Stigmergic peer-to-peer communication, where musical cues are deposited into and sensed from a shared memory ("pheromone field").
• Local prompts and neighborhood context conditioning agent output.
• Iterative consensus dynamics and role specialization, with traits such as risk-taking and theme loyalty updated via local peer feedback and networked averaging (Buehler, 15 Sep 2025).

2. Mathematical and Algorithmic Formulation

2.1 Painting Swarm: Heterogeneous Coverage Control

Let the canvas domain be $\Omega \subset \mathbb{R}^2$. Each color $j \in \{C, M, Y\}$ is represented by a density function

$$\phi_j(q, t) = \frac{K_j(t)}{2\pi \sigma_{x, j} \sigma_{y, j}} \exp \left[ -\frac{1}{2} \left( \left( \frac{q_x - \mu_{x, j}(t)}{\sigma_{x, j}} \right)^2 + \left( \frac{q_y - \mu_{y, j}(t)}{\sigma_{y, j}} \right)^2 \right) \right]$$

where $\mu_{x, j}(t), \mu_{y, j}(t)$ encode the current target position (via chord mapping), $K_j$ is intensity, and $\sigma_x, \sigma_y$ control spatial spread. The total density is $\phi(q, t) = \sum_{j} \phi_j(q, t)$.

A robot $i$ is assigned a set of pigments $P(i)$, and its coverage region $V_i(x)$ is defined by the Voronoi partition. The agent computes heterogeneous masses

$j \in \{C, M, Y\}$0

and centroids

$j \in \{C, M, Y\}$1

The velocity command sums centroid attractions: $j \in \{C, M, Y\}$2. Robots are modeled as unicycles, with turning gain $j \in \{C, M, Y\}$3 modulated according to tempo: high tempo yields smaller $j \in \{C, M, Y\}$4 (sharper turns). The global objective is reduction of the coverage cost

$j \in \{C, M, Y\}$5

Gradient descent is guaranteed: $j \in \{C, M, Y\}$6 (Cheng et al., 31 May 2025).

2.2 Compositional Swarm: Stigmergic Coordination

Each agent $j \in \{C, M, Y\}$7 composes a bar $j \in \{C, M, Y\}$8 using a shared context and pheromone field $j \in \{C, M, Y\}$9. Agents deposit musical feature vectors $$\phi_j(q, t) = \frac{K_j(t)}{2\pi \sigma_{x, j} \sigma_{y, j}} \exp \left[ -\frac{1}{2} \left( \left( \frac{q_x - \mu_{x, j}(t)}{\sigma_{x, j}} \right)^2 + \left( \frac{q_y - \mu_{y, j}(t)}{\sigma_{y, j}} \right)^2 \right) \right]$$0 at each step into the memory map $$\phi_j(q, t) = \frac{K_j(t)}{2\pi \sigma_{x, j} \sigma_{y, j}} \exp \left[ -\frac{1}{2} \left( \left( \frac{q_x - \mu_{x, j}(t)}{\sigma_{x, j}} \right)^2 + \left( \frac{q_y - \mu_{y, j}(t)}{\sigma_{y, j}} \right)^2 \right) \right]$$1. The update follows

$$\phi_j(q, t) = \frac{K_j(t)}{2\pi \sigma_{x, j} \sigma_{y, j}} \exp \left[ -\frac{1}{2} \left( \left( \frac{q_x - \mu_{x, j}(t)}{\sigma_{x, j}} \right)^2 + \left( \frac{q_y - \mu_{y, j}(t)}{\sigma_{y, j}} \right)^2 \right) \right]$$2

where $$\phi_j(q, t) = \frac{K_j(t)}{2\pi \sigma_{x, j} \sigma_{y, j}} \exp \left[ -\frac{1}{2} \left( \left( \frac{q_x - \mu_{x, j}(t)}{\sigma_{x, j}} \right)^2 + \left( \frac{q_y - \mu_{y, j}(t)}{\sigma_{y, j}} \right)^2 \right) \right]$$3 is decay. Each agent senses aggregated features in its radius $$\phi_j(q, t) = \frac{K_j(t)}{2\pi \sigma_{x, j} \sigma_{y, j}} \exp \left[ -\frac{1}{2} \left( \left( \frac{q_x - \mu_{x, j}(t)}{\sigma_{x, j}} \right)^2 + \left( \frac{q_y - \mu_{y, j}(t)}{\sigma_{y, j}} \right)^2 \right) \right]$$4 and updates its local state $$\phi_j(q, t) = \frac{K_j(t)}{2\pi \sigma_{x, j} \sigma_{y, j}} \exp \left[ -\frac{1}{2} \left( \left( \frac{q_x - \mu_{x, j}(t)}{\sigma_{x, j}} \right)^2 + \left( \frac{q_y - \mu_{y, j}(t)}{\sigma_{y, j}} \right)^2 \right) \right]$$5 accordingly.

Agents maintain trait vectors $$\phi_j(q, t) = \frac{K_j(t)}{2\pi \sigma_{x, j} \sigma_{y, j}} \exp \left[ -\frac{1}{2} \left( \left( \frac{q_x - \mu_{x, j}(t)}{\sigma_{x, j}} \right)^2 + \left( \frac{q_y - \mu_{y, j}(t)}{\sigma_{y, j}} \right)^2 \right) \right]$$6 (risk-taking, harmonic sensitivity, etc.), updated per consensus operator

$$\phi_j(q, t) = \frac{K_j(t)}{2\pi \sigma_{x, j} \sigma_{y, j}} \exp \left[ -\frac{1}{2} \left( \left( \frac{q_x - \mu_{x, j}(t)}{\sigma_{x, j}} \right)^2 + \left( \frac{q_y - \mu_{y, j}(t)}{\sigma_{y, j}} \right)^2 \right) \right]$$7

where $$\phi_j(q, t) = \frac{K_j(t)}{2\pi \sigma_{x, j} \sigma_{y, j}} \exp \left[ -\frac{1}{2} \left( \left( \frac{q_x - \mu_{x, j}(t)}{\sigma_{x, j}} \right)^2 + \left( \frac{q_y - \mu_{y, j}(t)}{\sigma_{y, j}} \right)^2 \right) \right]$$8 is a row-stochastic interaction matrix and $$\phi_j(q, t) = \frac{K_j(t)}{2\pi \sigma_{x, j} \sigma_{y, j}} \exp \left[ -\frac{1}{2} \left( \left( \frac{q_x - \mu_{x, j}(t)}{\sigma_{x, j}} \right)^2 + \left( \frac{q_y - \mu_{y, j}(t)}{\sigma_{y, j}} \right)^2 \right) \right]$$9 is the neighborhood average (Buehler, 15 Sep 2025).

3. Music-to-Behavior Mapping

MusicSwarm systems translate musical structure into emergent spatiotemporal or symbolic behavior by a chain of mappings:

• Chord $\mu_{x, j}(t), \mu_{y, j}(t)$0 Emotion/Color: Lookup tables assign detected chords to emotional categories, then to hues (Plutchik’s wheel). This enables affect-consistent visual or musical themes.
• Chord $\mu_{x, j}(t), \mu_{y, j}(t)$1 Position: The “chord wheel” provides a geometric embedding of each chord into a circular (canvas) arrangement; chord transitions shift target locations for swarm density centers.
• Tempo $\mu_{x, j}(t), \mu_{y, j}(t)$2 Motion Parameter: Tempo modulates motion agility (turning gain $\mu_{x, j}(t), \mu_{y, j}(t)$3): high tempo is mapped to more agile, dynamically responsive agent trajectories.
• Color Mixing: In painting context, color proportions $\mu_{x, j}(t), \mu_{y, j}(t)$4 enable continuous blending across CMY or RGB space.

For compositional swarms, peer-to-peer stigmergic signals represent musical structure and novelty, propagating motifs and tension arcs across the global form (Buehler, 15 Sep 2025).

4. Empirical Evaluation and Results

4.1 Robot Painting

Simulation and hardware studies spanned canonical Western music (Bach, Mozart, Beethoven, Chopin, Satie). Control studies showed:

• Larger swarms ($\mu_{x, j}(t), \mu_{y, j}(t)$5) produce denser and more resolved patterns.
• Smaller $\mu_{x, j}(t), \mu_{y, j}(t)$6 (increased agility) results in sharper, more intricate trails.
• Trail width $\mu_{x, j}(t), \mu_{y, j}(t)$7 enables a range of expressive “brushstroke” styles.
• Mixed-pigment swarms achieve fine-grained chromatic blending. In hardware, LED-equipped DJI Robomaster robots simulated paint via long-exposure photography; matching pattern/color trajectories emerged as in simulation. The performance metric is implicit: reduction of the global coverage cost and visual complexity (Cheng et al., 31 May 2025).

4.2 Compositional Swarm

Evaluations considered:

• Symbolic metrics: expectation violations, surprise density, unpredictability, risk-taking metrics based on note sequences.
• Audio metrics: harmonic tension, timbral diversity using audio feature extraction (librosa).
• Graph-theoretic metrics: clustering coefficient $\mu_{x, j}(t), \mu_{y, j}(t)$8, average shortest path $\mu_{x, j}(t), \mu_{y, j}(t)$9, small-worldness $K_j$0, community structure, long-range edge fraction.

Swarm-based systems exhibited small-world self-similarity networks, efficient motif bridging, and greater structural diversity compared to centralized or single-shot baselines:

• Higher long-range edge fraction and community diversity.
• The swarm dynamics converged to stable, specialized roles (e.g., “bridge” bars connecting musical communities).
• Quantitatively, swarm composition exceeded baselines in risk-taking, innovation, and structural flexibility (Buehler, 15 Sep 2025).

5. Real-World Implementation Considerations

5.1 Hardware and Mechatronic Aspects

• Six DJI Robomaster EP differential-drive robots with Wi-Fi and IMU-based localization.
• LED-based light-painting in a dark room replaced physical pigment deployment.
• Synchronization challenges in music feature extraction addressed via event buffering (~0.5 s look-ahead), and color blending in light-painting managed by LED brightness tuning.
• Odometry errors mitigated by periodic reset and Kalman filtering (Cheng et al., 31 May 2025).

5.2 Peer-to-Peer Data and Memory Management

In compositional swarms, agent memory is structured hierarchically:

• Short-term memory buffers local recent bars and feedback.
• Episodic memory enables motif recall and reuse across iterations.
• Shared “pheromone” memory mediates coordination and motif propagation without explicit parameter sharing or gradient updates (Buehler, 15 Sep 2025).

6. Applications, Extensions, and Future Directions

• The MusicSwarm painting framework is directly extensible to larger swarms, more complex (possibly multi-modal) density functions, and additional musical features (melodic contour, rhythmic complexity, amplitude).
• Permanent art can be created by replacing light-painting with physical paint deposition.
• User studies and expressive evaluation remain open, as do real-time performance deployments (Cheng et al., 31 May 2025).
• The compositional MusicSwarm paradigm generalizes beyond music, as demonstrated in collaborative writing (agents deposit and sense “semantic pheromones”), design (distributed engineering roles), and hypothesis-driven scientific discovery (evidence pheromones and consensus) (Buehler, 15 Sep 2025).

A plausible implication is that by shifting from traditional model-centric parameter adaptation to system-level organization—trait dynamics, emergent consensus, and stigmergic signaling—MusicSwarm architectures achieve scalable, creative, and data-efficient intelligence, opening new approaches to both machine collaboration and generative art.