Parallel Mechanisms in Integrated Systems

• Parallel mechanisms are computational processes that operate concurrently to achieve synchronized outcomes in complex systems.
• They coordinate multiple systems—software and hardware alike—through protocols like OSC and MQTT for real-time integration.
• These mechanisms drive innovations in AI, neural signal mapping, and multisensory installations, enhancing interactive experiences.

Below is a structured, in-depth account of “Simulacra Naturae” organized into eight sections. Wherever appropriate, we include the key equations in LaTeX, describe system diagrams, and tabulate major mappings.

1. High-Level Overview

• Concept – Simulacra Naturae is a multisensory media installation that re-materializes 131-channel brain-organoid spike data as an inhabitable ecosystem. Rather than “visualizing” neural time-series in a screen plot, it treats organoid rhythms as co-creative forces that drive:
  • an artificial-life environmental projection (termites, slime molds, boids),
  • a 16.2-channel spatial soundscape,
  • solenoid-struck ceramic vessels,
  • fiber-optic lighting from LED matrices,
  • a forest of live tropical plants on mulch and moss.
• Goals –
  1. To explore nonhuman agency and collective care through hybrid biological–computational processes.
  2. To demonstrate real-time coupling of high-density neural data with GPU-accelerated agent-based simulations and generative AI imagery.
  3. To integrate living plants and crafted clay artifacts as active “actants” in an emergent, multi-modal habitat.

• Key Contributions –

  • A real-time pipeline mapping 131 organoid channels onto ~50 million agents across three behavioral models (stigmergic termites, Physarum foragers, Reynolds boids) plus AI diffusion visuals.
  • A cyber-physical sound system with 27 solenoids striking morphogenic ceramic vessels in synchrony with “backbone” neurons.
  • A synchronized software/hardware architecture (TouchDesigner, Unity, Processing, Max/MSP, OSC, MQTT, NDI) achieving frame-accurate timing across visuals, audio, and actuation.
  • A qualitative framework for “distributed creative agency,” centering ecological ethics, empathy, and relational cognition.

1. Neural Signal Processing Pipeline A. Data Acquisition

• Organoids (human iPSC-derived) interfaced with CMOS MEAs sampling at 20 kHz.
• Kilosort2 spike-sorting → 131 active neuron channels; 27 “backbone” neurons selected for solenoid actuation.

B. Preprocessing and Time Scaling

• Original recording length = 3 minutes (180 k rows at 1 ms resolution).
• Playback slowed to 90 minutes (factor 30×) for human perceptibility.
• TouchDesigner master patch reads each row index $t$ and broadcasts via OSC to all subsystems.

C. Feature Extraction

• Instantaneous spike indicator for neuron $i$ at time $t$: $$s_i(t) = \begin{cases} 1, & \text{if neuron }i\text{ fires at }t,\ 0, & \text{otherwise.} \end{cases}$$
• Population firing rate: $$R(t) = \frac{1}{N}\sum_{i=1}^N s_i(t), \qquad N=131.$$
• Burst boundaries extracted as timepoints when $R(t)$ crosses a threshold $\theta_{\rm burst}$.

D. Mapping to Generative Parameters Table 1 summarizes the principal mapping functions used in audio, visuals, and swarm speed:

Table 1 – Neural→Parameter Mappings | Parameter | Definition | Mapping Function | |---------------------|--------------------------------------------|--------------------------------------------------| | Trail deposition | agent-level pheromone flag | $T_i(t) = s_i(t)$ | | Agent speed | movement velocity in slime/boid models | $v_i(t) = v_0 + \alpha\,s_i(t)$ | | Solenoid strike | actuation amplitude | $A_j(t) = A_0 + \beta\,s_j(t)$, $i$0 | | Tone event rate | sustained guitar density | $i$1 | | Grain event rate | granulation density | $i$2 | | Grain duration | sample‐grain length | $i$3 | | Flocking weights | cohesion/alignment/separation gains | $i$4 |

2. Formal Specification of Agent-Based Simulation All digital agents run on GPU compute shaders in Unity with the following abstract state per agent ($i$5):

• Position $i$6
• Velocity $i$7
• Orientation angle $i$8
• Internal pheromone deposit flag $i$9

A. Termite-inspired Stigmergy Model

1. Environment scalar field $t$0 (pheromone intensity).
2. Trail dynamics: $t$1
3. Agent update per timestep: pseudocode TermiteStep(i): sense forward, left, right samples of $t$2 at angles $t$3, distance $t$4 if forward ≥ max(left,right) then $t$5 else $t$6 (choosing direction of stronger signal) move: $t$7 deposit: $t$8

B. Physarum-inspired Foraging Model Based on Jones (2010), each agent:

• Samples local chemo-attractant field $t$9 at offset sensors.
• Rotates toward highest $$s_i(t) = \begin{cases} 1, & \text{if neuron }i\text{ fires at }t,\ 0, & \text{otherwise.} \end{cases}$$0, deposits a small quantity $$s_i(t) = \begin{cases} 1, & \text{if neuron }i\text{ fires at }t,\ 0, & \text{otherwise.} \end{cases}$$1 upon movement.
• Field update: $$s_i(t) = \begin{cases} 1, & \text{if neuron }i\text{ fires at }t,\ 0, & \text{otherwise.} \end{cases}$$2
• Key organoid modulation: sensor angle $$s_i(t) = \begin{cases} 1, & \text{if neuron }i\text{ fires at }t,\ 0, & \text{otherwise.} \end{cases}$$3, sensor distance $$s_i(t) = \begin{cases} 1, & \text{if neuron }i\text{ fires at }t,\ 0, & \text{otherwise.} \end{cases}$$4, speed $$s_i(t) = \begin{cases} 1, & \text{if neuron }i\text{ fires at }t,\ 0, & \text{otherwise.} \end{cases}$$5 all vary linearly with $$s_i(t) = \begin{cases} 1, & \text{if neuron }i\text{ fires at }t,\ 0, & \text{otherwise.} \end{cases}$$6.

C. Reynolds-Boid Flocking Model Agent‐to‐agent interactions for each pair $$s_i(t) = \begin{cases} 1, & \text{if neuron }i\text{ fires at }t,\ 0, & \text{otherwise.} \end{cases}$$7 within radius $$s_i(t) = \begin{cases} 1, & \text{if neuron }i\text{ fires at }t,\ 0, & \text{otherwise.} \end{cases}$$8: * Separation: $$s_i(t) = \begin{cases} 1, & \text{if neuron }i\text{ fires at }t,\ 0, & \text{otherwise.} \end{cases}$$9 * Alignment: $$R(t) = \frac{1}{N}\sum_{i=1}^N s_i(t), \qquad N=131.$$0 * Cohesion: $$R(t) = \frac{1}{N}\sum_{i=1}^N s_i(t), \qquad N=131.$$1 Overall acceleration:

$$R(t) = \frac{1}{N}\sum_{i=1}^N s_i(t), \qquad N=131.$$2

with each weight $$R(t) = \frac{1}{N}\sum_{i=1}^N s_i(t), \qquad N=131.$$3.

1. Architecture of the Generative Ecosystem A. Software Stack & Data Routing (see Figure 1 interaction diagram)

• TouchDesigner (Master Clock/OSC publisher): reads organoid rows, broadcasts index $$R(t) = \frac{1}{N}\sum_{i=1}^N s_i(t), \qquad N=131.$$4.
• Unity (ALife projections): receives OSC $$R(t) = \frac{1}{N}\sum_{i=1}^N s_i(t), \qquad N=131.$$5, $$R(t) = \frac{1}{N}\sum_{i=1}^N s_i(t), \qquad N=131.$$6, $$R(t) = \frac{1}{N}\sum_{i=1}^N s_i(t), \qquad N=131.$$7. Renders termites, slime-mold, boids in real time.
• Processing (secondary visuals): receives OSC $$R(t) = \frac{1}{N}\sum_{i=1}^N s_i(t), \qquad N=131.$$8 for generative AI overlays.
• Second TouchDesigner (AI diffusion): ingests floorplan vectors + organoid firing positions, runs Stable Diffusion 2.1 + LoRA to generate evolving textures.
• Max/MSP (16.2 audio + solenoid control): receives OSC $$R(t) = \frac{1}{N}\sum_{i=1}^N s_i(t), \qquad N=131.$$9, extracts pre-encoded .wav data for population rate and burst markers, drives granular/sustain synthesis and 27 solenoids.
• Network protocols: OSC for low-latency sync, MQTT for optional remote distribution, NDI for texture streaming between machines.

B. Hardware Components

• Two desktop workstations, each with NVIDIA RTX 4090, 10 Gbps switch.
• Projection: one 4K laser floor projector; three 4K laser wall projectors (total resolution ~10,184×2,160).
• Cyber-physical:
  • 27 solenoids embedded in clay vessels + glassware, wired to custom driver board.
  • Two 16×16 RGB LED matrices driving fiber-optic bundles in hydroponic glassware.
• Environment: hydroponic planters, living tropical plants (Monstera deliciosa, Alocasia, Strelitzia alba, Dracaena, etc.), mulch + artificial moss substrate.

2. Role of Material Ecologies

• Spatial Layout – plant placement and visitor corridors derive from topological clusters of backbone neurons (Figure 2 spatial arrangement).
• Living Plants – large-leaf and bamboo species provide olfactory, tactile, and humidity feedback; they inhabit zones mapped from neural firing neighborhoods.
• Clay Artifacts – vessels shaped by a rule-based differential-growth algorithm in Grasshopper/Rhino (Python), printed on Potterbot clay printer; post-fired, each has unique resonance profile.
• Integration – solenoid strikes excite vessel resonances that vary with form, thickness, humidity; fiber optics animate plant edges, blending living and cybernetic.

3. Emergent Dynamics & Co-Creative Interplay

• Case Study A (Min 00:15): a burst event cluster ($R(t)$0) rapidly increases slime-mold sensor angle, producing radial “sunburst” trails on floor projection; simultaneously, audio shifts via cue of burst boundary → harmonic transition in C-minor↔Phrygian textures.
• Case Study B (Min 00:45): sustained high‐frequency firing ($R(t)$1 Hz) drives boid alignment weight up, yielding flocking “wave” visuals wrapping around plant clusters; solenoid strike density ($R(t)$2) peaks and listeners report feeling “heartbeat” synchrony with visual motion.
• Quantitative Observation – cross-correlation between $R(t)$3 and event density in Channel 4 of the Max/MSP patch: $R(t)$4, $R(t)$5.
• Emergent Phenomena – visitors describe “morphogenic breathing” as the space expands/contracts in audio amplitude and light intensity, illustrating how co-evolutionary feedback between neurons, agents, and matter produces perception of agency in the nonhuman.

4. System Diagrams, Tables & Key Equations

• Figure 2 (Spatial Arrangement) – planar regions correspond to firing clusters.
• Figure 1 (Interaction Diagram) – TouchDesigner ↔ Unity/Processing ↔ Max/MSP ↔ hardware.
• Table 1 (Mapping Functions) above.
• Key field equations for pheromone/chemoattractant and boid forces appear in Section 3.

5. Ethical, Ecological & Experiential Dimensions

• Decentralizing Agency – by granting organoid rhythms equal participation in generative processes, the installation challenges anthropocentric authorship.
• Collective Care – relational cognition frameworks (Latour ANT, Haraway kin-making, Sheldrake’s entangled life) underpin a design that centers attentiveness, mutual shaping, and multispecies empathy.
• Ecology of Materials – living plants and clay vessels are not mere props but active responders with their own “agencies,” foregrounding material vitality (Ingold, Bennett).
• Experiential Impact – visitors inhabit a “soft machine” in which sight, sound, touch, and scent are modulated by neural patterns, provoking reflection on the boundaries between mind, matter, and environment.
• Future Ethical Pathways – authors propose live organoid integration and optional participatory controls for audiences, preserving endogenous neural dynamics while enhancing legibility and care.

In sum, Simulacra Naturae weaves together high-density neural data, artificial-life simulations, generative AI imagery, spatialized audio, responsive ceramics, and living botanicals into a singular eco-computational environment—one that materializes cognition as a shared, co-creative, and ethically fraught process.