Artistic Mediation over Black-Box Transparency

• Artistic mediation over black-box transparency is a framework that employs artistic strategies to materialize, modify, and engage with opaque AI model processes.
• It integrates arts, explainable AI, and human-computer interaction methodologies to reveal internal operations through sensory, interactive experiences.
• Practices such as model bending, interactive workflows, and tactile installations enable users to gain reflective, embodied insights into AI behaviors.

Artistic mediation over black-box transparency refers to a set of methodologies, practices, and theoretical frameworks in which artists and researchers employ artistic strategies to render the internal mechanisms, operational logics, and outcomes of opaque AI models both tangible and modifiable. This approach displaces the classical paradigm of passive transparency—where users are recipients of post-hoc explanations—by emphasizing active engagement, material intervention, and the cultivation of tacit understanding through direct manipulation, embodied experience, and iterative practice. The field synthesizes principles from explainable AI (XAI), human-computer interaction, and art practice, resulting in hybrid workflows and metrics for creative agency, legibility, and accountability in human-centered AI systems (Abuzuraiq et al., 10 Aug 2025, Hemment et al., 2023, Hemment et al., 2019).

1. From Black-Box Transparency to Experiential AI

Traditional black-box transparency is characterized by approaches such as feature-importance scores, local surrogate models, and saliency maps, which attempt to reverse-engineer the decision-making of complex models into human-interpretable summaries. These methods typically situate the user as a passive observer: the AI’s computations are presented as static, often low-dimensional visualizations or textual rationales, and users have limited opportunity to probe, manipulate, or co-create with the system (Hemment et al., 2019).

In contrast, artistic mediation—as articulated in the Experiential AI paradigm—materializes, embodies, and situates the operations of AI models as lived, tangible experiences. The goal is not solely to extract a “rational explanation” post-hoc but to open the full causal chain of AI systems (from data acquisition through transformation, training, and inference) as a sensorially accessible process. As Hemment et al. note, this can entail turning abstract code-level events into audio-visual, tactile, or narrative experiences, highlighting not only mechanics but also embedded values, biases, and sociotechnical stakes (Hemment et al., 2023).

2. Theoretical and Conceptual Foundations

Central to recent advances is Donald Schön’s concept of “reflection-in-action”—a process in which practitioners engage in a continuous conversation with their materials, adjusting interventions dynamically based on emergent feedback. Abuzuraiq & Pasquier transpose this construct to generative AI: rather than inspecting static, model-agnostic explanations, artists engage the internal structure of the model in real time, directly altering its weights, activations, or conditioning streams and immediately observing the visual or sonic consequences (Abuzuraiq et al., 10 Aug 2025).

Experiential AI formalizes artistic mediation as a mapping $f: C \rightarrow H$ where $C$ denotes code artifacts and intermediate data, and $H$ is a set of human-interpretable experiences (visuals, sounds, tactile events). This mapping often involves two stages: automated extraction of media primitives (visual, haptic, or auditory) and their artistic staging into situated experiences (Hemment et al., 2019). The Perceptual Clarity metric $PC(f)$ quantitatively assesses how well such mappings help users construct accurate mental models of AI operation.

Artistic mediation is further distinguished by its commitment to bridging not only the technical but also the ideological and material dimensions of AI, surfacing value judgments, biases, and cultural meanings inherent in model design and deployment (Hemment et al., 2023).

3. Methodologies and Interaction Design

A range of technical and artistic strategies operationalize artistic mediation:

• Model Bending Interfaces: In Abuzuraiq & Pasquier’s system for ComfyUI, custom nodes expose internal diffusion model components (such as layers of a UNet), allowing artists to select, manipulate, and visualize feature maps and flows. Core nodes include model bending (layer-wise transformations such as scaling, rotation, or noise injection), model inspector (hierarchical UI for selecting layers), feature map visualization, and conditioning nodes for text embeddings (Abuzuraiq et al., 10 Aug 2025).
• Interactive Workflows: Artists construct graph-based workflows (e.g., Load Model → Encode → Denoise → Decode), inserting bending/interrogation nodes at critical junctures. Because the graph executes per denoising step, any intervention dynamically propagates through the generation process.
• Dimension-of-Interest Control: In experiential AI installations, techniques such as SLIDER (Shaping Latent-spaces for Interactive Dimensional Exploration and Rendering) allow users to define or interpolate between interpretable latent directions, both for image and text models. The formalism is:

$$z(\alpha) = z_0 + \alpha \frac{d}{\|d\|}, \quad \hat y = G(z(\alpha))$$

where $z_0$ is an anchor latent vector, $d$ is the direction between mean embeddings of selected sets, and $G$ is the generator (Hemment et al., 2023).

• Formal Artistic Mapping:

| Stage | Example Implementation | Output Modality | |---------------|-------------------------------------------------|----------------------| | g: Code → V | Sample model activations, encode as colors | Layer-wise images | | h: V → H | Artistic staging in installation or interface | Sounds, visuals, VR |

Direct manipulation, role-play (as data points), and data-flow avatars are also used to ensure multisensory engagement (Hemment et al., 2019).

4. Mathematical and Algorithmic Interventions

Artistic mediation relies on explicit algorithmic touchpoints within the model architecture. Representative interventions include:

• Layer-wise Scaling: For a selected convolutional block $L$ with activation $a_L$, one can apply

$C$0

so that $C$1 accentuates, and $C$2 attenuates, feature amplification (Abuzuraiq et al., 10 Aug 2025).

• Noise Injection: Introducing Gaussian noise at a layer or timestep:

$C$3

where $C$4 modulates the degree of stochasticity, affecting global or local aspects depending on network depth.

• Text Conditioning Bending: For text embedding $C$5, artists apply small user-defined offsets $C$6:

$C$7

resulting in smooth transition of stylistic or semantic content.

• Visual Mapping: These interventions map to perceptible changes—early-layer manipulation impacts color and layout, mid-layer interventions alter object placement and semantic alignment, and late-layer modifications tune texture and fine detail (Abuzuraiq et al., 10 Aug 2025).

Through these controlled manipulations, participants build a practice-based, tacit understanding of the generative process.

5. Case Studies and Empirical Findings

Empirical approaches have ranged from gallery-based installations to hands-on workshops:

• In Abuzuraiq & Pasquier’s ComfyUI plugin, artists deployed rotational and noise-based bending during mid-UNet and late denoising steps, respectively, discovering the correspondence between internal layer transformations and coherent artistic effects (e.g., subtle figural twists, painterly texture overlays) (Abuzuraiq et al., 10 Aug 2025).
• Hemment et al. designed SLIDER-based systems for artists to probe “dimensions of interest” in latent spaces of GANs and text models. By interactively selecting data subsets, defining axes, and interpolating, artists uncovered latent biases, emergent visual-semantic bridges, and achieved “granular control” previously unavailable in prompt-driven interfaces. Formal and physical interfaces—including touchscreens, VR, and tangible sliders—supported real-time, multi-modal exploration (Hemment et al., 2023).
• Experiential AI installations have projected activations, encoded model confidence as sonification, and staged physical or navigable analogs of decision boundaries. Metrics such as Perceptual Clarity and Experiential Transparency Score have been proposed to assess the alignment between participants’ mental models and actual model behavior (Hemment et al., 2019).

6. Impact and Implications for Explainable AI

Artistic mediation transforms the role of the artist from recipient of static explanations to active co-author, fostering agency and sustained engagement. Rather than treating transparency as an endpoint—achieved when inner workings are “revealed”—the focus shifts to transparency as a dynamic, performative process: understanding is constructed through doing, reflection, and co-creation (Abuzuraiq et al., 10 Aug 2025).

These strategies enable:

• Legibility: Artists report a qualitative sense of being “in control,” with improved intuition about internal model relationships and emergent behaviors (Hemment et al., 2023).
• Accountability: By foregrounding the cultural, social, and ideological dimensions embedded within the model and its deployment, artistic mediation surfaces issues such as bias, data provenance, and responsible use, fostering public debate (Hemment et al., 2023, Hemment et al., 2019).
• Generalization: Approaches such as model bending and experiential mapping are extensible to a variety of architectures (e.g., transformer-based diffusion, GANs) and creative practices, with recommendations for exposing structural elements, supporting weight and token manipulation, and sharing community “bending recipes” (Abuzuraiq et al., 10 Aug 2025).
• Evaluation: Human-centered, situated metrics (e.g., agency, perceptual clarity, narrative coherence) complement or replace technical performance indicators.

A plausible implication is that as these methods mature, the boundary between model interpretation, artistic practice, and human-centered accountability becomes increasingly porous, fostering a paradigm in which transparency is not just an informational attribute but a lived, negotiated, and culturally resonant property of AI systems.