SnakeSynth: New Interactions for Generative Audio Synthesis
Abstract: I present "SnakeSynth," a web-based lightweight audio synthesizer that combines audio generated by a deep generative model and real-time continuous two-dimensional (2D) input to create and control variable-length generative sounds through 2D interaction gestures. Interaction gestures are touch and mobile-compatible with analogies to strummed, bowed, and plucked musical instrument controls. Point-and-click and drag-and-drop gestures directly control audio playback length and I show that sound length and intensity are modulated by interactions with a programmable 2D coordinate grid. Leveraging the speed and ubiquity of browser-based audio and hardware acceleration in Google's TensorFlow.js we generate time-varying high-fidelity sounds with real-time interactivity. SnakeSynth adaptively reproduces and interpolates between sounds encountered during model training, notably without long training times, and I briefly discuss possible futures for deep generative models as an interactive paradigm for musical expression.
- Adversarial Audio Synthesis, Feb. 2019. arXiv:1802.04208 [cs].
- GANSynth: Adversarial Neural Audio Synthesis, Apr. 2019. arXiv:1902.08710 [cs, eess, stat].
- P. M. Fitts. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6):381–391, June 1954.
- Generative Adversarial Networks, June 2014. arXiv:1406.2661 [cs, stat].
- Deep AutoRegressive Networks, May 2014. arXiv:1310.8499 [cs, stat].
- GANSpace: Discovering Interpretable GAN Controls, Dec. 2020. arXiv:2004.02546 [cs].
- S. Ioffe and C. Szegedy. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32nd International Conference on Machine Learning, pages 448–456. PMLR, June 2015.
- Progressive Growing of GANs for Improved Quality, Stability, and Variation, Feb. 2018. arXiv:1710.10196 [cs, stat].
- D. P. Kingma and M. Welling. Auto-Encoding Variational Bayes, May 2014. arXiv:1312.6114 [cs, stat].
- WaveNet: A Generative Model for Raw Audio, Sept. 2016. arXiv:1609.03499 [cs].
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, Jan. 2016. arXiv:1511.06434 [cs].
- S. Shahriar. GAN computers generate arts? A survey on visual arts, music, and literary text generation using generative adversarial network. Displays, 73:102237, July 2022.
- Implement Music Generation with GAN: A Systematic Review. In 2021 International Conference on Computer Engineering and Application (ICCEA), pages 352–355, Kunming, China, June 2021. IEEE.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
