Predicting Music Hierarchies with a Graph-Based Neural Decoder
Abstract: This paper describes a data-driven framework to parse musical sequences into dependency trees, which are hierarchical structures used in music cognition research and music analysis. The parsing involves two steps. First, the input sequence is passed through a transformer encoder to enrich it with contextual information. Then, a classifier filters the graph of all possible dependency arcs to produce the dependency tree. One major benefit of this system is that it can be easily integrated into modern deep-learning pipelines. Moreover, since it does not rely on any particular symbolic grammar, it can consider multiple musical features simultaneously, make use of sequential context information, and produce partial results for noisy inputs. We test our approach on two datasets of musical trees -- time-span trees of monophonic note sequences and harmonic trees of jazz chord sequences -- and show that our approach outperforms previous methods.
- S. Abdallah, N. Gold, and A. Marsden, “Analysing symbolic music with probabilistic grammars,” Computational music analysis, pp. 157–189, 2015.
- E. Nakamura, M. Hamanaka, K. Hirata, and K. Yoshii, “Tree-structured probabilistic model of monophonic written music based on the generative theory of tonal music,” in Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2016, pp. 276–280.
- M. Hamanaka, K. Hirata, and S. Tojo, “Time-span tree leveled by duration of time-span,” in Proceedings of the International Symposium on Computer Music Multidisciplinary Research (CMMR), 2021, pp. 155–164.
- C. Finkensiep and M. A. Rohrmeier, “Modeling and inferring proto-voice structure in free polyphony,” in Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2021, pp. 189–196.
- M. Rohrmeier, “Towards a generative syntax of tonal harmony,” Journal of Mathematics and Music, vol. 5, no. 1, pp. 35–53, 2011.
- M. Granroth-Wilding and M. Steedman, “A robust parser-interpreter for jazz chord sequences,” Journal of New Music Research, vol. 43, no. 4, pp. 355–374, 2014.
- D. Harasim, M. Rohrmeier, and T. J. O’Donnell, “A generalized parsing framework for generative models of harmonic syntax.” in Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2018, pp. 152–159.
- O. Melkonian, “Music as language: putting probabilistic temporal graph grammars to good use,” in Proceedings of the ACM SIGPLAN International Workshop on Functional Art, Music, Modeling, and Design, 2019, pp. 1–10.
- D. Harasim, T. J. O’Donnell, and M. A. Rohrmeier, “Harmonic syntax in time: rhythm improves grammatical models of harmony,” in Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2019, pp. 335–342.
- F. Foscarin, F. Jacquemard, and P. Rigaux, “Modeling and learning rhythm structure,” in Proceedings of the Sound and Music Computing Conference (SMC), 2019.
- F. Foscarin, F. Jacquemard, P. Rigaux, and M. Sakai, “A parse-based framework for coupled rhythm quantization and score structuring,” in Proceedings of the Mathematics and Computation in Music International Conference (MCM). Springer, 2019, pp. 248–260.
- F. Foscarin, R. Fournier-S’Niehotta, and F. Jacquemard, “A diff procedure for xml music score files,” in Proceedings of the International Conference on Digital Libraries for Musicology (DLfM), 2019.
- M. Rohrmeier, “Towards a formalization of musical rhythm.” in Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2020, pp. 621–629.
- W. T. Fitch and M. D. Martins, “Hierarchical processing in music, language, and action: Lashley revisited,” Annals of the New York Academy of Sciences, vol. 1316, no. 1, pp. 87–104, 2014.
- M. Tsuchiya, K. Ochiai, H. Kameoka, and S. Sagayama, “Probabilistic model of two-dimensional rhythm tree structure representation for automatic transcription of polyphonic midi signals,” in Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference. IEEE, 2013, pp. 1–6.
- I. Sakai, “Syntax in universal translation,” in Proceedings of the International Conference on Machine Translation and Applied Language Analysis, 1961.
- M. Hamanaka, K. Hirata, and S. Tojo, “Musical structural analysis database based on gttm,” in Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2014, pp. 325–330.
- D. Harasim, C. Finkensiep, P. Ericson, T. J. O’Donnell, and M. Rohrmeier, “The jazz harmony treebank,” in Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2020, pp. 207–215.
- M. Rohrmeier, “The syntax of jazz harmony: diatonic tonality, phrase structure, and form,” Music Theory and Analysis (MTA), vol. 7, no. 1, pp. 1–63, 2020.
- D. Harasim, “The learnability of the grammar of jazz: Bayesian inference of hierarchical structures in harmony,” Ph.D. dissertation, EPFL, 2020.
- C. Finkensiep, “The structure of free polyphony,” Ph.D. dissertation, EPFL, 2023.
- M. Hamanaka, K. Hirata, and S. Tojo, “Implementing “a generative theory of tonal music”,” Journal of New Music Research, vol. 35, no. 4, pp. 249–277, 2006.
- ——, “FATTA: Full automatic time-span tree analyzer,” in International Computer Music Conference (ICMC), vol. 1, 2007, pp. 153–156.
- ——, “deepGTTM-III: Multi-task Learning with Grouping and Metrical Structures,” in Proceedings of the International Symposium on Computer Music Multidisciplinary Research (CMMR), 2018, pp. 238–251.
- Y.-R. Lai and A. W.-Y. Su, “Deep learning based detection of GPR6 GTTM global feature rule of music scores,” in Proceedings of the International Conference on New Music Concepts, vol. 56, 2021.
- T. Dozat and C. D. Manning, “Deep biaffine attention for neural dependency parsing,” in Proceedings of the International Conference on Learning Representations (ICLR), 2017.
- ——, “Simpler but more accurate semantic dependency parsing,” in Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2018.
- X. Wang, J. Huang, and K. Tu, “Second-order semantic dependency parsing with end-to-end neural networks,” 2019, pp. 4609––4618.
- H. He and J. D. Choi, “Establishing strong baselines for the new decade: Sequence tagging, syntactic and semantic parsing with bert,” in Proceedings of the International Florida Artificial Intelligence Research Society Conference, 2019, pp. 228–233.
- M. Zhang, “A survey of syntactic-semantic parsing based on constituent and dependency structures,” Science China Technological Sciences, vol. 63, no. 10, pp. 1898–1920, 2020.
- L. Kong, A. M. Rush, and N. A. Smith, “Transforming dependencies into phrase structures,” in Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2015, pp. 788–798.
- N. Fradet, J.-P. Briot, F. Chhel, A. El Fallah Seghrouchni, and N. Gutowski, “MidiTok: A python package for MIDI file tokenization,” in Late-Breaking Demo Session of the International Society for Music Information Retrieval Conference (ISMIR), 2021.
- N. Fradet, J.-P. Briot, F. Chhel, A. E. F. Seghrouchni, and N. Gutowski, “Byte pair encoding for symbolic music,” arXiv preprint arXiv:2301.11975, 2023.
- EuroCC National Competence Center Sweden (ENCCS), “Graph neural networks and transformer workshop,” https://enccs.github.io/gnn_transformers/notebooks/session_1/1b_vector_sums_vs_concatenation/, 2022.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
- P. Shaw, J. Uszkoreit, and A. Vaswani, “Self-attention with relative position representations,” in Proceedings of the North American Chapter of the Association for Computational Linguistics, 2018.
- D. Fernández-González and C. Gómez-Rodríguez, “Transition-based semantic dependency parsing with pointer networks,” Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2020.
- J. M. Eisner, “Three new probabilistic models for dependency parsing: An exploration,” in Proceedings of the International Conference on Computational Linguistics (COLING), 1996.
- J. Edmonds, “Optimum branchings,” Journal of Research of the national Bureau of Standards, vol. 71, no. 4, pp. 233–240, 1967.
- Y.-J. Chu, “On the shortest arborescence of a directed graph,” Scientia Sinica, vol. 14, pp. 1396–1400, 1965.
- C. E. Cancino-Chacón, S. D. Peter, E. Karystinaios, F. Foscarin, M. Grachten, and G. Widmer, “Partitura: A Python Package for Symbolic Music Processing,” in Proceedings of the Music Encoding Conference (MEC), Halifax, Canada, 2022.
- D. Hendrycks and K. Gimpel, “Gaussian error linear units (GELUs),” arXiv preprint arXiv:1606.08415, 2016.
- I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations (ICLR), 2019.
- X. S. Huang, F. Perez, J. Ba, and M. Volkovs, “Improving transformer optimization through better initialization,” in Proceedings of the International Conference on Machine Learning (ICML), 2020, pp. 4475–4483.
- S. Mishra, B. L. Sturm, and S. Dixon, “Local Interpretable Model-agnostic Explanations for Music Content Analysis,” in Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2017, pp. 537–543.
- F. Foscarin, K. Hoedt, V. Praher, A. Flexer, and G. Widmer, “Concept-based techniques for "musicologist-friendly" explanations in a deep music classifier,” in Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2022.
- V. Praher, K. Prinz, A. Flexer, and G. Widmer, “On the veracity of local, model-agnostic explanations in audio classification: targeted investigations with adversarial examples,” in Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2021.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.