Lyrics segmentation via bimodal text–audio representation - Equipe Signal, Statistique et Apprentissage Accéder directement au contenu
Article Dans Une Revue Natural Language Engineering Année : 2022

Lyrics segmentation via bimodal text–audio representation

Résumé

Song lyrics contain repeated patterns that have been proven to facilitate automated lyrics segmentation, with the final goal of detecting the building blocks (e.g., chorus, verse) of a song text. Our contribution in this article is twofold. First, we introduce a convolutional neural network (CNN)-based model that learns to segment the lyrics based on their repetitive text structure. We experiment with novel features to reveal different kinds of repetitions in the lyrics, for instance based on phonetical and syntactical properties. Second, using a novel corpus where the song text is synchronized to the audio of the song, we show that the text and audio modalities capture complementary structure of the lyrics and that combining both is beneficial for lyrics segmentation performance. For the purely text-based lyrics segmentation on a dataset of 103k lyrics, we achieve an F-score of 67.4%, improving on the state of the art (59.2% F-score). On the synchronized text–audio dataset of 4.8k songs, we show that the additional audio features improve segmentation performance to 75.3% F-score, significantly outperforming the purely text-based approaches.
Fichier principal
Vignette du fichier
Bi_modal_Lyrics_Segmentation__NLE_journal__minor_revision_.pdf (3.37 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03295581 , version 1 (14-10-2021)

Identifiants

Citer

Michael Fell, Yaroslav Nechaev, Gabriel Meseguer-Brocal, Elena Cabrio, Fabien Gandon, et al.. Lyrics segmentation via bimodal text–audio representation. Natural Language Engineering, 2022, 28 (3), pp.317 - 336. ⟨10.1017/S1351324921000024⟩. ⟨hal-03295581⟩
191 Consultations
183 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More