Matthias Dorfer, Andreas Arzt, Gerhard Widmer,
"Towards Score Following in Sheet Music Images"
: Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR), 8-2016
Original Titel:
Towards Score Following in Sheet Music Images
Sprache des Titels:
Englisch
Original Buchtitel:
Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR)
Original Kurzfassung:
This paper addresses the matching of short music audio
snippets to the corresponding pixel location in images of
sheet music. A system is presented that simultaneously
learns to read notes, listens to music and matches the
currently played music to its corresponding notes in the
sheet. It consists of an end-to-end multi-modal convolu-
tional neural network that takes as input images of sheet
music and spectrograms of the respective audio snippets.
It learns to predict, for a given unseen audio snippet (cov-
ering approximately one bar of music), the corresponding
position in the respective score line. Our results suggest
that with the use of (deep) neural networks ? which have
proven to be powerful image processing models ? working
with sheet music becomes feasible and a promising future
research direction.