J. Urbano, Markus Schedl, Xavier Serra,
"Evaluation in Music Information Retrieval"
, in Journal of Intelligent Information Systems, Vol. 41, Seite(n) 345-369, 2013
Original Titel:
Evaluation in Music Information Retrieval
Sprache des Titels:
Englisch
Original Kurzfassung:
The field of Music Information Retrieval has always acknowledged the
need for rigorous scientific evaluations, and several efforts have set out to develop
and provide the infrastructure, technology and methodologies needed to carry out
these evaluations. The community has enormously gained from these evaluation forums,
but we have reached a point where we are stuck with evaluation frameworks
that do not allow us to improve as much and as well as we want. The community
recently acknowledged this problem and showed interest in addressing it, though
it is not clear what to do to improve the situation. We argue that a good place to
start is again the Text IR field. Based on a formalization of the evaluation process,
this paper presents a survey of past evaluation work in the context of Text IR,
from the point of view of validity, reliability and efficiency of the experiments. We
show the problems that our community currently has in terms of evaluation, point
to several lines of research to improve it and make various proposals in that line.
Forschungs-