[Zurück]


Beiträge in Tagungsbänden:

A. Lipani, M. Lupu, J. Palotti, G. Zuccon, A. Hanbury:
"Fixed Budget Pooling Strategies based on Fusion Methods";
in: "Proceedings of the Symposium on Applied Computing", ACM, New York, NY, USA, 2017, ISBN: 978-1-4503-4486-9, S. 919 - 924.



Kurzfassung englisch:
The empirical nature of Information Retrieval (IR) mandates strong experimental practices. The Cranfield/TREC evaluation paradigm represents a keystone of such experimental practices. Within this paradigm, the generation of relevance judgments has been the subject of intense scientific investigation. This is because, on one hand, consistent, precise and numerous judgements are key to reduce evaluation uncertainty and test collection bias; on the other hand, however, relevance judgements are costly to collect. The selection of which documents to judge for relevance (known as pooling) has therefore great impact in IR evaluation. In this paper, we contribute a set of 8 novel pooling strategies based on retrieval fusion methods. We show that the choice of the pooling strategy has significant effects on the cost needed to obtain an unbiased test collection; we also identify the best performing pooling strategy according to three evaluation measure.


"Offizielle" elektronische Version der Publikation (entsprechend ihrem Digital Object Identifier - DOI)
http://dx.doi.org/10.1145/3019612.3019692

Elektronische Version der Publikation:
http://publik.tuwien.ac.at/files/publik_264550.pdf


Erstellt aus der Publikationsdatenbank der Technischen Universitšt Wien.