11-7 Optional alternative reading

11-7 Ajanki et al. (2009). Can eyes reveal interest? Implicit queries from gaze patterns

11-7 Moshfeghi, Yashar, et al. Understanding relevance: an fMRI study

11-7 Vo, T., & Gedeon, T. Reading Your Mind: EEG during Reading Task

10-31 Mark Sanderson. CHAPTER 5 - Test collection based evaluation of information retrieval systems, Foundations and Trends in Information Retrieval, pp. 308-318.

10-31 Smucker, Mark D., James Allan, and Ben Carterette. A comparison of statistical significance tests for information retrieval evaluation. CIKM’07.

10-31 M. Sanderson and J. Zobel, Information retrieval system evaluation: effort, sensitivity, and reliability, SIGIR’05

10-31 Optional Alternative Reading

10-24 Optional alternative reading

10-24 Moffat, Alistair, Falk Scholer, and Paul Thomas. Models and metrics: IR evaluation as a user process.

10-24 Oard et al. Building an information retrieval test collection for spontaneous conversational speech. SIGIR 2004, pp. 41-48.

10-24 Maria Eskevich et al. Creating a data collection for evaluating rich speech retrieval. LREC’12.

10-17 Lydia Chilton et al. Task search in a human computation market

10-17 W. John Wilbur and Won Kim. Improving a gold standard: treating human relevance judgments of MEDLINE document pairs

10-17 Azzah Al‐Maskari and Mark Sanderson. A review of factors influencing user satisfaction in information retrieval

10-17 Optional Alternative Reading

10-10 Optional Alternative Reading

10-10 A Chouldechova, D Mease. Differences in search engine evaluations between query owners and non-owners. WSDM’13.

10-10 P Organisciak, M Efron, K Fenlon, M Senseney. Evaluating rater quality and rating difficulty in online annotation activities. ASIS&T’12.

10-10 Harris, Christopher G., and Padmini Srinivasan. Applying human computation mechanisms to information retrieval. ASIS&T 2012.