Login
Talks & Seminars
Title: New Directions in Information Retrieval Evaluation
Prof. Thomas Mandl, University of Hildesheim, Germany
Date & Time: October 15, 2010 10:30
Venue: Conference Room, 01st floor, C Block, Kanwal Rekhi Bldg.
Abstract:
In the development of search engines, it is necessary to find measures to know which system is better. Research has established a well accepted methodology to evaluate search engines. Nevertheless, the discussion on appropriate metrics, test design and the human effort involved is ongoing. New methods and ideas are being brought into the scientific discussion in order to account for the specific needs for large scale system like search engines, site and enterprise search and specific search applications. This talk will present ways to move toward more user oriented evaluation. One study on site search tried to adopt a user oriented perspective for an indicator based evaluation in a situation where the Cranfield paradigm is not appropriate. The results show that current site search has much room for improvement. The involvement of real users brings new challenges for researchers to find a balance between realism and control of the experiment. The presentation will introduce LogCLEF, a new track at CLEF which is dedicated to the analysis of queries and other logged activities as expression of user behavior. The main goal of LogCLEF is the analysis and classification of queries in order to understand search behavior in multilingual contexts and ultimately to improve search systems. Satisfaction of users can only be determined in user experiments. The talk presents results from a study on the expectations of users which shows that the design of user experiments is crucial for their success.
Speaker Profile:
Thomas Mandl is professor for information science at the University of Hildesheim in Germany. He studied information and computer science at the University of Regensburg, Germany and at the University of Illinois at Champaign/Urbana, USA. He worked as a research assistant at the Social Science Information Centre in Bonn, Germany and is now teaching in the programme *International Information Management* at the University of Hildesheim in Germany. He received both a doctorate degree and a post doctorate degree from the University of Hildesheim. His research interests include information retrieval, human-computer interaction and internationalization of information technology. He coordinated tracks at the Cross Language Evaluation Forum (CLEF), the European information retrieval evaluation initiative for six years and he is currently the chair of the special interest group in information retrieval (FGIR) of the German computer science society.
List of Talks

Webmail

Username:
Password:
Faculty CSE IT
Forgot Password
    [+] Sitemap     Feedback