Thursday, June 10, 2010

SimInt 2010: Simulation of Interaction

SIGIR Workshop on the Automated Evaluation of Interactive Information Retrieval will take place on July 23, at SIGIR to address themes such as: definitions of automated evaluation, methods and methodologies for automated evaluation, the ideal simulation (akin to the ideal test collection), types of experiments that can be performed, control and limitations of the different approaches, simulating queries, judgments, clicks, etc., user modelling and estimating models of interaction and research agenda with a road map of future challenges. According to their website, Interactive IR may be classified into four classes: (1) observing real users in real situations (real users, no simulation), (2) observing recruited users performing assigned tasks (real users/interactions, simulated tasks ), (3) performing simulations in the lab without users (simulation of interaction; no users), and, (4) traditional lab based (simulated) experiments (i.e. no users (or abstracted users), fixed interaction (query-response), no simulated interaction (or limited, i.e. relevance feedback.