Alan Smeaton and Peter Wilkins at Dublin City University, with comments from other TRECVID participants, have created a set of questionnaires to be used by groups doing interactive search experiments - in order to begin collecting some comparable data about user characteristics and satisfaction. The following is the explanatory material they have provided.

Dear TRECVid 2004 Participants

This message applies to those of you doing the interactive search task.

Many thanks to those of you who gave feedback on our first version of the questionnaires and below are the links to the online questionnaires which we would like your interactive search users to complete. Please read the instructions below carefully.

There are three separate questionnaires as follows:

• A. The pre-experiment user questionnaire should be completed once for each individual user who will take part in the interactive search task, before they start searching or training.
• B. The post-topic questionnaire should be completed once for each topic that each user completes on each system at your site.
• C. The post-experiment questionnaire should be completed once for each individual user who took part in the interactive search task after they have completed their searching.

In order for us to be able to identify the returns from what user searched what topic and at what site, each site is given a unique URL for each questionnaire.

On each form the user is asked for a UserID which is something that you must assign to each interactive user at your own site. You can use the user's name, or a code, just so long as it is unique to each user at your site and the UserIds are used consistently at your site. For the Post-Topic Questionnaire (B) your user is also asked to select the particular topic from the set of 24 that they are replying to.

The URL for each questionnaire is a *combination* of a common pathname, plus your site name. So, for City University of Hong King, for example, the URLs are:

• Pre-experiment CHUK
• Post-topic CHUK
• Post-experiment CHUK

and for the MediaMill group the URLs are:

• Pre-experiment MediaMill
• Post-topic MediaMill
• Post-experiment MediaMill

The list of site names and the mapping code appears below. They are case-sensitive. If you are planning to do the interactive search task and your sitename does not appear below then reply to me and to Peter Wilkins and we will update the form by adding you.

Accenture Technology Labs: ATL
Bilkent Uni: Bilkent
City Uni Hong King: CUHK
CMU: CMU
CWI & Tweak: CWI
Deakin Uni: Deakin
DCU: DCU
ETRI: ETRI
FX Palo Alto: FX
IBM: IBM
Imperial College London: ICL
Dan Albertson: Dan
Joanneum Uni: Joanneum
MediaMill: MediaMill
National Taiwan Uni: NTU
National Uni Singapore: NUS
Purdue Uni: Purdue
SCHEMA: SCHEMA
Uni Arizona: Arizona
Uni Bremen: Bremen
Uni Central Florida: Florida
Uni Maryland: Maryland
Uni Mass. : UM
Uni North Carolina: UNC
MediaTeam: MediaTeam
Viper Team: Viper

When any of the forms is submitted data is gathered into a database but of course each of us will want to play with the forms first before our search sessions, and even afterwards we may inadvertently submit entries via the forms. In order to allow us to distinguish real captured data from trials with the forms, when each form is submitted it generates a web page with a unique code number WHICH SHOULD BE RETAINED BY YOU FOR REFERENCE. After your search sessions are complete you can let us know which are the valid codes and we can delete the rogues leaving just the clean data which we will make available. The codes are contiguous which will help in gathering this information. We have checked this form on a number of web browsers and found it works OK. If you find a problem or an issue with any of the forms that needs a fix then reply to me and to Peter Wilkins rather than to the trecvid2004 list, and we will make the fix and announce to the list. Alan Smeaton

Last updated: Wednesday, 08-Sep-2004 15:55:06 EDT
Date created: Monday, 9-May-03
For further information contact