Call to TRECVID 2004
CALL FOR PARTICIPATION
TREC VIDEO RETRIEVAL EVALUATION (TRECVID)
February 2004 - October/November 2004
National Institute of Standards and Technology (NIST)
With support from:
Advanced Research and Development Activity (ARDA)
IntroductionIn 2001 and 2002 the Text Retrieval Conference (TREC) series, which encourages and facilitates research in information retrieval, sponsored a video "track" devoted to research in automatic segmentation, indexing, and content-based retrieval of digital video. In 2003, the track became an independent 2-day workshop taking place just before TREC.
TRECVID 2004 will focus on 4 sorts of tasks, each a subproblem or variant of the content-based video retrieval task. The four task types will be shot boundary determination, story segmentation, feature extraction, and search tasks using news video from 1998 in MPEG-1 format. Interactive as well as automatic search systems are welcome.
Participants will be encouraged to share resources and intermediate system outputs to lower entry barriers and enable analysis of various components' contributions and interactions.
Much like TREC, TRECVID will provide a large test collection, uniform scoring procedures, and a forum for organizations interested in comparing their approaches and results. The details about the predecessor TREC video track and the latest about TRECVID can be found at the TRECVID web site
Organizations may choose to participate in one or more of the tasks. TRECVID participants must submit results for at least one task in order to attend the workshop. For most tasks, training and test materials, though large, will be available no charge or a nominal one. The main training/test data will be distributed by the Linguistic Data Consortium (LDC) on IDE hard drives which must either be returned or paid for.
Rough ScheduleHere is a high-level schedule to provide a rough idea of TRECVID activities. It will be revised and made precise as part of defining the evaluation guidelines.
Task DescriptionsThe evaluation will address the four types of tasks mentioned above. Details about each will be worked out starting in February based in part on input from the participants, but will likely be similar to those in the guidelines for TRECVID 2003. The trecvid2004 email discussion list will serve as the main forum for such discussion and for dissemination of other information about TRECVID 2004. The evolving task definitions will be documented on the TRECVID webpage.
Workshop FormatThe workshop itself will be used as a forum both for presentation of results (including failure analyses and system comparisons), and for more lengthy system presentations describing retrieval techniques used, experiments run using the data, and other issues of interest to researchers in information retrieval. As there is a limited amount of time for these presentations, the evaluation coordinators and NIST will determine which groups are asked to speak and which groups will present in a poster session. Groups that are interested in having a speaking slot during the workshop should submit a 200-300 word abstract shortly before the workshop describing the experiments they performed. Speakers will be selected based on these abstracts.
As some organizations may not wish to describe their proprietary algorithms, TRECVID defines two categories of participation.
Category A: Full participation
Participants will be expected to present full details of system algorithms and various experiments run using the data, either in a talk or in a poster session.
Category C: Evaluation only
Participants in this category will be expected to submit results for common scoring and tabulation, and present their results in a poster session. They will not be expected to describe their systems in detail, but will be expected to report on time and effort statistics.
DataWe expect to use 80 hours of video captured by the Linguistic Data Consortium during 1998 from CNN Headline News and ABC World News Tonight for test data. The TRECVID 2003 data will also be available to participants as training data. Conversion to MPEG-1 will occur sometime during the first quarter of 2003. Distribution of the training and test data will be handled by LDC using loaner IDE drives, which must be returned or purchased by the time of the conference. The only charge to participants for data will be the cost of shipping the drive(s) back to LDC. More information about the data will be provided on the TRECVID website starting in February as we know more. Note: Participating groups from TRECVID 2003 who received a loaner drive and have not returned or bought the drive, are not eligible to participate in TRECVID 2004.
How to respond to this callOrganizations wishing to participate in TRECVID 2004 should respond to this call for participation by submitting an application. An application consists of an email with four parts: contact information, a one-paragraph description of your retrieval approach, whether you will participate as a Category A or a Category C group, and a list of tasks that you are likely to participate in.
Contact information includes a full regular address, voice and fax telephone numbers, and an email address of the one person in the organization who will be the main TRECVID contact. If desired, you may include a second email address that will be the address to which all TRECVID-related email will be sent. For example, this second email address may be the address of a local mailing list at your institution that distributes TRECVID mail internally to project participants. If only one email address is supplied, that address will be used for the TRECVID mailing list. Please note that email is the primary method of communication in TRECVID.
All applications should be submitted by February 16, 2004 to . Any administrative questions about conference participation, application format, content, etc.should be sent to the same address.
updated: Wednesday, 14-Jan-2004 09:40:11 EST
Date created: Friday, 9-Jan-03