TREC Video Retrieval Evaluation (TRECVID) Call to TRECVID 2004

Call to TRECVID 2004

CALL FOR PARTICIPATION in the TREC VIDEO RETRIEVAL EVALUATION (TRECVID) February 2004 - October/November 2004 Conducted by: National Institute of Standards and Technology (NIST) With support from: Advanced Research and Development Activity (ARDA)
Introduction In 2001 and 2002 the Text Retrieval Conference (TREC) series, which encourages and facilitates research in information retrieval, sponsored a video "track" devoted to research in automatic segmentation, indexing, and content-based retrieval of digital video. In 2003, the track became an independent 2-day workshop taking place just before TREC. TRECVID 2004 will focus on 4 sorts of tasks, each a subproblem or variant of the content-based video retrieval task. The four task types will be shot boundary determination, story segmentation, feature extraction, and search tasks using news video from 1998 in MPEG-1 format. Interactive as well as automatic search systems are welcome. Participants will be encouraged to share resources and intermediate system outputs to lower entry barriers and enable analysis of various components' contributions and interactions. Much like TREC, TRECVID will provide a large test collection, uniform scoring procedures, and a forum for organizations interested in comparing their approaches and results. The details about the predecessor TREC video track and the latest about TRECVID can be found at the TRECVID web site You are invited to participate in TRECVID 2004. Organizations may choose to participate in one or more of the tasks. TRECVID participants must submit results for at least one task in order to attend the workshop. For most tasks, training and test materials, though large, will be available no charge or a nominal one. The main training/test data will be distributed by the Linguistic Data Consortium (LDC) on IDE hard drives which must either be returned or paid for. Please note: Dissemination of TRECVID work and results other than in the (publicly available) conference proceedings is welcomed, but the conditions of participation specifically preclude any advertising claims based on TRECVID results. All retrieval results submitted to NIST are published in the Proceedings and on the public portions of TRECVID web site archive. The workshop is open only to participating groups that submit results for at least one task and to selected government personnel from sponsoring agencies. Rough Schedule Here is a high-level schedule to provide a rough idea of TRECVID activities. It will be revised and made precise as part of defining the evaluation guidelines. 12 Jan NIST sends out Call for participation in TRECVID 2004 16 Feb Applications for participation in TRECVID 2004 due at NIST 1 Mar Final versions of TRECVID 2003 papers due at NIST Training data (i.e., 2003 development and/or test data) available from LDC 1 Apr Guidelines complete ?? ??? Test data, common shot reference, and key frames available on IDE harddrives ?? ??? Shot boundary test collection DVDs shipped by NIST ?? ??? Search topics available from TRECVID website to active participants. ?? ??? Shot boundary detection submissions due at NIST for evaluation. ?? ??? Feature extraction task submissions due at NIST for evaluation. Feature extraction donations due at NIST ?? ??? Feature extraction donations available for active participants ?? ??? Story segmentation submissions due at NIST for evaluation ?? ??? Results of shot boundary evaluations returned to participants ?? ??? Search task submissions due at NIST for evaluation ?? ??? - ?? ??? Search and feature assessment at NIST ?? ??? Results of story segmentation evaluations returned to participants ?? ??? Results of search and feature extraction evaluations returned to participants ?? ??? Speaker proposals due at NIST ?? ??? Notebook papers due at NIST ?? ??? Workshop registration closes ?? ??? TRECVID Workshop at ACM MM in New York City (mid-Oct) or NIST in Gaithersburg, Md (mid-Nov) ?? ??? Workshop papers and slides publicly available ?? ??? Final versions of TRECVID 2004 papers due at NIST Task Descriptions The evaluation will address the four types of tasks mentioned above. Details about each will be worked out starting in February based in part on input from the participants, but will likely be similar to those in the guidelines for TRECVID 2003. The trecvid2004 email discussion list will serve as the main forum for such discussion and for dissemination of other information about TRECVID 2004. The evolving task definitions will be documented on the TRECVID webpage. Workshop Format The workshop itself will be used as a forum both for presentation of results (including failure analyses and system comparisons), and for more lengthy system presentations describing retrieval techniques used, experiments run using the data, and other issues of interest to researchers in information retrieval. As there is a limited amount of time for these presentations, the evaluation coordinators and NIST will determine which groups are asked to speak and which groups will present in a poster session. Groups that are interested in having a speaking slot during the workshop should submit a 200-300 word abstract shortly before the workshop describing the experiments they performed. Speakers will be selected based on these abstracts. As some organizations may not wish to describe their proprietary algorithms, TRECVID defines two categories of participation. Category A: Full participation Participants will be expected to present full details of system algorithms and various experiments run using the data, either in a talk or in a poster session. Category C: Evaluation only Participants in this category will be expected to submit results for common scoring and tabulation, and present their results in a poster session. They will not be expected to describe their systems in detail, but will be expected to report on time and effort statistics. Data We expect to use 80 hours of video captured by the Linguistic Data Consortium during 1998 from CNN Headline News and ABC World News Tonight for test data. The TRECVID 2003 data will also be available to participants as training data. Conversion to MPEG-1 will occur sometime during the first quarter of 2003. Distribution of the training and test data will be handled by LDC using loaner IDE drives, which must be returned or purchased by the time of the conference. The only charge to participants for data will be the cost of shipping the drive(s) back to LDC. More information about the data will be provided on the TRECVID website starting in February as we know more. Note: Participating groups from TRECVID 2003 who received a loaner drive and have not returned or bought the drive, are not eligible to participate in TRECVID 2004. How to respond to this call Organizations wishing to participate in TRECVID 2004 should respond to this call for participation by submitting an application. An application consists of an email with four parts: contact information, a one-paragraph description of your retrieval approach, whether you will participate as a Category A or a Category C group, and a list of tasks that you are likely to participate in. Contact information includes a full regular address, voice and fax telephone numbers, and an email address of the one person in the organization who will be the main TRECVID contact. If desired, you may include a second email address that will be the address to which all TRECVID-related email will be sent. For example, this second email address may be the address of a local mailing list at your institution that distributes TRECVID mail internally to project participants. If only one email address is supplied, that address will be used for the TRECVID mailing list. Please note that email is the primary method of communication in TRECVID. All applications should be submitted by February 16, 2004 to . Any administrative questions about conference participation, application format, content, etc.should be sent to the same address. Contacts
Last updated: Wednesday, 14-Jan-2004 07:40:11 MST Date created: Friday, 9-Jan-03