Guidelines for the TRECVID 2009 Evaluation

(last updated: Friday, 13-Jun-2014 11:02:19 EDT)

0. Table of Contents:


1. Introduction:

The main goal of the TREC Video Retrieval Evaluation (TRECVID) is to promote progress in content-based analysis of and retrieval from digital video via open, metrics-based evaluation. TRECVID is a laboratory-style evaluation that attempts to model real world situations or significant component tasks involved in such situations.

In 2006 TRECVID completed the second two-year cycle devoted to automatic segmentation, indexing, and content-based retrieval of digital video - broadcast news in English, Arabic, and Chinese. It also completed two years of pilot studies on exploitation of unedited video (rushes). Some 70 research groups have been provided with the TRECVID 2005-2006 broadcast news video and many resources created by NIST and the TRECVID community are available for continued research on this data independent of TRECVID. See the "Past data" section of the TRECVID website for pointers.

In 2007 TRECVID began exploring new data (cultural, news magazine, documentary, and education programming) and an additional, new task - video rushes summarization. In 2008 that work continued with the exception that the shot boundary detection task was retired and copy detection and surveillance event detection were added.

TRECVID 2009 will test systems on the following tasks:

Changes to be noted by 2008 participants:


2. Video data:

A number of datasets are available for use in TRECVID 2009. We describe them here and then indicate below which data will be used for development versus test for each task.

Sound and Vision

BBC rushes

TRECVID 2009 surveillance video

MUSCLE-VCD-2007


3. Data license agreements for active participants

In order to be eligible to receive the data, you must have have applied for participation in TRECVID. Your application will be acknowledged by NIST with a team ID, and active participant's password, and information about how to obtain the data.

Then you will need to complete the relevant permission forms (from the active participant's area) and email a scanned image (jpg) of each page or a pdf file of the document to In your email include the following:

As Subject: "TRECVID data request"
In the body: your name
             your team ID (given to you when you apply to participate)
             the kinds of data you are requesting (BBC, S&V, and/or Gatwick)

If you cannot provide a jpg/pdf of the forms, please create a cover sheet (Attention: Lori Buckland) that identifies you, your team ID, your email address, and each kind of data (BBC, S&V, and/or Gatwick) you are requesting. Then fax all pages to fax
number in the US. If we are unable to read your jpg/pdf files we will request a faxed copy of your forms.

Please ask only for the test data (and optional development data) required for the task(s) you apply to participate in and intend to complete. One permission form will cover 2007, 2008, and 2009 BBC data. One permission form will cover 2007, 2008, and 2009 Sound and Vision data. One permission form will cover the 2008 London Gatwick data to be used for development in 2009. The 2009 London Gatwick test data will be handled separately.

Within a few days after the permission forms have been received, you will be emailed the access codes you need to download the data using the information about data servers in the the active participant's area.