Call for Participation in TRECVID 2012


February 2012 - November 2012

Conducted by the National Institute of Standards and Technology (NIST)
With support from other US government agencies

I n t r o d u c t i o n:

The TREC Video Retrieval Evaluation series ( promotes
progress in content-based analysis of and retrieval from digital video
via open, metrics-based evaluation. TRECVID is a laboratory-style
evaluation that attempts to model real world situations or significant
component tasks involved in such situations.

D a t a:

In TRECVID 2012 NIST will use at least the following data sets:

      * IACC.1.A, IACC.1.B, and IACC.1.C

      Each has about 8000 Internet Archive videos (50GB, 200 hours)
      with Creative Commons licenses in MPEG-4/H.264 with duration
      between 10 seconds and 3.5 minutes. Most videos will have some
      metadata provided by the donor available e.g., title, keywords,
      and description


      Approximately 3200 Internet Archive videos (??GB, 200 hours)
      with Creative Commons licenses in MPEG-4/H.264 with durations
      between 3.6 and 4.1 minutes. Most videos will have some metadata
      provided by the donor available e.g., title, keywords, and

      * BBC rushes

      Unedited video (MPEG-1) provided by the BBC Archive, rushes, are
      the raw material for programming and in the case of this data
      include mostly travel video and material from several dramatic
      series. About 21,000 short clips were used in the Instance Search
      task in 2011 and will be available for system development.

      * Gatwick and i-LIDS MCT airport surveillance video

      The data consist of about 150 hours obtained from airport
      surveillance video data (courtesy of the UK Home Office). The
      Linguistic Data Consortium has provided event annotations for
      the entire corpus. The corpus was divided into development and
      evaluation subsets. Annotations for 2008 development and test
      sets are available.

      * HAVIC

      HAVIC is designed to be a large new collection of Internet
      multimedia. Construction by the Linguistic Data Consortium and
      NIST began in 2010.   HAVIC data used in 2010 and 2011 will be
      available for system development.  A new, ~4,000 hour Progress
      Test collection will be provided to participants and used
      through MED '15 as the test collection.

T a s k s:

In TRECVID 2012 NIST will evaluate systems on the following tasks
using the [data] indicated:

    * Known-item search task (interactive, manual, automatic) [IACC.1]

      The known-item search task models the situation in which someone
      knows of a video, has seen it before, believes it is contained
      in a collection, but doesn't know where to look. To begin the
      search process, the searcher formulates a text-only description,
      which captures what the searcher remembers about the target
      video. 300 topics are planned for automatic systems, a
      subset of 25 for human-in-the-loop systems.  The focus this year
      will be on dramatically reducing the percentage of known-items that  
      no system is able to find (~30%   in 2010 and 2011) .

    * Semantic indexing [IACC.1]

      Automatic assignment of semantic tags to video segments can be
      fundamental technology for filtering, categorization, browsing,
      search, and other video exploitation. Technical issues to be
      addressed include methods needed/possible as collection size and
      diversity increase, when the number of features increases, and
      when features are related by an ontology.  A "concept pair"
      version of the task will be added along with a "no annotation"

    * Interactive surveillance video event detection (interactive)  [i-LIDS]

      Detecting human behaviors efficiently in vast amounts  surveillance 
      video, both retrospectively and in realtime, is fundamental technology 
      for a variety of higher-level applications of critical importance to public 
      safety and  security. In 2012 this task participants will examine the
      performance of interactive surveillance video search for a known
      set of events.

    * Instance search [Internet video, sources to be determined] (interactive, automatic)

      An important need in many situations involving video collections
      (archive video search/reuse, personal video organization/search,
      surveillance, law enforcement, protection of brand/logo use) is
      to find more video segments of a certain specific person,
      object, or place, given a visual example. In 2012 this will
      still be a pilot task.  We expect to use a new collection of
      Internet video and a much larger set of topics (~50) for
      automatic systems.

    * Multimedia event detection [HAVIC]

      Exploding multimedia content in the Internet necessitates
      development of new technologies for content understanding and
      search for a wide variety of commerce, research, and government
      applications.  The 2012 evaluation will offer both the
      "Pre-Specified Event Detection" task where developers receive
      event kits in advance of building the Content Description
      Representation (CDR) and a Pilot "Ad Hoc Event Detection" task
      where developers are given event kits after the CDR is frozen.

In addition to the data, TRECVID will provide uniform scoring
procedures, and a forum for organizations interested in comparing
their approaches and results.

Participants will be encouraged to share resources and intermediate
system outputs to lower entry barriers and enable analysis of various
components' contributions and interactions.

* You are invited to participate in TRECVID 2012 *

The evaluation is defined by the Guidelines. A draft version is
available: and
details will be worked out starting in February based in part on input
from the participants.

You should read the guidelines carefully before applying to
participate: Guidelines

P l e a s e   n o t e:
1) Dissemination of TRECVID work and results other than in the
(publicly available) conference proceedings is welcomed, but the
conditions of participation specifically preclude any advertising 
claims based on TRECVID results.

2) All system results submitted to NIST are published in the
Proceedings and on the public portions of TRECVID web site archive.

3) The workshop is open only to participating groups that submit
results for at least one task and to selected government personnel
from sponsoring agencies and data donors.

4) Each participating group is required to submit a notebook
paper describing their experiments and results. This is true even
for groups who may not be able to attend the workshop.

5) By applying to participate you indicate your acceptance of the
above conditions and obligations.

T e n t a t i v e   s c h e d u l e

There is a tentative schedule for the tasks included in the Guidelines
webpage: Schedule

W o r k s h o p   f o r m a t

The 2 1/2 day workshop itself, November 26-28 at NIST in Gaithersburg,
Maryland near Washington,DC, will be used as a forum both for
presentation of results (including failure analyses and system
comparisons), and for more lengthy system presentations describing
retrieval techniques used, experiments run using the data, and other
issues of interest to researchers in information retrieval. As there
is a limited amount of time for these presentations, the evaluation
coordinators and NIST will determine which groups are asked to speak
and which groups will present in a poster session. Groups that are
interested in having a speaking slot during the workshop will be asked
to submit a short abstract before the workshop describing the
experiments they performed. Speakers will be selected based on these

H o w   t o   r e s p o n d   t o   t h i s   c a l l

Organizations wishing to participate in TRECVID 2012 must respond
to this call for participation by submitting an on-line application by
20. February.  Only ONE APPLICATION PER TEAM please, regardless of how
many organizations the team comprises.

*PLEASE* only apply if you are able and fully intend to complete the
work for at least one task. Taking the data but not submitting any
runs threatens the continued operation of the workshop and the
availability of data for the entire community.

Here is the application URL: 

Once you have applied, you'll be given the active participant's userid
and password, be subscribed to the tv12.list email discussion list,
and can participate in finalizing the guidelines as well as sign up to
get the data, which is controlled by separate passwords. 

T R E C V I D   2 0 1 2   e m a i l   d i s c u s s i o n   l i s t

The tv12.list email discussion list ( will serve as
the main forum for discussion and for dissemination information about
TRECVID 2012.  It is each participant's responsibility to monitor the
tv12.list postings.  It accepts postings only from the email addresses
used to subscribe to it.  At the bottom of the guidelines there is a
link to an archive of past postings available using the active
participant's userid/password.

Q u e s t i o n s

Any administrative questions about conference participation,
application format/content, subscriptions to the tv12.list,
etc. should be sent to Lori.Buckland at

If you would like to contribute to TRECVID in one or both of the
following ways, please contact Paul Over (info at bottom of page)
directly as soon as possible:

- agree to host 2012 test video data for download by other
  participants on a fast, password-protected site. (Asian and 
  European sites especially needed)

- agree to provide the output of your automatic speech recognition
  system run on the IACC.1.C test video (at least for the English

Best regards,

Paul Over
Alan Smeaton
Wessel Kraaij
Georges Quenot
Jon Fiscus

National Institute of
Standards and Technology Home Last updated: Wednesday, 01-Feb-2012 12:04:35 MST
Date created: Wednesday, 25-Jan-12
For further information contact