Call for Participation in TRECVID 2012
CALL FOR PARTICIPATION in the
2012 TREC VIDEO RETRIEVAL EVALUATION (TRECVID 2012)
February 2012 - November 2012
Conducted by the National Institute of Standards and Technology (NIST)
With support from other US government agencies
I n t r o d u c t i o n:
The TREC Video Retrieval Evaluation series (trecvid.nist.gov) promotes
progress in content-based analysis of and retrieval from digital video
via open, metrics-based evaluation. TRECVID is a laboratory-style
evaluation that attempts to model real world situations or significant
component tasks involved in such situations.
D a t a:
In TRECVID 2012 NIST will use at least the following data sets:
* IACC.1.A, IACC.1.B, and IACC.1.C
Each has about 8000 Internet Archive videos (50GB, 200 hours)
with Creative Commons licenses in MPEG-4/H.264 with duration
between 10 seconds and 3.5 minutes. Most videos will have some
metadata provided by the donor available e.g., title, keywords,
and description
* IACC.1.tv10.training
Approximately 3200 Internet Archive videos (??GB, 200 hours)
with Creative Commons licenses in MPEG-4/H.264 with durations
between 3.6 and 4.1 minutes. Most videos will have some metadata
provided by the donor available e.g., title, keywords, and
description
* BBC rushes
Unedited video (MPEG-1) provided by the BBC Archive, rushes, are
the raw material for programming and in the case of this data
include mostly travel video and material from several dramatic
series. About 21,000 short clips were used in the Instance Search
task in 2011 and will be available for system development.
* Gatwick and i-LIDS MCT airport surveillance video
The data consist of about 150 hours obtained from airport
surveillance video data (courtesy of the UK Home Office). The
Linguistic Data Consortium has provided event annotations for
the entire corpus. The corpus was divided into development and
evaluation subsets. Annotations for 2008 development and test
sets are available.
* HAVIC
HAVIC is designed to be a large new collection of Internet
multimedia. Construction by the Linguistic Data Consortium and
NIST began in 2010. HAVIC data used in 2010 and 2011 will be
available for system development. A new, ~4,000 hour Progress
Test collection will be provided to participants and used
through MED '15 as the test collection.
T a s k s:
In TRECVID 2012 NIST will evaluate systems on the following tasks
using the [data] indicated:
* Known-item search task (interactive, manual, automatic) [IACC.1]
The known-item search task models the situation in which someone
knows of a video, has seen it before, believes it is contained
in a collection, but doesn't know where to look. To begin the
search process, the searcher formulates a text-only description,
which captures what the searcher remembers about the target
video. 300 topics are planned for automatic systems, a
subset of 25 for human-in-the-loop systems. The focus this year
will be on dramatically reducing the percentage of known-items that
no system is able to find (~30% in 2010 and 2011) .
* Semantic indexing [IACC.1]
Automatic assignment of semantic tags to video segments can be
fundamental technology for filtering, categorization, browsing,
search, and other video exploitation. Technical issues to be
addressed include methods needed/possible as collection size and
diversity increase, when the number of features increases, and
when features are related by an ontology. A "concept pair"
version of the task will be added along with a "no annotation"
condition.
* Interactive surveillance video event detection (interactive) [i-LIDS]
Detecting human behaviors efficiently in vast amounts surveillance
video, both retrospectively and in realtime, is fundamental technology
for a variety of higher-level applications of critical importance to public
safety and security. In 2012 this task participants will examine the
performance of interactive surveillance video search for a known
set of events.
* Instance search [Internet video, sources to be determined] (interactive, automatic)
An important need in many situations involving video collections
(archive video search/reuse, personal video organization/search,
surveillance, law enforcement, protection of brand/logo use) is
to find more video segments of a certain specific person,
object, or place, given a visual example. In 2012 this will
still be a pilot task. We expect to use a new collection of
Internet video and a much larger set of topics (~50) for
automatic systems.
* Multimedia event detection [HAVIC]
Exploding multimedia content in the Internet necessitates
development of new technologies for content understanding and
search for a wide variety of commerce, research, and government
applications. The 2012 evaluation will offer both the
"Pre-Specified Event Detection" task where developers receive
event kits in advance of building the Content Description
Representation (CDR) and a Pilot "Ad Hoc Event Detection" task
where developers are given event kits after the CDR is frozen.
In addition to the data, TRECVID will provide uniform scoring
procedures, and a forum for organizations interested in comparing
their approaches and results.
Participants will be encouraged to share resources and intermediate
system outputs to lower entry barriers and enable analysis of various
components' contributions and interactions.
***************************************************
* You are invited to participate in TRECVID 2012 *
***************************************************
The evaluation is defined by the Guidelines. A draft version is
available: http://www-nlpir.nist.gov/projects/tv2012/tv2012.html and
details will be worked out starting in February based in part on input
from the participants.
You should read the guidelines carefully before applying to
participate: Guidelines
P l e a s e n o t e:
1) Dissemination of TRECVID work and results other than in the
(publicly available) conference proceedings is welcomed, but the
conditions of participation specifically preclude any advertising
claims based on TRECVID results.
2) All system results submitted to NIST are published in the
Proceedings and on the public portions of TRECVID web site archive.
3) The workshop is open only to participating groups that submit
results for at least one task and to selected government personnel
from sponsoring agencies and data donors.
4) Each participating group is required to submit a notebook
paper describing their experiments and results. This is true even
for groups who may not be able to attend the workshop.
5) By applying to participate you indicate your acceptance of the
above conditions and obligations.
T e n t a t i v e s c h e d u l e
There is a tentative schedule for the tasks included in the Guidelines
webpage: Schedule
W o r k s h o p f o r m a t
The 2 1/2 day workshop itself, November 26-28 at NIST in Gaithersburg,
Maryland near Washington,DC, will be used as a forum both for
presentation of results (including failure analyses and system
comparisons), and for more lengthy system presentations describing
retrieval techniques used, experiments run using the data, and other
issues of interest to researchers in information retrieval. As there
is a limited amount of time for these presentations, the evaluation
coordinators and NIST will determine which groups are asked to speak
and which groups will present in a poster session. Groups that are
interested in having a speaking slot during the workshop will be asked
to submit a short abstract before the workshop describing the
experiments they performed. Speakers will be selected based on these
abstracts.
H o w t o r e s p o n d t o t h i s c a l l
Organizations wishing to participate in TRECVID 2012 must respond
to this call for participation by submitting an on-line application by
20. February. Only ONE APPLICATION PER TEAM please, regardless of how
many organizations the team comprises.
*PLEASE* only apply if you are able and fully intend to complete the
work for at least one task. Taking the data but not submitting any
runs threatens the continued operation of the workshop and the
availability of data for the entire community.
Here is the application URL:
http://ir.nist.gov/tv-submit.open/application.html
Once you have applied, you'll be given the active participant's userid
and password, be subscribed to the tv12.list email discussion list,
and can participate in finalizing the guidelines as well as sign up to
get the data, which is controlled by separate passwords.
T R E C V I D 2 0 1 2 e m a i l d i s c u s s i o n l i s t
The tv12.list email discussion list (tv12.list@nist.gov) will serve as
the main forum for discussion and for dissemination information about
TRECVID 2012. It is each participant's responsibility to monitor the
tv12.list postings. It accepts postings only from the email addresses
used to subscribe to it. At the bottom of the guidelines there is a
link to an archive of past postings available using the active
participant's userid/password.
Q u e s t i o n s
Any administrative questions about conference participation,
application format/content, subscriptions to the tv12.list,
etc. should be sent to Lori.Buckland at nist.gov
If you would like to contribute to TRECVID in one or both of the
following ways, please contact Paul Over (info at bottom of page)
directly as soon as possible:
- agree to host 2012 test video data for download by other
participants on a fast, password-protected site. (Asian and
European sites especially needed)
- agree to provide the output of your automatic speech recognition
system run on the IACC.1.C test video (at least for the English
speech)
Best regards,
Paul Over
Alan Smeaton
Wessel Kraaij
Georges Quenot
Jon Fiscus
Last
updated: Wednesday, 01-Feb-2012 12:04:35 MST
Date created:
Wednesday, 25-Jan-12
For further information contact