TRECVID Task Proposal Process


TRECVID consists of a series of "tasks", each focused on some facet of the multimedia retrieval problem. 
Examples of tasks include ad-hoc video search, instance search, and event detection. The tasks invigorate 
TRECVID by encouraging research in new areas of multimedia retrieval. The tasks also support the multimedia 
research community by creating the infrastructure such as test collections necessary for task-specific research.

The set of tasks run in any particular year depends on the interests of the participants and sponsors, 
as well as on the suitability of the problem to the TRECVID environment. Selecting which tasks to include 
in a particular TRECVID in the past was done informally. However, TRECVID is receiving increasingly many 
proposals for new tasks. With a limit of at most seven tasks that can be run at one time, the requests for 
new tasks have surpassed the available space. Thus, the TRECVID program committee has decided to institute 
a more formal proposal mechanism for selecting tasks.

Under this scheme, proposals for TRECVID tasks must be submitted in writing. For TRECVID 2018 tasks, proposals 
are due by September 1st, 2017. The proposals will be distributed to the TRECVID program committee members shortly 
after the deadline, and the PC will decide on the set of tasks to run in year X+1 by the time of the conference in 
year X. All tasks (both existing and proposed) must submit a proposal by the deadline, and the PC will consider the 
entire set of tasks to produce the best possible task selection for the X+1 TRECVID year. Generally, the PC will have 
to decide to terminate an existing task to accommodate a newly proposed task, so it is certainly possible that no new 
tasks will be selected in some year. 

The criteria for judging a task proposal are: a strong advocate who is willing to be the task coordinator 
(task coordinator is a volunteer position); a large enough core of interested researchers to make the task viable; 
the availability of sufficient resources such as appropriate corpora and assessors with expertise in the area; and 
the fit with other tasks. 

Proposals need to contain enough information for the PC to assess the criteria above. Proposals should contain an explicit 
statement of the goals of the task (i.e., what is expected to be learned and/or what infrastructure would be created if 
the task were run). Where is the data coming from?  Who has agreed in principle to participating in the task, and how will 
the task coordinators attract participants?  If relevance judging (or some similar sort of annotation) is required, the 
proposal needs to include where the judging would occur (NIST or elsewhere?), any special qualifications the assessors would 
need (special domain expertise required?), as well as an estimate of the amount of time such assessing would require. Any 
special constraints on the document sets needed should also be noted. Finally, proposals must contain full contact details 
of the proposer. On the flip side, proposals need to be concise and to-the-point: 
if your proposal is more than four pages, it is too long. 


Send a proposal as either a PDF or ASCII document to george.awad@nist.gov. 
Proposals in other formats will be bounced unopened.