Hi,
This message is for the Ad-hoc Video Search (AVS) Task Participants
of TRECVID 2019 :
In order to make sure all teams participating in the task are in sync
with the task guidelines
we would like to point out some important information and milestones
about the task:
System Task:
Given the test collection (V3C1), master video shot boundary reference,
and set of Ad-hoc queries released by NIST,
return for each query a list
of at most 1000 shot IDs from the test collection ranked according to
their likelihood of
containing the target query.
Video Data:
- All Data agreement forms for various datasets can be found in the active participants area:
- Please request the V3C1 data by submitting data agreement form to
Angela Ellis ([email protected]). The form is available here:
https://www-nlpir.nist.gov/projects/tv2019/active/data.permission.forms/V3C1/V3C_Org.Form.txt
- Using the username/password NIST will send you after processing your
form, you can download the data from this URL:
https://ir.nist.gov/tv2019/V3C1/
- From NIST you can download the:
- Master shot boundary files:
https://www-nlpir.nist.gov/projects/tv2019/active/V3C1.master.shot.reference/
- Keyframes, thumbnails and metadata json files
- The segmented video shots (as webm files) based on the master shot boundary
- There is also a copy of the V3C1 dataset (with the whole raw videos
with no segmentation of shots) from ITEC university:
ftp://ftp3-itec.aau.at/ (username/password found in the file:
https://www-nlpir.nist.gov/projects/tv2019/active/data.download.info/tv19.data.servers
)
Development Data:
- Teams can use any external or IACC (previous TRECVID datasets) for
training and development purposes. NO V3C1 data is allowed for training.
- To request the IACC.2 or IACC.3 data please submit your data agreement form(s) to Angela Ellis ([email protected]). Testing Data:
- All V3C1 will be used as testing dataset.
Topics (Queries):
- According to the schedule
(https://www-nlpir.nist.gov/projects/tv2019/schedule.html) A set of 50
(textual only) queries will be distributed by NIST in 2019.
30 queries
will be evaluated and scored in the main task, and 20 queries for the
progress task will be reserved for usage in 2020 and 2021 to measure
systems
progress (see the guidelines for more details).
Run Submissions:
- Please use the provided DTD file (see the guidelines for more details)
to format your runs:
https://www-nlpir.nist.gov/projects/tv2019/dtds/videoAdhocSearchRunResult.dtd
- This year a new "Novelty" run type is introduced to label runs that
target diverse unique shots not anticipated to be commonly found across
many runs.
A new xml field in run files will be used to label a
submitted run if it is novel or common type.
- Please consult the guidelines for detailed information about number of
runs allowed for submission, training types, and submission formats.
- ALL shots submitted in runs must match the correct shot ID file names
(shotX_Y) where X is the video file name and Y is the shot number.
All shot IDs should match the NIST segmented webm video file names.
- Please plan ahead to submit your runs on time according to the schedule.
Any questions?
Please don't hesitate to send any questions, feedback, suggestions or
concerns you have to the task coordinators or the general tv19.list
mailing list.
Important Dates:
Jun. 12 : Ad-hoc video search topics available from NIST
Jul. 24 : Submissions due at NIST for evaluation
Aug. 30 : Evaluation results returned
Oct. 1 : Workshop Speaker proposals due at NIST
Oct. 29 : Workshop notebook papers due at NIST
Nov. 6 : TRECVID 2019 Workshop registration closes
Nov. 12-13 : TRECVID workshop at NIST, Gaithersburg, MD, USA
Best of Luck!
Georges Quenot
George Awad