This page presents in a little more detail the steps we plan to follow in administering the TREC video track's shot boundary detection task. Our work is divided into three main parts: extraction of a test subset from the test collection, different methods to produce the reference segmentation, and specification of the comparison system. Much of this process will follow the work done by ISIS groups on methodologies for the evaluation of multi-media indexing systems. In particular, we will build on the work of the first evaluation. You can find more information about their workshops using this URL www-asim.lip6.fr/AIM
Our task here was to pick a subset of 5 hrs. which is as varied in its use of various shot transition types as our collection allows. We also want a mix of other characteristics of the videos which interact with shot detection - motion, lighting, etc. The definition of the test subset will be announced shortly before the systems are to be evaluated on it.
Participating researchers will need to make sure the systems they use in the test do not reflect training on the test set.
NIST will construct the reference segmentation manually.