31. March 2008 TRECVID 2008 High Level Feature Evaluation G e n e r a l r u l e s: The following rules are used by the NIST assessors in judging TRECVID feature and search system results and so should also be used by TRECVID training data annotators and system builders. a) Features are meant to describe the presence or absence of VIDEO of some target person, place, thing, activity, etc., NOT INFORMATION about that target. So, for example, video just of someone talking about X is not by itself sufficient to assert that the feature X is true with respect to the video. b) When a feature definition says a shot must contain x, that is short for "contain x to a degree sufficient for x to be recognizable as x to a human". This means among other things that unless explicitly stated, partial visibility or audibility may suffice. c) The fact that a segment contains video of physical objects REPRESENTING the feature target, such as photos, paintings, models, or toy versions of the feature target, should NOT be grounds for judging the segment relevant/true. Containing video of the target within the video segment may be grounds for doing so. F e a t u r e s f o r e v a l u a t i o n Please use the numbers listed below when submitting results for these features 001 Classroom: a school- or university-style classroom scene. One or more students must be visible. A teacher and teaching aids (e.g. blackboard) may or may not be visible. 002 Bridge: a structure carrying a pathway or roadway over a depression or obstacle. Such structures over non-water bodies such as a highway overpass or a catwalk (e.g., as found over a factory or warehouse floor) are included. 003 Emergency_Vehicle: external view of, for example, a police car or van, fire truck or ambulance. There may be other sorts of emergency vehicles. Included may be UN vehicles, but NOT military vehicles 004 Dog: any kind of dog, but not wolves 005 Kitchen: a room where food is prepared, dishes washed, etc. 006 Airplane_flying: external view of a heavier than air, fixed-wing aircraft in flight - gliders included. NOT balloons, helicopters, missiles, and rockets 007 Two people: a view of exactly two people (not as part of a larger visible group) 008 Bus: external view of a large motor vehicle on tires used to carry many passengers on streets, usually along a fixed route. NOT vans and SUVs 009 Driver: a person operating a motor vehicle or at least in the driver's seat of such a vehicle 010 Cityscape: a view of a large urban setting, showing skylines and building tops. NOT just street-level views of urban life 011 Harbor: a body of water with docking facilities for boats and/or ships such as a harbor or marina, including shots of docks. NOT shots of offshore oil rigs, piers that do not look like they belong to a harbor or boat dock 012 Telephone: any kinds of telephone, but more than just a headset must be visible. 013 Street: a regular paved street NOT a highway, dirt road, or special type of road or path 014 Demonstration_Or_Protest: an outdoor, public exhibition of disapproval carried out by multiple people, who may or may not be walking, holding banners or signs 015 Hand: a close-up view of one or more human hands, where the hand is the primary focus of the shot. 016 Mountain: a landmass noticably higher than the surrounding land, higher than a hill, with the slopes visible 017 Nighttime: a shot that takes place outdoors at night. NOT sporting events under lights 018 Boat_Ship: exterior view of a boat or ship in the water, e.g. canoe, rowboat, kayak, hydrofoil, hovercraft, aircraft carrier, submarine, etc. 019 Flower: a plant with flowers in bloom; may just be the flower 020 Singing: one or more people singing - singer(s) visible and audible, solo or accompanied, amateur or professional