Activities in Extended Video (ActEV)

Task Coordinators: ActEV NIST team

The Activity Extended Video (ActEV) challenge main focus is on human activity detection in multi-camera video streams. Activity detection has been an active research area in computer vision in recent years. The ability to detect human activities is an important task in computer vision due to its potential in a wide range of applications such as public safety and security, crime prevention, traffic monitoring and control, eldercare/childcare, human-computer interaction, human-robot interaction, smart homes, hospital activity monitoring, and many more. Here, by activity detection, we mean the detection of visual events (people/objects engaged in particular activities) in a large collection of video data.

The ActEV evaluation is being conducted to assess the robustness of automatic activity detection for a multi-camera streaming video environment. The evaluation is based on a portion of the MEVA Known Facility (KF) datasets and run as a self-reported leaderboard evaluation. You can download the public Video dataset for free at mevadata.org; and more info about the datasets is on the "data tab" of the website. The evaluation is based on around 20 activities from the activities listed in the "activities tab".

The TRECVID ActEV 2023 Challenge is based on the Multiview Extended Video with Activities (MEVA) Known Facility (KF) dataset. The large-scale MEVA dataset is designed for activity detection in multi-camera environments. It was created on the Intelligence Advanced Research Projects Activity (IARPA) Deep Intermodal Video Analytics (DIVA) program to support DIVA performers and the broader research community. You can download the public MEVA resources (training video, training annotations and the test set) as described on the website.

Evaluation Type

The TRECVID ActEV 2023 Challenge is a self-reported leaderboard (take-home) evaluation; participants can download a TRECVID ActEV 2023 test dataset on May 15, run their activity detection algorithms on the test set using their own hardware platforms, and then submit their system output to the evaluation server for scoring results.

Metrics

For both the AD and AOD tasks, the submitted results are measured by Probability of Missed Detection (Pmiss) at a Rate of Fixed False Alarm (RateFA) of 0.1 and 0.2 (Pmiss@0.1RFA and Pmiss@0.2RFA). RateFA is the average number of false alarms activity instances per minute. Pmiss is the portion of activity instances where the system did not detect the activity within the required temporal (AD) and spatio-temporal (AOD) overlap requirements. Submitted results are scored for Pmiss and RateFA at multiple thresholds (based on confidence scores produced by the systems), creating a detection error tradeoff (DET) curve.

ActEV website: https://actev.nist.gov/SRL
ActEV email: actev-nist@nist.gov