Activities in Extended Video (ActEV)

Task Coordinators: ActEV NIST team

ActEV is a series of evaluations to accelerate development of robust, multi-camera, automatic activity detection algorithms for forensic and real-time alerting applications. ActEV is an extension of the annual TRECVID Surveillance Event Detection (SED) evaluation where systems will also detect, and track objects involved in the activities. Each evaluation will challenge systems with new data, system requirements, and/or new activities. Recently we ran will two challenges, the ActEV Sequestered Data Leaderboard based on the MEVA dataset, and the self-reported ActEV TRECVID 2019 leaderboard based on the VIRAT dataset.

What is Activity Detection in Videos?

For this evaluation, an activity is defined to be “one or more people performing a specified movement or interacting with an object or group of objects”.

Who: NIST invites all organizations, particularly universities and corporations, to submit their results using their technologies to the ActEV evaluation server. The evaluation is open worldwide. Participation is free. NIST does not provide funds to participants.

How: To take part in the ActEV evaluation you need to create an account on the ActEV scoring server (Please visit the ActEV task website). Using a valid registered account, you will be able to upload your JSON format results (see the ActEV task website for: Evaluation Plan) to the ActEV scoring server and participate in the ActEV evaluation. Each team is limited to a fixed number of submissions (see website for: Evaluation Plan). Only results that are submitted during the evaluation period and follow the Evaluation Plan will be considered valid and posted on the leaderboard website.

Why: The primary driver of the evaluation is to support investigations of crime by automatic activity detection in streaming video. These videos are of interest to NIST's partner agencies that seek to employ automatic activity detection in investigating of crime. ActEV evaluation seeks to develop robust automatic activity detection for a multi-camera streaming video environment. Activities will be enriched by person and object detection. ActEV will address activity detection for both forensic applications and for real-time alerting.

Data

The TrecVID ActEV 2020 evaluation will be based on the VIRAT V1 and V2 dataset with 38 activities. The TrecVID ActEV 2019 was based on VIRAT V1 and V2 dataset with 18 activities. The Multiview Extended Video with Activities (MEVA) will be used in a future evaluation, but we would like the participants to help us annotate the data for the 38 activities as defines in the annotation guide (coming soon) . The data will be provided in MPEG-4 and AVI format. Please click on the data tab for information on how to download the data. ActEV task website for: Licensing and Participation Agreements, and Evaluation Plan. For provided development and testing data please see the ActEV task website. please check the ActEV task website for: Licensing and Participation Agreements, and Evaluation Plan. For provided development and testing data please see the ActEV task website.

Metrics

The main scoring metrics will be based on detection, temporal localization using evaluation measures that include the Probability of Missed Detection and Time-Based False Alarm” (TFA) measure. See details in the evaluation plans. NIST maintains the ActEV Scoring Software on the Scoring software for the Activities in Extended Video (ActEV) evaluation GitHub repo. (e.g., Probability of Missed Detection, Time-Based False Alarm” (TFA)) and self-reported speed (see ActEV website for: Evaluation Plan and ActEV Scoring Software: https://github.com/usnistgov/ActEV_Scorer)

NIST Role:

NIST will be conducting a series of evaluations to accelerate development of robust, multi-camera, automatic activity detection algorithms for forensic and real-time alerting applications. Each evaluation will challenge systems with new data, system requirements, and/or new activities.

ActEV website: actev.nist.gov and actev.nist.gov/trecvid20
ActEV email: actev-nist@nist.gov