Streaming Multimedia Knowledge Base Population (Pilot Evaluation)
System task
Given streams of text, speech, images, video, and their associated metadata, from a variety of genres, both formal (e.g.,news) and informal (e.g., social media, blogs), systems are asked to analyze each incoming information item (~100k items in total) and produce a set of structured representations (knowledge elements) about events, sub-events or actions, entities, relations, locations, time, and sentiments (beliefs) that are observable in that information item (all of them, not just the most confident!) given an ontology/schema and zero or more background (context) information.
More detailed guidelines about the task evaluation, metrics and data and schedule will be available soon on the Text Analysis Conference (TAC) website.
Evaluation Plan (version 0.1)
Please use your active participants username and password to access the
evaluation plan document
This task in 2018 is considered pilot (dry run). The only purpose for running it is to test the evaluation pipeline and for teams to get familiar with the data and the requirements but not to test systems' performance.
Development and Testing Data
- Please fill, sign and submit your TRECVID dissimenation of results form before requesting the data from LDC.
- Please fill, sign and submit your data License agreement located here to LDC. Please note that in addition to the core development and testing data distributed by LDC, there is an optional data resources (mainly textual data and not imaging/video) on the data greement form for you to select as well if you feel it is needed to complete your research work on the task.