Evaluation Completed for DUC
Metric: Precision of labels given to folders
- For top one
- For top two
- For top five
Three Methods Evaluated
- “UTD”
- An algorithm to automatically identify and name topics from unsupervised training (NO human annotation)
- “PSM”-based
- OnTopic topic classifier
- Human annotation using catalog of 5,000 news labels from Primary Source Media as training
- “Tf.idf”
- Select labels from words with highest tf.idf in the folder