TREC Collections Are Viable Tools
Relative effectiveness is stable despite marked differences in relevance sets
- Authors vs. non-authors
- Different non-authors
- Single judge vs. combination of opinions
- Same task vs. different task
Algorithmic variants of single retrieval system are particularly stable