Incompleteness
Relatively new concern regarding test collection quality
- early test collections were small enough to have complete judgments
- current collections can have only a small portion examined for relevance for each query; portion judged is usually selected by pooling