IRS13 Evaluation Parameters chapter E. M. Keen Harvard University Gerard Salton Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. 11-59 high recall need is defined to mean that a full 1.0 recall is required, then the best performance will be achieved when perfect recall is quickly reached and has high precision, so that in Fig. 31 cases A, D, and B are superior to C, F, and E. Making the further distinction that A, B, and C are specific and D, E, and F are general requests, this hypothetical example shows that the high precision user is served best on the average by the general requests, and the high recall user by the specific requests. The cases in Fig. 31 are chosen to be typical of the results obtained in the Cran-l request sets being used, and full discussion of these results appears in section I part 6B. One method of presenting average results that reflects the success achieved in meeting the two different types of user need is given in Fig. 32. The high precision and high recall needs are based on the definitions given in the previous paragraph. An average rank position is thus calculated for the first and second ranked relevant documents (for a high precision merit), and for the last ranked relevant document (for a high recall merit). It can now be concluded that the high precision user is served best by the general requests, and the high recall user by the specific requests. However, the computation of the arithmetic mean rank is sometimes a poor representation of the data since the variance can be large and one or two very bad requests can unduly influence the average. Some type of histogram would solve this problem, but at the cost of a somewhat more com- plex presentation. One compromise solution is suggested by Fig. 33, where data on the rank of the first relevant is re-arranged to show the numbers of search requests that gave a given rank (in three ranges) to the first relevant.