IRS13 Evaluation Parameters chapter E. M. Keen Harvard University Gerard Salton Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. 11-63 measures provide the same merit between specific and general requests as that shown by the fallout versus recall plot. This means that the normalized measures tend to reflect the merit experienced by a high recall user. Some of the comparisons made in Section X (see part 5c) are thus seen to be recall oriented. 9. The Presentation of Data as Individual Request Merit Evaluation using averaged measures always suppresses some data, and when the variance of individual requests is large, the arithmetic mean may be a poor measure of merit. For this reason, presentation of results using averages in section III, V, VI, and VII is followed in each case by data on the individual requests. For example, if normalized recall averaged over 12 requests shows one option to be quite superior to another, individual request examination might reveal that 6 of the requests favored the option that was superior or average, 4 favored the other option, and 2 had an identical performance on both options. The tables used to present this type of data usually give both the numbers of requests favoring each option together with those equal on both options; in addition, percentages are produced to aid speedy interpretation. Several ways of computing the percentages are possible, and six meth[OCRerr]ds are illustrated in Fig. 35. The technique adopted for the sections listed is nearly always that of giving percentages for each option ignoring the commonly few cases where both options have equal merit. Percentages including the equal cases are needed when the number of such cases is large, and may be given either in the form of row 3 or 4. The "Superiority Percentage" has the advantage of a single number presentation.