IRS13 Evaluation Parameters chapter E. M. Keen Harvard University Gerard Salton Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. 11-26 precision or recall ratios are reached. The use of precision values, although theoretically possible, has not been tested, primarily because recall is more suitable for this, since precision does not monotonically decrease with r[OCRerr]nk (the upward sloping `[OCRerr]steps'1 in Figure [OCRerr] indicate that more than one cut-off can achieve a given precision). Although recall does monotonically increase, there is still one problem that requires solution. The vertical segments of the `stept curve for an individual request (Figure [OCRerr]) show that at some recall points, more than one cut-off point may exist from which to choose, each giving a different precision ratio. At least five possible solutions are available concerning the choice of a cut-off, namely, that having the highest precision, the lowest precision, the precision of the t1middle" document, a precision ratio computed from the average precision over all cut-off points, or a precision ratio computed as the average of the top and bottom points only. Figure 15 indicates an example of each of these possible solutions. There is a further question, relating to the precision values to be used at recall points where no vertical part of the step is encountered, such as at 0.5 recall in Figure [OCRerr]. It is possible, for example, by using one of the five possible points at the vertical segments, to join up the chosen point: on the vertical segment by a new interpolation line. Figure 16 a) shows that when the cut-off having the highest precision is chosen for use at the vertical segments, interpolation between these points of an individual request produces a smooth performance curve, that is quite suitable for averaging Over sets of requests. This example of Figure 16 a) is the one most frequently used by SMART, and the description appeared first in [[OCRerr]]. This type of average curve normally uses ten recall levels, 0.1, 0.2, and so on, and is re[OCRerr]erred to as the ??Quasi[OCRerr]CranfieldIl method. Its advantage is that it can be quite