IRS13 Thesaurus, Phrase and Hierarchy Dictionaries chapter E. M. Keen Harvard University Gerard Salton Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. vII-22 improved by the IRE-3 thesaurus, and hardly significantly improved over stem on Cran-l and ADI. Data on individual request preferences based on this average rank evaluation is given in Fig. 15. It is noteworthy that the rank posi- tion of the first relevant is unchanged by the use of a thesaurus in over one quarter of the requests. This is most strong[OCRerr] seen in the IRE-3 re- sult, which shows that the drop in average rank of the first relevant with the thesaurus is caused by only very few requests being inferior to stem. The only small reversal of merit in Fig. 15 is the Cran-l indexing result using the average rank of the last relevant, where it is seen that on an individual request basis, stem has a slight edge over thesaurus. The use of mean rank position as in Fig. 14, is not very well suited to some of the data presented. For example, the median rank position of the first relevant document is nearly always one, so addi- tional data on the rank position of the first relevant is given in Fig. 16. Here it may be seen that the thesaurus dictionaries all produce results for which two to six more of the requests have their f£rst relevant in rank positions one or two; in the Cran-l and ADI collections, the number of requests having the first relevant ranked later than ten is also reduced by the thesaurus. The results in Figs. 6 and 7 which were based on matching functions other than cosine numeric are not presented in the form of complete precision recall graphs, but a simplified table giving the merit at three positions on the precision-recall curves appears in Fig. 17. In general, the merit is the same as that seen for the normalized measures: the cases where stem performs better than the thesaurus are of interest