IRS13 Thesaurus, Phrase and Hierarchy Dictionaries chapter E. M. Keen Harvard University Gerard Salton Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. VII-31 thesaurus alone. The use of "Parents" and "All Relations" does, however, give some performance increase over the thesaurus. Previous tests on the same hierarchy using the IRE-2 collection with 17 requests showed the hierarchy to be always inferior to the thesaurus, (two of the results appear in f16)). Precision versus recall graphs are to be seen in Figs. 21, 22, and 23. The phrase results in Fig. 21 agree with the results based on the normalized measures; in the case of IRE-3 and ADI, the phrase superiority is very small indeed. The hierarchy results in Figs. 22 and 23 reveal that the "Parents" and "All Relations" results are superior to the thesaurus over portions of the curve only. Individual request data based on the normalized measures are given for three of the phrase results and the two best hierarchy results in Fig. 24. Nearly 70% of the requests favor phrases on IRE-3, 58% favor phrases on ADI, and between 60% and 70% of the requests on Cran-l favor the thesaurus rather than phrases. The hierarchy results show that the requests equally favor the "Parents" relation, and, unexpectedly, 62% to 67% of the requests favor the thesaurus rather than the "All Relations". This last result completely re- verses the picture previously presented, and reveals that for this hierarchy option, a few requests which do very well, cause the averages to favor the hierarchy. A comparison of merit in Fig. 25, makes use of the average rank of the first, second and last relevant documents, and the results follow the expected pattern. The average rank improvement with phrases on IRE-3 is seen to be quite small for the high precision user, however. The very large improvements in average rank of the first relevant document with both the hierarchy options is caused by only one or two requests. Fig. 26 gives