IRS13
Scientific Report No. IRS-13 Information Storage and Retrieval
Thesaurus, Phrase and Hierarchy Dictionaries
chapter
E. M. Keen
Harvard University
Gerard Salton
Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government.
VII-14
matching functions. These again display a superiority for thesaurus in
the expected cases, except for the "ADI Text Thesaurus-i Overlap Logical"
result.
Precision versus recall graphs are used to repeat the most impor-
tant comparisons, with IRE-3 in Fig. 8, Cran-l Abstracts in Figs. 9 and 10,
Cran-l Indexing in Fig. 11, and ADI Abstracts and Text in Fig. 12. Thes-
aurus works better than stem in all cases except in that of the first
Cranfield version. Fig. 10 compares thesaurus with suffix `S', since
on Cran-l, stem is not as good as suffix `5' (both at the high precision
end and between 0.65 and 0.85 recall suffix `5' is superior). The
figures do show that the thesaurus dictionaries are superior by a much
greater amount for both IRE-3 and ADI than for Cran-l.
Using all comparisons of stem with the final versions of a given
thesaurus, the data in Fig. 13 shows how the individual requests favor
each dictionary using the normalized measures as an indicator of merit.
Between 54% and 82% of the requests favor the thesaurus, with IRE-3
showing the clearest advantage for the thesaurus, ADI next, and Cran-1
the least advantage; this agrees with the precision-recall graphs.
Fig. 14 gives a further performance comparison using the average
rank position of the first, second and last ranked relevant documents,
to simulate high precision and high recall needs. Unexpectedly, the
Cran-l and ADI comparisons show the thesaurus to be more effective in
meeting the high precision needs than stem, whereas in IRE-3, the
thesaurus worsens high precision performance where one relevant only
is required. However, high recall needs are seen to be dramatically