SP500207 Overview of the First Text REtrieval Conference (TREC-1) chapter D. Harman National Institute of Standards and Technology Donna K. Harman There were also some category B results for routing, and the best of these are shown in Figure 14. Again there is a much wider spread in the curves here, and widely differing systems being shown. The "pircsl" and "pircs2" results are correspondingly automatic and manually constructed queries using relevance feedback learn- ing from the training sample (see Kwok, Padadopoulos & Kwan paper). The "fairsi't system uses a combination of different term weighting methods (see Chang, Dediu, Assam & Du paper). The "nyuirl" results come from a system using natural languages techniques and probably reflect the short amount of time available to construct this very complicated system (see Strzalkowski paper). 0.9 0.8 0.7 0.6 0 0.5 C,) 0.4 0.3 0.2 0.1 0 Routing -- Category B 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 Recall pircs2 piresi fairsi nyuirl Figure 14. Routing Results for Category B. SA Summary The TREC- 1 conference demonstrated a wide range of different approaches to the retrieval of text from large document collections. Because of the preliminary nature of the results, very little can be said about which tech- niques seem to perform the best. It was clear that the simple systems did the task well, but it is too early to pass judgment on the more complicated systems. The automatic construction of queries from the topics did as well as, or better than, manual construction of queries, and this is encouraging for groups supporting the use of simple natural language interfaces for retrieval systems. There will be a second TREC conference in 1993, and all the Systems that participated in TREC- 1 will be back, along with additional groups. The results from this second conference should better identify the more promising retrieval techniques. REFERENCES Cleverdon C.W., (1962). Report on the Testing and Analysis of an Investigation into the Comparative Efficiency of Indexing Systems. College of Aeronautics, Cranfield, England, 1962. 19