SP500207 Overview of the First Text REtrieval Conference (TREC-1) chapter D. Harman National Institute of Standards and Technology Donna K. Harman Adhoc -- Category B 0.9 0.8 0 0.5 [OCRerr] 0.4 0.3 0.2 0.1 0 0.00 0[OCRerr]1O 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 Recall crnlB pircsl pircs4 nyuir[OCRerr] Figure 10. Adhoc Results for Category B. 5[OCRerr] Routing Results There were 22 sets of results for routing evaluation, with 16 of them based on runs for the full data seL Note that all routing techniques suffered from the lack of sufficient and accurate training data, and therefore these results are even more preliminary than the adhoc results. Of the 16 systems using the full data set, 8 used automatic construction of queries, and 8 used manual construction. Figure 11 shows the recall/precision curve for the three TREC-1 runs with the highest 11-point averages using automatic construction of queries. Two of the curves, based on the use of the Cornell SMART system, show very different results. The "fuhra2" results came from using a probabilistically-based relevance feedback (see Fuhr & Buckley paper), whereas the "crnla2" results came from doing traditional relevance feedback methods using the vector space model (see Buckley, Sal- ton & Allan paper), The "cityri" results also came from using traditional relevance feedback, but using a different probabilistic model and term weighting (see Robertson, Walker, Hancock-Beaulieu, Gull & Lau paper). The "cpgcn2" system used filtering methods rather than more traditional information retrieval methods to achieve results similar to the feedback results (see Jones, leung, and Pape paper). 0.9 0.8 0.7 0.6 0 0.5 C,) [OCRerr] 0.4 0.3 0.2 0.1 Best Automatic Routing 0 -[OCRerr] 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 Recall __ fuhra2 cnil[OCRerr] cityri cpgcn2 Figure 11. The Best Routing Results using Automatic Query Construction. 17