SP500207 Overview of the First Text REtrieval Conference (TREC-1) chapter D. Harman National Institute of Standards and Technology Donna K. Harman It is useful to contrast the three methods of query construction. Figure 7 shows a comparison of five sets of results, two from automatic query construction, two using manual query construction, and the one relevance feedback run. It should be noted that there is relatively little difference between the results from automatic query construction versus manual query construction, although the relevance feedback results (citym2) were poor in this case. Figure 8 shows a histogram of the same information, but for all adhoc systems working with the full data set. In general it shows that the automatic query construction seems to work well for many systems, and that certainly it can be concluded that for ThEC-l the automatic construction of queries was as effective as the manual construction. Adhoc Manual vs Automatic 0.9 0.8 A 0.7 0.6 0 C,, 0.5 a) [OCRerr] 0.4 0.3 0.2 0.1 0 0.00 0.10 0.20 0.30 0.40 0.60 0.70 0.80 0.90 1.00 0.50 Recall fflhrpl crnlpl cnqst2 clarib citym2 Figure 7. A Comparison of Adhoc Results using Different Query Construction Methods. 0.25 Adlioc -- All Category A Systems [OCRerr]> 0.15 .[OCRerr] 0.1 0 0.05 "/X/;;/',;[OCRerr] 0 Ilihi U Automatic U Manual Figure 8. A Comparison of Adhoc Results using Automatic and Manual Query Construction. 15