ISR11 A Modified Two-Level Search Algorithm Using Request Clustering chapter V. R. Lesser Harvard University Gerard Salton Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. v'I-27 Therefore, approximately 70[OCRerr] of the requests consist of queries which are similar to the set of previous queries introduced into the system. The cases 13-18 indicate that for this type of request distribution the modified two-level search scheme is still more effective then the normal two-level search. In a real user population, 3 much higher percentage of requests should be similar to the previous requests so that the modified two-level search scheme should be significantly better than the normal two- level search. 6. A New Criterion for Search Effectiveness It is felt that the use of the three criteria to evaluate search effectiveness is not completely adequate since there is no intuitive proce&[OCRerr]e to combine the values for each criterion into one composite score which represents overall search effectiveness. This ina[OCRerr]equacy in using the three criteria for search effectiveness is demonstrated by the difficulty encountered in evaluating the test results in this experi- ment. The follo[OCRerr]:Tinb- is a procedure to calcul[OCRerr]e the value of a single overall criterion for search effectiveness: for a given test query end for a given search scheme a total ranking is induced on the documents contained in the test document collection; the first document retrieved by the given search scheme for the given tcst query is ranked number 1, th end likewise the n document retrieved is ranked number n. This procedure is continued until all relevent documents are retrieved. The ranking of the documents relevant to the given test query is determined, and the evaluation measure 1tranked recall'1 [6] is calculated. The average ranked recall for all test queries is then taken as indicative of search