ISR11 Design Criteria for Automatic Information Systems chapter M. E. Lesk G. Salton Harvard University Gerard Salton Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. V-33 under altered conditions. The most effective procedure tried so far is the "relevance feedback" process, in which the user returns to the system a list of document numbers previously retrieved, together with information concerning the usefulness of each document for his search purpose. The system then automatically adjusts the original query by increasing the weight of query terms originally contained in documents identified as relevant, and simultaneously. decreasing the weight of query terms contained in the nonrelevant document set. This process can, of course, be repeated several times, and results each time in a modification of the query in the ttdirection" of the document set termed relevant, and away from the document set termed nonrelevant. The results of two iterations performed with 2[OCRerr] search requests processed against the IRE - 2 collection are shown in Fig. 15. The first step of query modification is seen to result in a large-scale improvement in retrieval effectiveness, while the second iteration provides a smaller, but still pronounced increase in effectiveness. Realistic tests of iterative search techniques can only be made in a real-time environment with adequate time-sharing equipment. The initial tests performed so far do, however, suggest the following rule: Rule 9 : Iterative search techniques, based on feedback information supplied by the user as a result of previous retrieval procedures, appear to offer major promise for more effective search operations. H) Summary The principal conclusions resulting from the tests conducted with the SMART system are sunmiarized in Fig. 16. These results suggest that