ISR11 Operating Instructions for the SMART Text Processing and Document Retrieval System chapter M. E. Lesk Harvard University Gerard Salton Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. i'-i8 The specifications which control this process are: DoDERD a the correlation mode used for the request-document correlation is taken as the cosine mode or overlap mode, according as a is COS" or "OVLAP" respectively. Cosine is the noz[OCRerr]ial mode; CUTRI) x the cutoff for request-document correlation is taken as x, a number between 0.0 and 1.0. Normal setting is 0.35; ANSWER a SM[OCRerr]T can print out the answers to requests in three formats. If a is "S[OCRerr]RT" twelve-char[OCRerr]cter identifiers are used for both requests and documents. If a is "MEDIUM", the full text of the request is used, and one line is used for the document identifier. If a is 1'WNG" the complete request and the complete document citations are printed for each answer. These formats assume, of course, that the necessary information is supplied with each document at read-in time ([OCRerr].l); SCORES this specification causes the automatic evaluation procedure to be performed. A list of relevant documents must be provided, according to the format described in [OCRerr] The output provided by the SOORES specification includes the following: a) for each request, the fifteen documents with the highest correlations, and the values of those correlations; b) for each request, the relevant documents, their ranks in the correlation list, and their correlations; c) for each request, the normalized and un-normalized recall and precision measures; d) the averages of the recall and precision measures over all of the requests used in this computer run;