ISR11 Operating Instructions for the SMART Text Processing and Document Retrieval System chapter M. E. Lesk Harvard University Gerard Salton Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. 11-55 requests and methods. [OCRerr]RVAL accepts as input a set of decks of rank positions p[OCRerr]mched by the SCORES specification at run time. The program produces for each method and each request the top fifteen documents, and the ranks of the relevant documents. All documents are abbreviated to three-character identifiers to permit a compact output format. Thus, to use [OCRerr]RVAL properly the twelve-character identifiers used for the documents should be chosen so that the first three of these characters are adequate to uniquely specify a document. [OCRerr]RVAL also produces normalized and unnormalized evaluation measures, and recall-precision graphs of two types (averaged over requests, and cumulated[OCRerr]type graphs). [OCRerr]RVAL is also capable of evaluating `tmerged methods??, where the results of several methods are merged. [OCRerr]RVAL is described in reference [15], where c[OCRerr]nplete instructions for using the program are given. 6.3. SOCCER SOCCER is a concordance generating program. All occurrences of each word in an input document collection can be listed with their context. Such a concordance greatly simplifies maziy tasks of word use analysis. SOCCER contains full facilities for suppressing concordances on unwanted words; or restricting the concordance to selected words. It also provides a variety of statistics about the collection of text processed. Because of the simpler nature of the processing, SOCCER will function with virtually any input format for the text. In particular, the SM[OCRerr]T formats are satisfactory, and are convenient for some of the format options of SOCCER. SOCCER is a relatively fast program; a concordance prepared for a