MONO91 Other Potentially Related Research chapter Mary Elizabeth Stevens National Bureau of Standards physics, then transformed the 50 questions into search prescriptions using three different methods. The first method for the development of the search instructions was to choose appropriate index entries from a subject heading list tailored to the contents of the sample library. Search was then made manually against a card catalog which recorded the results of manual indexing of the same 100 articles to the entries of this list. The second method of search prescription tested involved the specification of combinations of words and phrases likely to be found in any paper which would in fact be relevant to the search question. The third method involved modification of the second by the use of a thesaurus-type glossary which suggested various alternative terms. Both the latter two types of search instructions were fed to a computer program which carried out searches against the natural language text consisting of 250, 000 words from the original articles. The results were then evaluated in terms of ratings of relevance made by the physicists who had analyzed the papers. Retrieval effectiveness was not high: [OCRerr]. . in no case did the average amount of relevant material . . . retrieval (taken over 50 questions) exceed 42 per cent of that which was judged . .. to be present in the library II 1/ However, the results were indicative of the superiority of the machine methods to the manual cata- log search.2/For this library in particular, in the case of `source documents'1 (the articles from which the search questions were taken), only 38 percent of the relevant papers were located by the manual search, whereas 68 percent of the relevant items were retrieved by machine search of the text for specified words and phrases in various TiandI! and 11or1 combinations. Machine search based on search instructions that had been developed with the assistance of the thesaurus-glossary yielded 86 percent of the relevant source item documents. 6.4.3 Full Text Searching - Legal Literature "The retriever of documents may be satisfied with a sample of descriptors that represent the contents; the fact retriever or the question answerer must often have access to every word in the text". 3/ The objective of fact retrieval is a major goal in the experimentation that is being carried forward in the field of natural language text searching of legal material, especially the texts of statutes of the State and Federal Governments. The most extensive program to date is that of Horty and his colleagues at the University of Pittsburgh Health Law Center (1960 [277], 1961 [276, 309], 1962 [196, 278], 1963 [24, 280]). Wilson at the Southwestern Legal Foundation is experimenting with a modified version of the Horty-Pittsburgh System for legal cases dealing with arbitration in five of 1/ 2/ 3/ Swanson, 1960 [582], p. 25. Ibid, p.1: "On the whole, retrieval effectiveness was rather poor, yet machine search of the text of the model library was significantly better than was human searching of the subject heading index." Simmons and McConlogue, 1962[555], p. 3. 135