SP500207 OCLC Online Computer Library Center, Inc. chapter R. Thompson National Institute of Standards and Technology Donna K. Harman Most AP of OF these DTS homocides NNS VBZ NNS homoc ides have HV been BEN related VBD VBN VBD to RI TO the AT city's NN$ burgeoning VBG drug NN VB NN drug trade NN VB NN trade In previously reported work on FASIT (Dillon and Gray 1983, Dillon and McDonald 1983), additional processing was done on the extracted concepts. Nouns and adjectives which might be judged not to be indicative of the document's content, such as "current account" or "little flexibility", and might therefore be expected to reduce precision, were filtered out. Secondly, the earlier work reported an algorithm which might increase precision by performing a rudimentary degree of phrase clustering. Phrases such as "life insurance" and "life insurance policy" were collected to form a type of thesaurus entry because they share many stems. Both improvements were eliminated from the present study because of our interest in establishing a baseline performance for FASIT. However, both are active areas of research. Data Preparation All data is processed automatically; there is no hand-guiding. Queries are in the narrative-concept format. Queries and documents are represented as FASIT output. In addition, the component words of the phrases identified by FASIT are represented as terms. All words and phrases are stemmed. Because we are Plan B participants, we are working with the 350M subset of the Wall Street Journal data. A SMART database with ATC weighting was built from the FASIT output and submitted to the TREC sponsors for evaluation. We have also performed extensive testing and failure analysis of a 46449-record subset of the TREC training database which consists of all Wall Street Journal articles from 1987. We chose this subset because all of the documents relevant to the training queries for the Plan B participants were drawn from the 1987 subset. A baseline database for these experiments was created using SMART, ATC weighting and stem indexing. Results The results reported here are from the 1987 subset of the Wall Street Journal database because the results of the full database were not available at the time of this writing. Table II shows precision scores for eleven levels of recall in the baseline database; Table III shows results for a test database constructed with FASIT processing of queries and documents. Table II Precision Results for Eleven Levels of Recall with Stem Indexing 190