SP500207 Retrieval Experiments with a Large Collection using PIRCS chapter K. Kwok L. Papadopoulos K. Kwan National Institute of Standards and Technology Donna K. Harman C. proximity operators d. addition of terms not included in topic (1) source of terms e. other (describe) C. Feedback (ad-hoc) 1. initial query built by method 1 or method 2? 2. type of person doing feedback a. domain expert NO b. system expert YES 3. average time to do complete feedback a. cpu time (total cpu seconds for all iterations) 12 PER QUERY PER IThR[OCRerr]ON - NO EXPANSION. 50 " " `I - [OCRerr] EXPANSION. NO YES WORD-PAIR PHRASE FILE. METhOD 1 b. clock time from initial construction of query to completion of final query (minutes) 60 PER QUERY TO DO RELEVANCE JUDGMENT. 3. average number of iterations 1 a. average number of documents examined per iteration 10 4. minimum number of iterations 5. maximum number of iterations 6. what determines the end of an iteration? 6. feedback methods used a. automatic term reweighting from relevant documents 1 1 DEADLINE + LACK OF MANPOWER YES b. automatic query expansion from relevant documents (1) all terms in relevant documents added NO (2) only top X terms added (what is X) TOP 20 MOST `A[OCRerr]VAThD' ThRMS ThAT HAVE DOCUMENT FREQUENCY <2000 WERE USED. BECAUSE MANY WERE ALREADY IN QUERY, ABOUT 12 ON THE AVERAGE WERE NEW AND ADDED PER QUERY. (3) user selected terms added NO c. other automatic methods brief description FEEDBACK IS BASED ON SUB-DOCUMENTS. NO d. manual methods (1) using individual judgment with no set algorithm (2) following a given algorithm (brief description) D. Automatically built queries (routing) 1. topic fields used 2. total computer time to build query (cpu seconds) 3. which of the following were used in building the query? a. terms selected from (1) topic YES (2) all training documents NO (3) only documents with relevance judgments NO 169 [OCRerr]ThE>, <DESC>, <NARR>, <CON> 5 (AVERAGE FOR EACH QUERY).