SP500215 Description of the PRC CEO Algorithm for TREC-2 chapter P. Thompson National Institute of Standards and Technology D. K. Harman 3. Conclusion Although selecting the best individual TREC- 1 VPI&SU retrieval methods for combination and weighting them by their TREC- 1 performance seemed to be a reasonable strategy which would yield better retrieval results than unweighted combination of all methods, in fact CEO performance was worse on TREC-2. This was due, in part, to changes made in the individual methods which made methods that had been best in TREC- 1 less effective than other methods for TREC-2. These results suggest that selection and weighting of methods for combination based on performance of earlier versions of the methods is unwarranted. References Fox, E.A.; Koushik, P.; Shaw, 3.; Modlin, R.; Rao, D. 1993. "Combining Evidence from Multiple Searches" this proceedings. Fox, E.A.; Nunn, G.L.; Lee, W.C. 1988. "Coefficients for Combining Concept Classes in a Collection" Proceedings of the 11th. International Conference on Research and Development in Information Retrieval, June 13-15, Grenoble, France, p.291-307. Katzer, J.; McGill, M.J.; Tessier, J.A.; Frakes, w.; DasGupta, P.1982. "A study of the overlap among document representations" Information Technology: Research and Development, vol. 2, p. 261-274. Lindley, D.V. 1983. "Reconciliation of probability distributions" Operations Research, v.31, no.5, p.866-880. Thompson, P. 1990a. "A Combination of Expert Opinion Approach to Probabilistic Information Retrieval, Part 1: The 274 Conceptual Model" Information Processing and Management, vol.26, no.3, p. 371-382. Thompson, P. 1990b. "A Combination of Expert Opinion Approach to Probabilistic Information Retrieval, Part 2: Mathematical Treatment of CEO Model 3" Information Processing and Management, vol.26, no.3, p. 383-394. Thompson, P. 1991. "Machine Learning in the Combination of Expert Opinion Approach to IR" In Birnbaum, L. and Collins, G. (eds.) Machine Learning: Proceedings of the Eighth International Workshop (ML91), San Mateo: Morgan Kaufmann, p.270-274.