Retrieval Group Papers

Retrieval Group Published Papers

	[by author]
	Most of these documents are available in electronic format below. If a paper copy is required, please send your request to the Retrieval Group Manager.
2005	Soboroff, I., & Harman, D., (October 2005). Novelty Detection: The TREC Experience. In Proceedings of HLT/EMNLP 2005, Vancouver, Canada, pp. 105-112.
2004	Soboroff, I., (July 2004). On Evaluating Web Search With Very Few Relevant Documents. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Sheffield, UK., pp. 530-531.
2003	Soboroff, I., Voorhees, E., & Craswell, N., (September 2003). Summary of the SIGIR 2003 Workshop on Defining Evaluation Methodologies for Terabyte-scale Test Collections. ACM SIGIR Forum, 37(2), ACM Press. Soboroff, I.. & Robertson, S., (July 2003). Building a Filtering Test Collection for TREC 2002. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Toronto, Canada, pp. 243-250. Alemayehu, N., (2003). Analysis of Performance Variation Using Query Expansion. Journal of American Society for Information Science and Technology 54(5):379-391.
2002	Soboroff, I., (September 2002). Do TREC Web Collections Look Like the Web? In ACM SIGIR Forum, 36(2), ACM Press. Chowdhury, A., & Soboroff, I., (August 2002). Automatic Evaluation of World Wide Web Search Services. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Tampere, Finland, pp. 421-422. Soboroff, I., (August 2002). Does WT10g Look Like the Web? In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Tampere, Finland, Voorhees, E. (July 2002) *Whither Music IR Evaluation Infrastructure: Lessons to be Learned from TREC, to appear in Proceedings from Workshop on the Creation of Standardized Test Collections, Tasks, and Metrics for Music Information Retrieval (MIR) and Music Digital Library (MDL) Evaluation, Portland, OR, USA. Voorhees, E. (2002) The Philosophy of Information Retrieval Evaluation, To appear in Proceedings of the 2nd Workshop of the Cross-Language Evaluation Forum, CLEF 2001, Darmstadt, Germany. Voorhees, E. (May, 2002) The Evaluation of Question Answering Systems: Lessons Learned from the TREC QA Track*, Proceedings of the LREC 2002 Workshop on Question Answering --- Strategy and Resources, Las Palmas de Gran Canaria, Spain, pp. 1--4.
2001	Voorhees, E. (Nov. 2001) Question Answering in TREC In Proceedings of the Tenth International Conference on Information and Knowledge Management, Atlanta, GA, USA, pp. 535-537 Voorhees, E. (Sept. 2001) Evaluation by Highly Relevant Documents. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, New Orleans, LA, USA, pp. 74-82. Voorhees, E. (2001) The TREC Question Answering Track, Natural Language Engineering, volume 7, number 4, pages 361-378.
2000	Buckley, C. & Voorhees, E. (2000, July). *Evaluating Evaluation Measure Stability.* In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Athens, Greece, pp. 33-40. Voorhees, E. & Tice, D. (2000, July). *Building a Question Answering Test Collection.* In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Athens, Greece, pp. 200-207. Voorhees, E. & Tice, D. (2000, May/June). The TREC-8 Question Answering Track. In proceedings of Language Resources and Evaluation Conference, Athens Greece. Voorhees, E. & Tice, D. (2000, May). Implementing a Question Answering Evaluation. In proceedings of "Using Evaluation Within HLT Programs: Results and Trends", a workshop of the Second International Conference on Language Resources and Evaluation, Athens, Greece. Kantor, P. & Voorhees, E. (2000) The TREC-5 Confusion Track: Comparing Retrieval Methods for Scanned Text. Information Retrieval, 2(2/3), 165-176. Voorhees, E. & Harman, D. (2000). Overview of the Sixth Text REtrieval Conference. Information Processing and Management, 36(1), 3-35.
1999	Voorhees, E. (1999). Natural Language Processing and Information Retrieval. In M. T. Pazienza, (Ed.), Information Extraction: Towards Scalable, Adaptable Systems (pp.32-48). Germany: Springer. (Entry in lecture notes in artificial intelligence 1714) Schmidt, C. & Over, P. (1999, August). Digital Video Test Collection. In proceedings of the Twenty-Second Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Berkeley, California, USA. Schmidt, C. & Harman, D. (1999, August). Information Retrieval Library (IRLIB). In proceedings of the Twenty-Second Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Berkeley, California, USA. Tebbutt, J. (1999). User Evaluation of Automatically Generated Semantic Hypertext Links in a Heavily Used Procedural Manual. Information Processing and Management, 35(1), 1-18. Downey, L. & Tice, D. (1999). A Usability Case Study Using TREC and ZPRISE. Information Processing and Management, 35(5), 589-603.
1998	Lagergren, E., & Over, P. (1998, August). *Comparing Interactive Information Retrieval Systems Across Sites: the TREC-6 Interactive Track Matrix Experiment.* In proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Melbourne, Australia. Voorhees, E. (1998, August). *Variations in Relevance Judgements in the Measurement of Retrieval Effectiveness.* In proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Melbourne, Australia. Towell, G. & Voorhees, E. (1998). Disambiguating Highly Ambiguous Text. Computational Linguistics, 24(1), 125-145. Harman, D. (1998, June). *The Text REtrieval Conferences (TRECs) and the Cross-Language Track.* In proceedings of the First International Conference on Language Resources and Evaluation, Granada, Spain. Tebbutt, J. (1998, June). *Finding Links.* In proceedings of the Ninth ACM Conference on Hypertext and Hypermedia, Pittsburgh, Pennsylvania, USA. Dimmick, D., O'Brien, G., Over, P., & Rogers, W. (1998). *Guide to Z39.50/Prise 2.0: Its Installation, Use, & Modification.* Gaithersburg, Maryland, USA: unpublished resource. Harman, D. (1998). The Text Retrieval Conferences. Bulletin of the American Society for Information Retrieval, vol#(iss#), pps. Garofolo, J., Voorhees, E., Stanford, V., & Sparck Jones, K. (1998, February). *TREC-6 1997 Spoken Document Retrieval Task Overview and Results.* In proceedings of the DARPA Broadcast News Transcription and Understanding Workshop, Lansdowne, Virginia, USA. Voorhees, E. (1998). Using WordNet for Text Retrieval. In C. Fellbaum, (Ed.), WordNet: An Electronic Lexical Database (pp.285-303). Cambridge, Massachusetts, USA: The MIT Press.
1997	Garris, M., Omidvar, O., Blue, J., Candela, G., Dimmick, D., Geist, J., Grother, P., Janet, S., & Wilson, C. (1997). Design of a Handprint Recognition System. Journal of Electronic Imaging, 6(2), 231-243. B. Fordham, (1997, August). Evolving Databases: An Application to Electronic Commerce. In proceedings of the International Database Engineering and Applications Symposium 1997, Montreal, Canada. Harman, D. (1997). Information Retrieval. In Encyclopedia of Computer Science, Fourth Edition (pp. xx-xx). City, State, Country: Publishing Company Name. Hoffman, D., & Downey, L. (1997, June). *Lessons Learned in an Informal Usability Study.* In proceedings of the 20th International ACM SIGIR Conference on Research & Development in Information Retrieval, Philadelphia, Pennsylvania, USA. Voorhees, E., & Tong, R. (1997, June). *Multiple Search Engines in Database Merging.* In proceedings of the 2nd ACM International Conference on Digital Libraries, Philadelphia, Pennsylvania, USA. Voorhees, E., & Harman, D. (1997, November). *Overview of the Sixth Text REtrieval Conference (TREC-6).* In proceedings of the Sixth Text REtrieval Conference (TREC-6), Gaithersburg, Maryland, USA. Harman, D. (1997). The SMART Lab Report -- The Early Cornell Years. SIGIR Forum, 31(1), 6-12. Downey, L., Laskowski, S., Buie, E., Hefley, W. (1997). Symposium Report - Usabilty Engineering 2: Measurement and Methods. Special Interest Group on Computer & Human Interaction (SIGCHI) Bulletin, vol#(iss#), pps. Voorhees, E., Garofolo, J., & Sparck Jones, K. (1997, February). *The TREC-6 Spoken Document Retrieval Track.* In proceedings of the 1997 DARPA Speech Recognition Workshop, Chantilly, Virginia, USA. Garofolo, J., Voorhees, E., Stanford, V., & Sparck Jones, K. (1997, November). *TREC-6 1997 Spoken Document Retrieval Task Overview and Results.* In proceedings of the Sixth Text REtrieval Conference (TREC-6), Gaithersburg, Maryland, USA. Over, P. (1997, November). *TREC-6 Interactive Track Report.* In proceedings of the Sixth Text REtrieval Conference (TREC-6), Gaithersburg, Maryland, USA.
1996	Garris, M., & Dimmick, D. (1996). Form Design for High Accuracy Optical Character Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(6), 653-656. Voorhees, E., & Harman, D. (November, 1996). *Overview of the Fifth Text REtrieval Conference (TREC-5).* In proceedings of the Fifth Text REtrieval Conference (TREC-5), Gaithersburg, Maryland, USA. Harman, D. (1996, August). *Panel: Building and Using Test Collections.* In proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Zurich, Switzerland. Kantor, P., & Voorhees, E. (November, 1996). *Report on the TREC-5 Confusion Track.* In proceedings of the Fifth Text REtrieval Conference (TREC-5), Gaithersburg, Maryland, USA. Voorhees, E. (November, 1996). *The TREC-5 Database Merging Track.* In proceedings of the Fifth Text REtrieval Conference (TREC-5), Gaithersburg, Maryland, USA. Over, P. (1996, November). *TREC-5 Interactive Track Report.* In proceedings of the Fifth Text REtrieval Conference (TREC-5), Gaithersburg, Maryland, USA. Downey, L., Laskowski, S., Buie, E., Hartson, H. (1996). Usability Engineering: Industry-Government Collaboration for System Effectiveness and Efficiency. Special Interest Group on Computer & Human Interaction (SIGCHI) Bulletin, 28(4), 66-67.
1995	Harman, D. (1995). Lab Report Special Section: Natural Language Processing and Information Retrieval Group Information Access and User Interfaces Division, National Institute of Standards and Technology. SIGIR Forum, 29(2), 6-10. Garris, M., Blue, J., Candela, G., Dimmick, D., Geist, J., Grother, P., Janet, S., & Wilson, C. (1995, October). *Off-line Handwriting Recognition From Forms.* In proceedings of the 1995 IEEE International Conference on Systems, Man, and Cybernetics, Vancouver, British Columbia, Canada. Harman, D. (1995, November). *Overview of the Fourth Text REtrieval Conference (TREC-4).* In proceedings of the Fourth Text REtrieval Conference (TREC-4), Gaithersburg, Maryland, USA. Rogers, W., Candela, G., Harman, D. (1995, April). *Space and Time Improvements for Indexing in Information Retrieval.* In proceedings of the Fourth Annual Symposium on Document Analysis and Information Retrieval (SDAIR '95), Las Vegas, Nevada, USA. Garris, M., Blue, J., Candela, G., Dimmick, D., Geist, J., Grother, P., Janet, S., & Wilson, C. (1995, February). *Public Domain OCR.* In proceedings of SPIE's Conference on Document Recognition Technologies, San Jose, CA, USA. Harman, D. (1995, month). *The TREC Conferences.* Keynote paper presented at the Hypertext-Information Retrieval Multimedia Conference, Konstanz, Germany.
1994	Harman, D. (1994, October). *Analysis of Data from the Second Text REtrieval Conference (TREC-2).* In proceedings of the Intelligent Multimedia Information Retrieval Systems and Management Conference (RIAO '94) at Rockefeller University, New York, New York, USA. Harman, D. (1994, month). *Data Preparation.* In proceedings of the TIPSTER Text Program Phase I, San Mateo, California, USA. Harman, D. (1994, month). *Document Detection: Summary of Results.* In proceedings of the TIPSTER Text Program Phase I, San Mateo, California, USA. Harman, D. (1994, November). *Overview of the Third Text REtrieval Conference (TREC-3).* In proceedings of the Third Text REtrieval Conference (TREC-3), Gaithersburg, Maryland, USA. Willman, N. (1994, October). *A Prototype Information Retrieval System to Perform a Best-Match Search for Names.* In proceedings of the Intelligent Multimedia Information Retrieval Systems and Management (RAIO '94) at Rockefeller University, New York, New York, USA. Harman, D. (1994, April). *The Text REtrieval Conference (TREC).* Keynote paper presented at the Third Annual Symposium on Document Analysis and Information Retrieval, Las Vegas, Nevada, USA.
1993
1992	Harman, D. (1992). Automatic Indexing. In R. Fidel et. al. (Eds.), Challenges in Indexing Electronic Text and Images (pp. 247-264), Medford, New Jersey, USA: Learned Information Inc. Harman, D. (1992). Evaluation Issues in Information Retrieval. Information Processing and Management, 28(4), 439-440. Harman, D., Fox, E., Baeza-Yates, R., & Lee, W. (1992). Inverted Files. In W. Frakes & R. Baeza-Yates (Eds.), Information Retrieval: Data Structures and Algorithms (pp.28-43), Englewood Cliffs, New Jersey, USA: Prentice-Hall. Harman, D. (1992). Ranking Algorithms. In W. Frakes & R. Baeza-Yates (Eds.), Information Retrieval: Data Structures and Algorithms (pp.363-392), Englewood Cliffs, New Jersey, USA: Prentice-Hall. Harman, D. (1992, June). *Relevance Feedback Revisited.* In proceedings of the 15th International Conference on Research and Development in Information Retrieval, Copenhagen, Denmark. Harman, D. (1992). Relevance Feedback and Other Query Modification Techniques. In W. Frakes & R. Baeza-Yates (Eds.), Information Retrieval: Data Structures and Algorithms (pp.241-263), Englewood Cliffs, New Jersey, USA: Prentice-Hall. Harman, D. (1992). User Friendly Systems Instead of User Friendly Front-Ends. Journal of the American Society for Information Science, 43(2), 164-174.
1991	Harman, D. (1991). How Effective is Suffixing? Journal of the American Society for Information Science, 42(1), 7-15.
1990	Harman, D., & Candela, G. (1990). Bringing Natural Information Language Retrieval Out of the Closet. SIGCHI Bulletin, 21(1), 42-48. Harman, D., & Candela, G. (1990). Retrieving Records from a Gigabyte of Text on a Minicomputer Using Statistical Ranking. Journal of the American Society for Information Science, 41(8), 581-589.
1989	Harman, D., & Candela, G. (1989). A Very Fast Prototype Retrieval System Using Statistical Ranking. SIGIR Forum, 23(3 and 4), 100-110.
1988	Harman, D. (1988). *IRX: An Information Retrieval System for Experimentation and User Applications.* In proceedings of the RIAO '88 Conference on User-Oriented Content-Based Text and Image Handling Conference, Boston, Massachusetts, USA. Harman, D. (1988). *Towards Interactive Query Expansion.* In proceedings of the 11th International Conference on Research and Development in Information Retrieval, Grenoble, France.
1987	Harman, D. (1987, month). *A Failure Analysis of the Limitation of Suffixing in an Online Environment.* In proceedings of the 10th Annual International ACM SIGIR Conference, New Orleans, Louisiana, USA.
1986	Harman, D. (1986, September). *An Experimental Study of Factors Important in Document Ranking.* In proceedings of the ACM Conference on Research and Development, Pisa, Italy.
	Last updated: Wednesday, 11-Jan-2006 10:32:01 EST Date created: Monday, 31-Jul-00