SP500215 Probabilistic Learning Approaches for Indexing and Retrieval with the TREC-2 Collection chapter N. Fuhr U. Pfeifer C. Bremkamp M. Pollmann National Institute of Standards and Technology D. K. Harman 1 0.9- `dortVl" [OCRerr] "dortPl" I 0.8[OCRerr] 0.7- 0.6- Precision 0.5- 0.4- 0.3- 0.2- 0.1- 0 I i I I I I I I I I 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Recall Figure 2: Recall-precision curves of routing runs and the RPI retrieval function yields Q(qk,dm)= [OCRerr] log(cikuim+1). tIEq[OCRerr]T 4.2 Experiments In principle, the RPI formula can be applied with or without query expansion. For our experiments in TRECi, we did not use any query expansion. The final results showed that this was reasonable, mainly with respect to the small amount of relevance feed- back data available. In contrast, for TREC2 there were about 2000 relevance judgements per query, so there was clearly enough training data for applying query expan- sion methods. As basic criterion for selecting the expansion terms, we considered the number of relevant documents in which a term ocurs, which gave us a ranking of candidates; docu- ment indexing weights were considered for tie-breaking. Then we varied the the number of terms which are added to the original query. expansion result 0 0.2909 10 0.3047 30 0.3035 50 0.3002 100 0.2832 Table 8: Effect of number of expansion terms In a first series of experiments, we considered single word only. We used Q2/D1 (lsp document indexing) as training sample and Q2/D2 (ltc indexing) as test sample. As can be seen from table 8, query expansion clearly improves retrieval quality, but only for a limited (7) number of expansion terms. For larger numbers, we get worse results. This effect seems to be due to parameter estimation problems. expansion phraseweight single w. phrases result 0.5 0 0 0.3476 0.5 20 0 0.3713 1.0 20 0 0.3730 0.5 20 10 0.3728 1.0 20 10 0.3605 0.5 30 10 0.3729 1.0 30 10 0.3626 Table 9: Query expansion with phrases In a second series of experiments, we looked at the combination of single words and phrases. These ex- periments were performed as retrospective runs, with Q2/D12 as training sample and Q2/D2 as test sample (both with ltc document indexing). For the number of expansion terms, we treated single words and phrases separately. Furthermore, similar to the adhoc runs, we used an additional factor for downweighting the query term weights of phrases. The different parameter com- binations tested and the corresponding results are given in table 9. Obviously, phrases as expansion terms gave no improvement, so we decided to have only single words as expansion terms (but the phrases from the original query still are used for retrieval). Furthermore, the re- trieval quality reaches its optimum at about 20 terms. 72