IRE An experiment: search strategy variations in SDI profiles chapter Lynn Evans Butterworth & Company Karen Sparck Jones All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, including photocopying and recording, without the written permission of the copyright holder, application for which should be addressed to the Publishers. Such written permission must also be obtained before any part of this publication is stored in a retrieval system of any nature. 310 An experiment: search strategy variations in SDT profiles are given in Table 14.9. Although they seem to show a consistently bettet overall retrieval performance for the controlled-language boolean profile%. further analysis using the sign test for significant difference did not suppoi[OCRerr] this. Table 14.10 records the number of times controlled-language (CL) t[OCRerr]i free-language (FL) profiles showed superior retrieval performance on runs 6, 7 and 8. The highest [OCRerr]2 value for this data is 2.0 50 no significant difference is indicated even at the 10 per cent level. TABLE 14.9. Retrieval performance of controlled-language and free-language boolean profiles Ruo No. ol Pry/ile t[OCRerr]pe A[OCRerr][OCRerr]eraging method Retriei[OCRerr]al perf([OCRerr]rmance no. queries Recall (½) Precision (o/[OCRerr]) RJ RI/2 RI RJ/2 6 24 Controlled-language Av. of nos. 75.2 60.3 29.1 61.9 Av. of ratios 64.6 50.6 25.6 59.6 Free-language Av. of nos. 56.1 43.8 23.1 48.0 Av. of ratios 60.3 44.8 27.4 53.0 7 34 Controlled-language Av. of nos. 58.0 49.4 20.8 57.6 Av. of ratios 57.6 46.5 25.6 58.6 Free-language Av. of nos 51.4 39.7 17.6 44.4 Av. of ratios 57.2 42.8 21.4 49.8 8 32 Controlled-language Av. of nos. 63.7 53.0 19.5 50.4 Av. of ratios 53.7 45.7 18.1 43.9 Free-language Av. of nos. 57.9 45.9 15.8 39.0 Av. of ratios 53.6 42.8 14.3 40.7 TABLE 14.10. Retrieval performance of controlled-language (CL) and free-language (FL) boolean profiles Ru'i Recall Precision Ri do(uments RJ/2 [OCRerr]k)cuments RI documents RJ/2 documents CL FL Same CL FL Same CL FL Same CL FL Same better better better better better better better better 6 8 4 9 12 9 3 8 10 6 14 10 0 7 10 8 11 15 12 7 13 15 6 20 12 2 8 8 9 12 15 12 5 15 9 8 19 10 3 14.4 Conclusions As is often the case with experiments in information retrieval where conditions are peculiar to one situation or organization, the results obtained in the major project may be valid only for the INSPEC database. In particular a factor that might be expected to influence the experiment would be the medium used for matching profiles and documents; in this case the free- index terms assigned to all items in the database. INSPEC's operational statistics at the time indicated that the free-index field contained on average