CRANV2 Supplementary tests and results chapter Cyril Cleverdon Michael Keen Cranfield An investigation supported by a grant to Aslib by the National Science Foundation. Use, reproduction, or publication, in whole or in part, is permitted for any purpose of the United States Government. - 228 - Coord - ination Level 1 2 3 4 5 6 7 FIGURE 6.7T. of numbers 1.1% 2.7% 5.2% 13.5% 23.8% 37.2% 50.0% Average of Ratios 1 2 3 (Total divided (Total divided (Total divided by 35) by figure by figure shown in shown in brackets) brackets) 1.4% 1.4%(35) 1,4%(35) 4.3% 4.3%(35) 4.4%(34) 8.0% 8.2%(34) 9.0%(31) 16.0% 20.1%(28) 24.4%(23) 21.7% 42.2%(18) 54.3%(14) 11.0% 55.0%(7) 64.2%(6) 6.7% 77.8%(3) 77.8%(3) PRECISION RATIOS OBTAINED BY THREE DIFFERENT AVERAGE OF RATIOS PROCEDURES. use the average of numbers. Comparison of documents dealing with aerodynamics and structures The main sets of test results in Chapter 4 were concerned with a subset of 42 questions all of which dealt with aerodynamics rather than structures. For comparison purposes, a set of 42 questions on structures was prepared. Searched on the 1400 document collection, with index language I.l.a, the tests results are given in Fig. 6.8T. Comparison is made in Fig. 6.9P with the results as given in Fig. 4.120T for the 42 aerodynamic questions under the same conditions. This plot shows an unusual characteristic, in that at the higher recall levels, the structure questions have superior precision, but at a recall ratio of about 25%, the curves cross over, and the aerodynamic questions have the better performance. There are two" reasons why one would expect the structure questions to do better. Firstly there are more relevant documents, and therefore the generality number is higher, namely 4.3 as against 3.4. Secondly, although to calculate the generality number N is presumed to be 1400, real N must (as argued on pages 71 - 76) be considerably less than this number. If the position at a coordination level of 3 is considered, the perform- ance figures are as follows: Aerodynamics Structures (As'Fig. 4.120T) (As Fig. 6.9T) Recall Precision Fallout Recall Precision Fallout Ratio Ratio Ratio Ratio Ratio Ratio 66.7% 3.2% 6.790% 67.5% 8.6% 1.732% To allow for the difference in the generality number, the precision ratio for the aerodynamic questions can be adjusted by the equation given on page 73 and this would result in a new precision ratio of 4.1% which continues to be well below the comparable figure for the structures questions.