Introduction: For the non-colorblind who really want to wallow in the details, BeadPlot is a freely available, 100% Java tool for visually comparing TREC result sets at a low level - in terms of which "documents" are retrieved at which ranks for a given topic across multiple result sets. Each result is represented as a string of beads (colored rectangles) laid out horizontally with the highest ranked document represented by the leftmost bead in the strand. One of the results is designated as the reference result and colors are assigned in spectral sequence (ROYGBIV) to its documents' beads if they fall within a specified range (the reference range). Other documents in the reference result are represented by gray rectangles. Each colored bead has two parts - the upper shows its color/rank neighborhood while the lower helps distinguish beads within a neighborhood. If a document from the reference result appears in another result, it will be represented the same there as in the reference result. Documents not retrieved in the reference result are not represented anywhere.

Example plot:

Interactive functions: You can use the items under the View menu to...

• change the topic
• look at a different subset of the result sets
• move the starting rank of the reference range
• make the reference range larger/smaller
• see only (non)relevant documents

• pick a different reference result
• move a result to the bottom
• highlight a document everywhere it appears
• undo the document highlighting

Code: The source code - two classes - is currently available in an alpha version. Download the two files and invoke the java compiler on BeadPlot.java. BeadPlot assumes Java 2. Invoke Beadplot (java BeadPlot) and click on Help for information about what BeadPlot requires and can do. Beadplot expects a qrels file (a version with only documents judged relevant will do or the full version) and result sets in TREC (trec_eval) format. The more result sets displayed, the more memory it needs; so start off with a dozen or fewer and see how it does on your machine. The alpha version still outputs some debugging information and generally is not defending itself against various error conditions. Error messages may be less than helpful.

Restrictions on BeadPlot's use: BeadPlot was produced at the National Institute of Standards and Technology by employees of the Federal Government in the course of their official duties and, pursuant to title 17 Section 105 of the United States Code, is not subject to copyright protection. It is in the public domain and can be redistributed and/or modified freely provided that any derivative works bear some notice that they are derived from it, and any modified versions bear some notice that they have been modified.

Disclaimers: BeadPlot is an experimental system. NIST assumes no responsibility whatsoever for its use by other parties, and makes no guarantees, expressed or implied, about its quality, reliability, or any other characteristic. We would appreciate acknowledgement if the software is used.

Downloads