Using IRF as-is with the sample application

---

Introduction to SampleApp

Running SampleApp

java gov.nist.nlpir.irfapps.SampleApp [options]

The command line options are as follows:

help

Display the list of command line options.

noinput

Run in batch mode. If this option is not present then the sample application will expect input from standard input - see Interactive or Pseudo-Interactive mode. In "noinput" mode the default path to the raw document data and a query are relative and assume the current directory is ....gov/nist/nlpir/irfapps

irmDir pathname

Use pathname instead of the default (current directory) to find/save the serialized IrfManager and related files. As shipped, these are contained in files with names starting "DB".

indexDir pathname

Use pathname instead of the default (current directory) to find/save the index data for a collection. As shipped, these are contained in files with names starting "DB". Saved with the IrfManager is the location of existing collections so the path to them is not needed once the IrfManager knows about them and has itself been saved.

detail

Print detailed trace information during execution.

medium

Print medium-level trace information during execution.

coarse

Print coarse trace information during execution.

Interactive operation

MENU:
 1 - Display available collections
 2 - Choose a collection to work with
 3 - Present all indexes for current collection
 4 - Display index statistics for current collection
 5 - Display stopword statistics for current collection
 6 - Add document(s) to the current collection
 7 - Update current collection's indexes (idf etc)
 8 - Retrieve
 9 - Set indexing modalities for current collection
11 - Show IR manager
12 - Show statistics on all indexes for current collection
13 - Dump in-memory proxy table

 0 - QUIT

• All sessions involving interaction with an index (create, append, update and retrieve) should commence by choosing a collection to work with (option 2);
• When creating an index, the indexing modalities for a document class are derived from the document class' converter class - it is necessary to supply the full name of the converter class in order to create an index in option 2, for example "gov.nist.nlpir.irfapps.hci.Bib2AppDocConv" or "gov.nist.nlpir.irfapps.trec.cran.CranfieldConverter";
• When creating an index, it is necessary to set the indexing modalities (option 9) for the index before documents can be added. The default modalities may be adopted or edited prior to building the index. Note that while the interface allows for the addition of modalities to be indexed, this operation is only meaningful where the documents in the collection contain a field or element corresponding to the specified modality;
• Documents may be added to an index (option 6) at any time. However, the indexing modalities may only be set (option 9) at the time the index is first created;
• Once an index has been created or augmented (option 6), it should be updated (option 7) prior to retrieval or exiting SampleApp.

Pseudo-interactive operation

On platforms which allow redirection of standard input, the input supplied by a user can be provided instead by a file. The contents of this file should mimic exactly the input to SampleApp as if the required operation were to be performed in interactive mode, with each input field separated by a newline. The file may also contain comments, which should be preceded with a "#". The irfapps directory contains a sample file for use with the HCI collection on Unix systems: testHci.inp. (It can also serve as a guide for an interactive session.) The file directs SampleApp to create a new index in the current directory, update the index, perform a retrieval operation on it, taking the query from the named file, and finally display the highest-ranking document before exiting. The relative paths assume the current directory is irfapps.

Usage examples

Assuming Unix and that current directory is irfapps:

java gov.nist.nlpir.irfapps.SampleApp coarse < testHci.inp

java gov.nist.nlpir.irfapps.SampleApp

testHci.inp

This is what testHci.inp contains:

2					# choose/create a collection
0					# create new collection
.					# no name
.					# no description
.					# store in current directory
gov.nist.nlpir.irfapps.hci.Bib2AppDocConv # name of converter class
9					# set indexing modalities
d					# use default indexing modalities
0					# return to previous menu
6					# index 
hci/data/bib1000			# source of raw document data
1					# starting doc index
1000					# ending doc index
y					# index an index at a time
7					# update
8					# search
1					# weight of index
1					# weight of index
1					# weight of index
lin					# use linear combination of indexes
f					# take query from file
hci/data/q1				# source of query
1					# present first doc in result
0					# end doc display
n					# no more queries
0					# exit application

For information on weighting factors for indexing modalities and the methods for combining indexing modality results to give a final document score, see the section on retrieving.

Batch mode operation

SampleApp is run in batch, or noinput, mode by specifying the "noinput" command line option, as described above. The default noinput path to the raw document data and a query are relative and assume the current directory is ....gov/nist/nlpir/irfapps. Once noinput has been specified, the following additional command line parameters become available:

index

Perform indexing only.

update

Perform indexing and updating only.

retrieve

Perform retrieval only, from existing collection number 1.

from X

Start indexing at document number X. Default is 1.

to Y

End the index at document number Y. Default is 20.

with Z

Index the document collection contained in file Z. Default is hci/data/bib1000.

query Q

Use the query contained in file Q. Default is hci/data/q1.

converter C

Use the converter class C. Default is gov.nist.nlpir.irfapps.hci.Bib2AppDocConv.

byindex

Index by indexing feature as opposed to by document, i.e., build all indexes one at a time by scanning each document only for the relevant indexing feature, as opposed to scanning each document for all indexing features and incrementing every index each time a document is scanned. Default is false.

Usage examples

Assuming Unix and that current directory is irfapps and no DB* files exist:

Perform default indexing and retrieval:

java gov.nist.nlpir.irfapps.SampleApp noinput

rm DB*; java gov.nist.nlpir.irfapps.SampleApp noinput to 50 index

rm DB*; java gov.nist.nlpir.irfapps.SampleApp noinput to 50 update

java gov.nist.nlpir.irfapps.SampleApp noinput retrieve

Sample document collections