org.apache.lucene.benchmark.byTask.feeds
Class TrecContentSource

java.lang.Object
  org.apache.lucene.benchmark.byTask.feeds.ContentSource
      org.apache.lucene.benchmark.byTask.feeds.TrecContentSource

public class TrecContentSource
extends ContentSource

Implements a ContentSource over the TREC collection.

Supports the following configuration parameters (on top of ContentSource):

• work.dir - specifies the working directory. Required if "docs.dir" denotes a relative path (default=work).
• docs.dir - specifies the directory where the TREC files reside. Can be set to a relative path if "work.dir" is also specified (default=trec).
• html.parser - specifies the HTMLParser class to use for parsing the TREC documents content (default=DemoHTMLParser).
• content.source.encoding - if not specified, ISO-8859-1 is used.
• content.source.excludeIteration - if true, do not append iteration number to docname

Field Summary

Fields inherited from class org.apache.lucene.benchmark.byTask.feeds.ContentSource

BUFFER_SIZE, encoding, forever, logStep, verbose

Constructor Summary

TrecContentSource()

Method Summary

void	close() Called when reading from this content source is no longer required.
DocData	getNextDocData(DocData docData) Returns the next DocData from the content source.
void	resetInputs() Resets the input for this content source, so that the test would behave as if it was just started, input-wise.
void	setConfig(Config config) Sets the Config for this content source.

Methods inherited from class org.apache.lucene.benchmark.byTask.feeds.ContentSource

addBytes, addDoc, collectFiles, getBytesCount, getConfig, getDocsCount, getInputStream, getTotalBytesCount, getTotalDocsCount, shouldLog

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TrecContentSource

public TrecContentSource()

Method Detail

close

public void close()
           throws IOException

Description copied from class: ContentSource

Called when reading from this content source is no longer required.

Specified by:

close in class ContentSource

Throws:

IOException

getNextDocData

public DocData getNextDocData(DocData docData)
                       throws NoMoreDataException,
                              IOException

Description copied from class: ContentSource

Returns the next DocData from the content source.

Specified by:

getNextDocData in class ContentSource

Throws:

NoMoreDataException

IOException

resetInputs

public void resetInputs()
                 throws IOException

Description copied from class: ContentSource

Resets the input for this content source, so that the test would behave as if it was just started, input-wise.

NOTE: the default implementation resets the number of bytes and documents generated since the last reset, so it's important to call super.resetInputs in case you override this method.

Overrides:

resetInputs in class ContentSource

Throws:

IOException

setConfig

public void setConfig(Config config)

Description copied from class: ContentSource

Sets the Config for this content source. If you override this method, you must call super.setConfig.

Overrides:

setConfig in class ContentSource