com.sun.labs.minion.pipeline
Class AbstractPipelineImpl

java.lang.Object
  com.sun.labs.minion.pipeline.AbstractPipelineImpl

All Implemented Interfaces:

Pipeline

Direct Known Subclasses:

AsyncPipelineImpl, SyncPipelineImpl

public abstract class AbstractPipelineImpl
extends java.lang.Object
implements Pipeline

An abstract implementation of pipeline.

Field Summary

protected java.lang.String	currKey The current key that we're working on.
protected java.text.SimpleDateFormat	d64 A date format for dates including milliseconds.
protected long	docSize The size of the current document, in characters.
protected Dumper	dumper A place where we can dump our data.
protected SearchEngine	engine The engine that is using this pipeline.
protected Stage	head The stage at the head of the pipeline.
protected static java.lang.String	logTag Our log tag.
protected java.util.List<Stage>	pipeline The list of stages making up the pipeline.
protected char[]	text Character data taken from a field.

Constructor Summary

AbstractPipelineImpl(PipelineFactory factory, SearchEngine engine, java.util.List<Stage> pipeline, Dumper dumper)
Creates a AbstractPipelineImpl

Method Summary

FieldInfo	addImpliedField(java.lang.String name)
void	endDocument() Ends the current document.
SearchEngine	getEngine()
Stage	getHead() Gets the head of the pipeline.
Stage	getIndexer() Gets the indexer stage for this pipline.
protected void	handleField(FieldInfo fi, java.lang.Object val) Handles a single field value, which may have several instances.
protected long	handleField(FieldInfo fi, java.lang.Object val, IndexableString.Type type) Handle the characters for one instance of a given field.
protected void	indexDoc(java.lang.String key, java.util.Map m) Does the actual work of indexing a document.
boolean	isIndexed(java.lang.String key) Indicates whether a document has been indexed or not.
protected void	realDump() Does actual dumping of the indexed data and resetting of the indexing stage so that dumps may proceed asynchronously, if required.
protected void	realPurge() Purges the current in-memory data, not dumping it out to disk.
void	setIndexer(Stage s) Sets the indexing stage for this pipeline.
void	startDocument(java.lang.String key) Starts the indexing of a document.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.sun.labs.minion.Pipeline

dump, flush, index, purge, shutdown

Field Detail

pipeline

protected java.util.List<Stage> pipeline

The list of stages making up the pipeline.

head

protected Stage head

The stage at the head of the pipeline.

dumper

protected Dumper dumper

A place where we can dump our data.

d64

protected java.text.SimpleDateFormat d64

A date format for dates including milliseconds.

engine

protected SearchEngine engine

The engine that is using this pipeline.

logTag

protected static java.lang.String logTag

Our log tag.

text

protected char[] text

Character data taken from a field.

currKey

protected java.lang.String currKey

The current key that we're working on.

docSize

protected long docSize

The size of the current document, in characters.

Constructor Detail

AbstractPipelineImpl

public AbstractPipelineImpl(PipelineFactory factory,
                            SearchEngine engine,
                            java.util.List<Stage> pipeline,
                            Dumper dumper)

Creates a AbstractPipelineImpl

Method Detail

getEngine

public SearchEngine getEngine()

getHead

public Stage getHead()

Description copied from interface: Pipeline

Gets the head of the pipeline.

Specified by:

getHead in interface Pipeline

Returns:

the stage at the head of the pipeline.

getIndexer

public Stage getIndexer()

Gets the indexer stage for this pipline. This can be used by a simple indexer.

setIndexer

public void setIndexer(Stage s)

Sets the indexing stage for this pipeline. This allows us to put in a new indexing stage while dumps are happening asynchronously.

handleField

protected void handleField(FieldInfo fi,
                           java.lang.Object val)
                    throws java.io.IOException

Handles a single field value, which may have several instances.

Throws:

java.io.IOException

handleField

protected long handleField(FieldInfo fi,
                           java.lang.Object val,
                           IndexableString.Type type)

Handle the characters for one instance of a given field.

Parameters:

fi - The field information for the field we're adding to.

val - The data as a string.

type - The type of markup in the value. This is take from the IndexableString.Type enum.

Returns:

the number of characters indexed

See Also:

IndexableString, IndexableFile

indexDoc

protected void indexDoc(java.lang.String key,
                        java.util.Map m)
                 throws SearchEngineException

Does the actual work of indexing a document.

Throws:

SearchEngineException

addImpliedField

public FieldInfo addImpliedField(java.lang.String name)

isIndexed

public boolean isIndexed(java.lang.String key)

Indicates whether a document has been indexed or not. A document has been indexed if its document key is in the index and the document has not been deleted.

Parameters:

key - the document key for the document that we wish to check.

Returns:

true if the document is in the index, false otherwise. A document has been indexed if its document key is in the index and the document has not been deleted.

endDocument

public void endDocument()

Ends the current document.

Throws:

SearchEngineException - if there is any error ending the document.

startDocument

public void startDocument(java.lang.String key)

Starts the indexing of a document.

Parameters:

key - the document key for the document. If this is a duplicate, then any old data associated with the document will be removed.

Throws:

java.lang.NullPointerException - if the document key is null

realDump

protected void realDump()

Does actual dumping of the indexed data and resetting of the indexing stage so that dumps may proceed asynchronously, if required.

realPurge

protected void realPurge()

Purges the current in-memory data, not dumping it out to disk.