|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object com.sun.labs.minion.indexer.partition.PartitionManager
public class PartitionManager
For any particular collection that we will index, there can be multiple
entry types and for each entry type there can be multiple partitions.
An PartitionManager
is used to manage all of the partitions
in a collection that have the same entry type.
The static getManager
method can be used to retrieve the
partition manager for a particular entry type.
The PartitionManager
maintains a static HashMap that maps
index directories and entry names to the actual manager instance for
entries of that type. The key for the hash is
<indexDir
>/<EntryType
>, which will
allow us to open multiple collections in the same VM.
The PartitionManager
for a particular entry type provides
access to the set of DiskPartitions
that contain entries of
that type.
The PartitionManager
is also responsible for providing two
kinds of data for partition use. First, it hands out the numbers that
are used for the partitions. These numbers are local to the
PartitionManager
for a given entry type.
A partition manager can be given a name via the setName
method. This name can be used to find the corresponding paritition
manager object. The name defaults to the name of the index directory.
Nested Class Summary | |
---|---|
protected class |
PartitionManager.ExtFilter
A class that implements SimpleFilter so that we can find partition files with various extensions. |
protected class |
PartitionManager.HouseKeeper
An inner class that does housekeeping duties during querying. |
class |
PartitionManager.Merger
A threadable class used for merging a list of partitions. |
Field Summary | |
---|---|
protected java.io.File |
activeFile
The file containing the list of active partitions. |
protected FileLock |
activeLock
A lock for our active file. |
protected java.util.List<DiskPartition> |
activeParts
The list of partitions that we're managing. |
protected SearchEngineImpl |
engine
The search engine that is using us. |
protected java.util.List<java.lang.String> |
fieldsToLoad
A list of field names that should be loaded into main memory for faster processing. |
protected IndexConfig |
indexConfig
The index configuration for the index we'll be managing. |
protected java.lang.String |
indexDir
The directory where the index is. |
protected java.io.File |
indexDirFile
The File containing the directory where the index is
held. |
protected PartitionManager.HouseKeeper |
keeper
A house keeper class. |
protected java.lang.Thread |
keeperThread
A thread running the housekeeping duties. |
protected java.util.Date |
lastPurgeTime
The last time that a purge was called. |
protected java.io.File |
lockDirFile
The directory where locks will be put. |
protected java.lang.String |
logTag
The tag for this module. |
protected java.util.List<DiskPartition> |
mergedParts
A list of partitions that have been merged. |
protected FileLock |
mergeLock
A lock for the collection so that only one merge may be ongoing at any time. |
protected int |
mergeRate
The rate of partition merges - bigger means less merges, faster indexing, more parts, slower queries |
protected int |
mergeSpace
The amount of space (in bytes) that we're willing to devote to buffering postings entries during merges. |
protected java.lang.Thread |
mergeThread
A thread to be used during merge operations. |
protected MetaFile |
metaFile
The MetaFile containing the number for the next partition
to write and the field name maps. |
protected java.lang.String |
name
The configuration name for this partition manager. |
static java.lang.String |
PROP_ACTIVE_CHECK_INTERVAL
|
static java.lang.String |
PROP_ASYNC_MERGES
|
static java.lang.String |
PROP_CALCULATE_DVL
|
static java.lang.String |
PROP_INDEX_CONFIG
|
static java.lang.String |
PROP_LOCK_DIR
|
static java.lang.String |
PROP_MAX_MERGE_SIZE
|
static java.lang.String |
PROP_MERGE_RATE
|
static java.lang.String |
PROP_OPEN_PARTITION_HIGH_WATER_MARK
A property for the maximum number of open partitions that we'll allow. |
static java.lang.String |
PROP_OPEN_PARTITION_LOW_WATER_MARK
A property for the "low water" number of open partitions that we'll allow. |
static java.lang.String |
PROP_PART_CLOSE_DELAY
|
static java.lang.String |
PROP_PART_REAP_DELAY
|
static java.lang.String |
PROP_PARTITION_FACTORY
|
static java.lang.String |
PROP_REAP_DOES_NOTHING
|
static java.lang.String |
PROP_STARTING_DATA
A configuration property that can be used to name an index directory whose contents should be copied into the current directory when it is created. |
static java.lang.String |
PROP_TERMSTATS_DICT_FACTORY
|
protected java.util.Timer |
queryTimer
A timer that can be used during querying to time tasks. |
protected int |
randID
A random number that we can use to tell the difference between various partition managers. |
protected java.lang.String |
subDir
The subdirectory of the main index directory where we will put our partitions. |
protected java.util.List<Closeable> |
thingsToClose
The list of parts to close. |
Constructor Summary | |
---|---|
PartitionManager()
Instantiates a PartitionManager with the given index
configuration. |
Method Summary | |
---|---|
void |
addIndexListener(IndexListener il)
|
protected void |
addNewPartition(DiskPartition dp,
java.util.Set<java.lang.Object> keys)
|
protected void |
addNewPartition(int partNumber,
java.util.Set<java.lang.Object> keys)
Adds a new partition to this manager. |
void |
checkHK()
Checks to make sure that the housekeeper is still alive. |
void |
deleteDocument(java.lang.String key)
Deletes a single document from whatever partition that it is in. |
void |
deleteDocuments(java.util.List<java.lang.String> keys)
Deletes a set of documents from whatever partition that they are in. |
protected void |
deleteKeys(java.util.Set<java.lang.Object> keys,
java.util.List parts)
Delete the given keys from the given list of partitions. |
java.util.List<DiskPartition> |
getActivePartitions()
Returns a list of the currently active partitions. |
java.util.List |
getAllFieldValues(java.lang.String field,
java.lang.String key)
Gets all of the field values associated with a given field in a given document. |
boolean |
getCalculateDVL()
|
double |
getDistance(int d1,
int d2,
java.lang.String name)
|
double |
getDistance(java.lang.String k1,
java.lang.String k2,
java.lang.String name)
Gets the distance between two documents, based on the values stored in in a given feature vector saved field. |
DocKeyEntry |
getDocumentTerm(java.lang.String key)
Gets a term from a document dictionary corresponding to the given key. |
DocumentVector |
getDocumentVector(java.lang.String key)
Gets a document vector for the given document key. |
DocumentVector |
getDocumentVector(java.lang.String key,
java.lang.String field)
Gets a document vector for the given document key. |
DocumentVector |
getDocumentVector(java.lang.String key,
WeightedField[] fields)
Gets a composite document vector for the given document key. |
SearchEngine |
getEngine()
Gets the search engine associated with this PartitionManager instance |
FieldInfo |
getFieldInfo(java.lang.String name)
Gets the information for a named field. |
FieldIterator |
getFieldIterator(java.lang.String field)
Gets an iterator for all the values in a field. |
FieldIterator |
getFieldIterator(java.lang.String field,
boolean ignoreCase)
Gets an iterator for all the values in a field. |
java.util.List<java.lang.String> |
getFieldNames()
|
java.lang.Object |
getFieldValue(java.lang.String field,
java.lang.String key)
Gets a single field value associated with a given field in a given document. |
IndexConfig |
getIndexConfig()
Gets the index configuration for this manager. |
java.lang.String |
getIndexDir()
Get the directory where the index is. |
java.util.Date |
getLastPurgeTime()
|
java.lang.String |
getLockDir()
|
java.util.SortedSet<FieldValue> |
getMatching(java.lang.String field,
java.lang.String pattern)
Gets the values for the given field that match the given pattern. |
PartitionManager.Merger |
getMerger()
Gets an instance of the merger class that can be used to merge any partitions that require it. |
PartitionManager.Merger |
getMerger(java.util.List<DiskPartition> l)
Gets an instance of the merger class in order to merge a list of partitions. |
PartitionManager.Merger |
getMerger(java.util.List<DiskPartition> l,
FileLock localMergeLock)
Gets an instance of the merger class in order to merge a list of partitions. |
PartitionManager.Merger |
getMergerFromNumbers(java.util.List<java.lang.Integer> l)
Gets a merger that will merge the partitions represented by the given list of partition numbers. |
MetaFile |
getMetaFile()
Get the meta file for this index. |
int |
getNActive()
Returns the number of active partitions being managed. |
java.lang.String |
getName()
Gets the name of the index. |
int |
getNDocs()
Gets the total number of documents managed. |
protected int |
getNextPartitionNumber()
Gets the next number to use for a partition. |
int |
getNFields()
|
int |
getNTerms()
Gets the total number of terms indexed. |
long |
getNTokens()
Gets the total number of tokens indexed. |
int |
getPartCloseDelay()
|
protected java.util.List<DiskPartition> |
getPartitions(java.util.List<java.lang.Integer> partNums)
Gets a list of partitions from the corresponding partition numbers. |
protected java.util.List<java.lang.Integer> |
getPartNumbers(java.util.List<DiskPartition> parts)
Gets a list of the partition numbers for the given partitions. |
QueryConfig |
getQueryConfig()
Gets the query configuration for this manager. |
java.util.Timer |
getQueryTimer()
Get a timer that can be used during querying to time tasks. |
int |
getRandID()
|
ResultSet |
getSimilar(java.lang.String key,
java.lang.String name)
Gets a set of results ordered by similarity to the given document, calculated by computing the euclidean distance based on the feature vector stored in the given field. |
TermStatsImpl |
getTermStats(java.lang.String name)
Gets the term statistics for a term |
TermStatsDictionary |
getTermStatsDict()
Gets the term statisitics dictionary for this index |
java.util.List<FieldFrequency> |
getTopFieldValues(java.lang.String field,
int n,
boolean ignoreCase)
Gets a list of the top n most frequent field values for a given named field. |
boolean |
hasFieldedVectors()
Indicates whether this index uses fielded document vectors. |
protected void |
init()
Initializes the PartitionManager. |
boolean |
isCasedIndex()
Indicates whether this index uses a cased main dictionary. |
boolean |
isIndexed(java.lang.String key)
Checks to see if a document is in the index. |
protected java.io.File |
makeActiveFile()
Makes a File for the active file. |
java.io.File |
makeDeletedDocsFile(int partNumber)
Makes a File for the file containing the bitmap of
deleted documents. |
static java.io.File |
makeDeletedDocsFile(java.lang.String iD,
int partNumber)
Makes a File for the file containing the bitmap of
deleted documents. |
java.io.File |
makeDictionaryFile(int partNumber,
java.lang.String type)
Makes a File for a dictionary. |
static java.io.File |
makeDictionaryFile(java.lang.String iD,
int partNumber,
java.lang.String type)
Makes a File for a dictionary. |
protected java.io.File |
makeMetaFile()
Makes a File for the meta file. |
java.io.File |
makePostingsFile(int partNumber,
java.lang.String type)
Makes a File for a postings file. |
java.io.File |
makePostingsFile(int partNumber,
java.lang.String type,
int number)
Makes a File for a postings file. |
static java.io.File |
makePostingsFile(java.lang.String iD,
int partNumber,
java.lang.String type,
int number)
Makes a File for a postings file. |
java.io.File |
makeRemovedPartitionFile(int partNumber)
Makes a File that we'll use to indicate that this
partition has been merged away. |
static java.io.File |
makeRemovedPartitionFile(java.lang.String iD,
int partNumber)
Makes a File that we'll use to indicate that this
partition has been merged away. |
java.io.File |
makeTaxonomyFile(int partNumber)
|
static java.io.File |
makeTaxonomyFile(java.lang.String iD,
int partNumber)
|
protected java.io.File |
makeTermStatsFile(int tsn)
Makes a File for the global term stats. |
java.io.File |
makeVectorLengthFile(int partNumber)
Makes a File for the file containing the lengths of
document vectors. |
static java.io.File |
makeVectorLengthFile(java.lang.String iD,
int partNumber)
Makes a File for the file containing the lengths of
document vectors. |
protected DiskPartition |
merge(java.util.List<DiskPartition> parts)
Merges together the partitions in the provided list. |
DiskPartition |
mergeAll()
Merges all partitions from the active list into a new partition. |
java.util.List<DiskPartition> |
mergeGeometric()
This is a geometric merge heuristic controlled by the mergeRate. |
protected DiskPartition |
mergeInPieces(java.util.List<DiskPartition> parts)
Breakss a list of partitions into blocks of mergeBlockSize , and merges those. |
protected DiskPartition |
newDiskPartition(java.lang.Integer partNum,
PartitionManager m)
Instantiates a disk partition of the correct type for this manager |
void |
newProperties(com.sun.labs.util.props.PropertySheet ps)
|
void |
noMoreMerges()
|
void |
purge()
Purges the collection. |
protected java.util.List<java.lang.Integer> |
readActiveFile()
Reads the numbers of the active partitions from the active file. |
protected DiskPartition |
realMerge(java.util.List<DiskPartition> diskParts,
boolean calculateDVL)
Merges a list of partitions. |
void |
reap()
Reaps deleted partitions from the collection. |
protected void |
reapPartition(int partNumber)
A method to reap a single partition. |
void |
recalculateTermStats()
Regenerates the term stats for the currently active partitions. |
static void |
recover(java.lang.String iD)
Recovers an index directory. |
void |
removeIndexListener(IndexListener il)
|
void |
setEngine(SearchEngineImpl engine)
Sets the search engine associated with this partition manager. |
void |
setLockDir(java.lang.String lockDir)
|
void |
setMergeRate(int rate)
Sets the rate of partition merges during indexing. |
void |
setPartCloseDelay(int partCloseDelay)
|
void |
shutdown()
Shuts down the manager. |
protected void |
startHK()
Starts the housekeeping thread if it's null or dead. |
java.util.List<DiskPartition> |
updateActiveParts(boolean addNew)
Reads the active file and adds any new partitions to our active list. |
protected void |
updateTermStats()
|
protected void |
writeActiveFile(java.util.List<DiskPartition> parts)
Writes a list of partition numbers to the active file. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected SearchEngineImpl engine
protected IndexConfig indexConfig
protected java.util.Timer queryTimer
protected java.lang.String name
protected java.util.Date lastPurgeTime
protected java.io.File lockDirFile
protected java.io.File activeFile
protected FileLock activeLock
protected FileLock mergeLock
protected java.util.List<DiskPartition> activeParts
protected java.util.List<Closeable> thingsToClose
protected java.util.List<DiskPartition> mergedParts
protected java.util.List<java.lang.String> fieldsToLoad
protected java.lang.Thread mergeThread
protected PartitionManager.HouseKeeper keeper
protected java.lang.Thread keeperThread
protected MetaFile metaFile
MetaFile
containing the number for the next partition
to write and the field name maps.
protected java.lang.String indexDir
protected java.io.File indexDirFile
File
containing the directory where the index is
held.
protected int mergeSpace
protected int mergeRate
protected int randID
protected java.lang.String logTag
protected java.lang.String subDir
@ConfigComponent(type=DiskPartitionFactory.class) public static final java.lang.String PROP_PARTITION_FACTORY
@ConfigComponent(type=IndexConfig.class) public static final java.lang.String PROP_INDEX_CONFIG
@ConfigInteger(defaultValue=5) public static final java.lang.String PROP_MERGE_RATE
@ConfigInteger(defaultValue=40) public static final java.lang.String PROP_OPEN_PARTITION_HIGH_WATER_MARK
@ConfigInteger(defaultValue=20) public static final java.lang.String PROP_OPEN_PARTITION_LOW_WATER_MARK
@ConfigInteger(defaultValue=20) public static final java.lang.String PROP_MAX_MERGE_SIZE
@ConfigInteger(defaultValue=1000) public static final java.lang.String PROP_ACTIVE_CHECK_INTERVAL
@ConfigInteger(defaultValue=15000) public static final java.lang.String PROP_PART_CLOSE_DELAY
@ConfigBoolean(defaultValue=true) public static final java.lang.String PROP_ASYNC_MERGES
@ConfigInteger(defaultValue=15000) public static final java.lang.String PROP_PART_REAP_DELAY
@ConfigBoolean(defaultValue=true) public static final java.lang.String PROP_CALCULATE_DVL
@ConfigString public static final java.lang.String PROP_LOCK_DIR
@ConfigComponent(type=TermStatsFactory.class, mandatory=false) public static final java.lang.String PROP_TERMSTATS_DICT_FACTORY
@ConfigBoolean(defaultValue=false) public static final java.lang.String PROP_REAP_DOES_NOTHING
@ConfigString(defaultValue="", mandatory=false) public static final java.lang.String PROP_STARTING_DATA
Constructor Detail |
---|
public PartitionManager()
PartitionManager
with the given index
configuration.
Method Detail |
---|
public void addIndexListener(IndexListener il)
public void removeIndexListener(IndexListener il)
protected void init()
protected void startHK()
protected java.util.List<java.lang.Integer> readActiveFile() throws java.io.IOException, FileLockException
java.io.IOException
- if there are any errors reading the
active file.
FileLockException
- if there is any error
locking or unlocking the active file.protected void writeActiveFile(java.util.List<DiskPartition> parts) throws java.io.IOException, FileLockException
parts
- The list of partitions or partition numbers to write to
the active file.
java.io.IOException
- if there is an error writing the file.
FileLockException
- if there is any error
locking the active file.public java.util.List<DiskPartition> updateActiveParts(boolean addNew) throws java.lang.Exception
addNew
- Whether newly activated partitions should be added to
the active list immediately.
java.lang.Exception
- if anything goes wrong.protected void addNewPartition(int partNumber, java.util.Set<java.lang.Object> keys)
partNumber
- The number of the partition.keys
- A list of String
s representing the document
keys for the documents indexed into the new partition. They will be
removed from the old partitions.protected void addNewPartition(DiskPartition dp, java.util.Set<java.lang.Object> keys)
public int getNActive()
public java.util.List<DiskPartition> getActivePartitions()
protected DiskPartition newDiskPartition(java.lang.Integer partNum, PartitionManager m) throws java.io.IOException
partNum
- the partition numberm
- the manager
java.io.IOException
- if there is any error opening the partitionpublic java.util.Timer getQueryTimer()
public java.lang.String getIndexDir()
public void setMergeRate(int rate)
rate
- Controls the rate of merging. Must be >= 2. Lower
value leads to fewer partitions, faster searches, more merges,
slower indexingpublic boolean isIndexed(java.lang.String key)
key
- the key for the document that we wish to check.
true
if the document is in the index. A
document is considered to be in the index if a document with the
given key appears in the index and has not been deleted.public void deleteDocument(java.lang.String key)
key
- The document key for the document to be deleted.public void deleteDocuments(java.util.List<java.lang.String> keys)
keys
- The list of keys of the documents to be deleted.protected void deleteKeys(java.util.Set<java.lang.Object> keys, java.util.List parts)
keys
- The list of document keys to delete.parts
- The partitions to delete the keys from.public DocKeyEntry getDocumentTerm(java.lang.String key)
key
- the key for which we want a term from the document dictionary
null
if this key does not appear in
the index or if the associated document has been deletedpublic DocumentVector getDocumentVector(java.lang.String key)
key
- the key of the document for which we want a vector
null
if this key does not appear in the index or if the
associated document has been deleted.public DocumentVector getDocumentVector(java.lang.String key, java.lang.String field)
key
- the key of the document for which we want a vectorfield
- the field for which we want a document vector. If this
parameter is null
, then a vector containing the terms from
all vectored fields in the document is returned. If this value is the empty string, then a
vector for the contents of the document that are not in any field are
returned. If this value is the name of a field that was not vectored
during indexing, an empty vector will be returned.
null
if this key does not appear in the index or if the
associated document has been deleted.public DocumentVector getDocumentVector(java.lang.String key, WeightedField[] fields)
key
- the key of the document for which we want a vectorfields
- the fields for which we want a document vector.
null
if this key does not appear in the index or if the
associated document has been deleted.public ResultSet getSimilar(java.lang.String key, java.lang.String name)
key
- the key of the document to which we'll compute similarity.name
- the name of the field containing the feature vectors that
we'll use in the similarity computation.
public double getDistance(int d1, int d2, java.lang.String name)
public double getDistance(java.lang.String k1, java.lang.String k2, java.lang.String name)
k1
- the first keyk2
- the second keyname
- the name of the feature vector field for which we want the
distance
Double.POSITIVE_INFINITY
is returned.public java.util.SortedSet<FieldValue> getMatching(java.lang.String field, java.lang.String pattern)
field
- the saved, string field against whose values we will match.
If the named field is not saved or is not a string field, then the empty
set will be returned.pattern
- the pattern for which we'll find matching field values.
public FieldIterator getFieldIterator(java.lang.String field)
field
- The name of the field who's values we need an iterator
for.
public FieldIterator getFieldIterator(java.lang.String field, boolean ignoreCase)
field
- The name of the field who's values we need an iterator
for.ignoreCase
- whether the iterator should ignore case when returing
results
public java.util.List getAllFieldValues(java.lang.String field, java.lang.String key)
field
- The name of the field for which we want the values.key
- The key of the document whose values we want.
List
containing values of the appropriate
type. If the named field is not a saved field, or if the given
document key is not in the index, then an empty list is returned.public java.util.List<FieldFrequency> getTopFieldValues(java.lang.String field, int n, boolean ignoreCase)
field
- the name of the field to rankn
- the number of field values to return
List
containing field values of the appropriate
type for the field, ordered by frequencypublic int getNFields()
public java.lang.Object getFieldValue(java.lang.String field, java.lang.String key)
field
- The name of the field for which we want the values.key
- The key of the document whose values we want.
Object
of the appropriate type for the named
field. If the named field is not a saved field, or if the given
document key is not in the index, then null
is
returned.
Note that if there are multiple values for the given field, there is no guarantee which of the values will be returned by this method.
getAllFieldValues(java.lang.String, java.lang.String)
public void reap()
protected void reapPartition(int partNumber)
partNumber
- the number of the partition to reap.public void purge()
public IndexConfig getIndexConfig()
public QueryConfig getQueryConfig()
public int getNDocs()
public long getNTokens()
public int getNTerms()
public void checkHK()
public void shutdown() throws java.io.IOException
java.io.IOException
- if there is an error writing the active
file of partitions or closing one of the partitions.public static void recover(java.lang.String iD) throws java.io.IOException
iD
- The index directory to recover.
java.io.IOException
- if there are any errors recovering the
directoryprotected java.io.File makeActiveFile()
File
for the active file.
protected java.io.File makeTermStatsFile(int tsn)
File
for the global term stats.
tsn
- the number of the term statistics file to make
protected java.io.File makeMetaFile()
File
for the meta file.
public static java.io.File makeDictionaryFile(java.lang.String iD, int partNumber, java.lang.String type)
File
for a dictionary.
iD
- The index directorypartNumber
- The number of the partition for which we're making
a dictionary File
.type
- The dictionary type.
File initialized with an appropriate path.
name.
public java.io.File makeDictionaryFile(int partNumber, java.lang.String type)
File
for a dictionary.
partNumber
- The number of the partition for which we're making
a dictionary File
.type
- the dictionary type
File initialized with an appropriate path.
name.
public static java.io.File makePostingsFile(java.lang.String iD, int partNumber, java.lang.String type, int number)
File
for a postings file.
iD
- The index directorypartNumber
- The number of the partition for which we're making
a postings filetype
- The type of postings filenumber
- The number of the postings file. If this is less than
0, it will be ignored.
File
initialized with an appropriate path.
name.public java.io.File makePostingsFile(int partNumber, java.lang.String type, int number)
File
for a postings file.
partNumber
- The number of the partition for which we're making
a postings filetype
- The type of postings filenumber
- The number of the postings file. If this is less than
0, it will be ignored.
File
initialized with an appropriate path.
name.public java.io.File makePostingsFile(int partNumber, java.lang.String type)
File
for a postings file.
partNumber
- The number of the partition for which we're making
a postings filetype
- The type of postings file
File
initialized with an appropriate path.
name.public static java.io.File makeDeletedDocsFile(java.lang.String iD, int partNumber)
File
for the file containing the bitmap of
deleted documents.
iD
- The index directorypartNumber
- The number of the partition for which we're making a
taxonomy File
.
File
initialized with an appropriate path.
name.public java.io.File makeDeletedDocsFile(int partNumber)
File
for the file containing the bitmap of
deleted documents.
partNumber
- The number of the partition for which we're making a
taxonomy File
.
File
initialized with an appropriate path.
name.public java.io.File makeVectorLengthFile(int partNumber)
File
for the file containing the lengths of
document vectors.
partNumber
- The number of the partition for which we're making a
taxonomy File
.
File
initialized with an appropriate path.
name.public static java.io.File makeVectorLengthFile(java.lang.String iD, int partNumber)
File
for the file containing the lengths of
document vectors.
iD
- The index directorypartNumber
- The number of the partition for which we're making a
removed File
.
File
initialized with an appropriate path.
name.public static java.io.File makeRemovedPartitionFile(java.lang.String iD, int partNumber)
File
that we'll use to indicate that this
partition has been merged away.
iD
- The index directorypartNumber
- The number of the partition for which we're making a
removed File
.
File
initialized with an appropriate path.
name.public java.io.File makeRemovedPartitionFile(int partNumber)
File
that we'll use to indicate that this
partition has been merged away.
partNumber
- The number of the partition for which we're making a
removed File
.
File
initialized with an appropriate path.
name.public static java.io.File makeTaxonomyFile(java.lang.String iD, int partNumber)
public java.io.File makeTaxonomyFile(int partNumber)
public SearchEngine getEngine()
public void setEngine(SearchEngineImpl engine)
engine
- the engine associated with this managerpublic void noMoreMerges()
public DiskPartition mergeAll()
public java.util.List<DiskPartition> mergeGeometric()
null
if none should be merged.setMergeRate(int)
protected DiskPartition merge(java.util.List<DiskPartition> parts)
mergeBlockSize
partitions, ordered by the partition number. We do this in order to
avoid problems with running out of file handles for the files making
up the partitions.
parts
- a list of partitions to merge
protected DiskPartition mergeInPieces(java.util.List<DiskPartition> parts)
mergeBlockSize
, and merges those. This method will work
recursively if there are enough blocks to justify it.
parts
- the partitions that we want to merge
protected DiskPartition realMerge(java.util.List<DiskPartition> diskParts, boolean calculateDVL)
diskParts
- the actual partitions to mergecalculateDVL
- if true
, calculate the document vector
lengths for the documents in the merged partition
protected int getNextPartitionNumber()
protected java.util.List<java.lang.Integer> getPartNumbers(java.util.List<DiskPartition> parts)
parts
- the partitions for which we want the numbers
protected java.util.List<DiskPartition> getPartitions(java.util.List<java.lang.Integer> partNums)
partNums
- the partition numbers for the partitions that we want.
public MetaFile getMetaFile()
public java.util.List<java.lang.String> getFieldNames()
public FieldInfo getFieldInfo(java.lang.String name)
name
- the name of the field for which we want information
null
if there is no field with the given namepublic java.lang.String getName()
public java.util.Date getLastPurgeTime()
public PartitionManager.Merger getMerger()
null
is returned.
Merger
that can be used to merge
these partitions, or null
if no merge is currently
possible.public PartitionManager.Merger getMerger(java.util.List<DiskPartition> l)
l
- A list of partitions to merge, such as that returned by
mergeGeometric
.
Merger
that can be used to merge
these partitions, or null
if no merge is currently
possible.public PartitionManager.Merger getMerger(java.util.List<DiskPartition> l, FileLock localMergeLock)
l
- A list of partitions to merge, such as that returned by
mergeGeometric
.localMergeLock
- the lock to use for running the merge.
Merger
that can be used to merge
these partitions, or null
if no merge is currently
possible.public PartitionManager.Merger getMergerFromNumbers(java.util.List<java.lang.Integer> l)
l
- a list of the numbers of some partitions that we would like to
merge
public int getRandID()
public void newProperties(com.sun.labs.util.props.PropertySheet ps) throws com.sun.labs.util.props.PropertyException
newProperties
in interface com.sun.labs.util.props.Configurable
com.sun.labs.util.props.PropertyException
public boolean isCasedIndex()
true
if the index stores cased information,
false
otherwise.public boolean hasFieldedVectors()
true
if the document vectors for this index contain
field information, false
otherwise.public int getPartCloseDelay()
public void setPartCloseDelay(int partCloseDelay)
public boolean getCalculateDVL()
public java.lang.String getLockDir()
public void setLockDir(java.lang.String lockDir)
public TermStatsDictionary getTermStatsDict()
public TermStatsImpl getTermStats(java.lang.String name)
name
- the name of the term for which we want term statistics
public void recalculateTermStats() throws java.io.IOException, FileLockException
java.io.IOException
- if there is any error writing the new term
stats.
FileLockException
- if there is an error
locking the meta file to get the number for the next term stats dictionary.protected void updateTermStats() throws java.io.IOException, FileLockException
java.io.IOException
FileLockException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |