|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object com.sun.labs.minion.document.tokenizer.HandyTokenizer
public class HandyTokenizer
A helper class to tokenize strings and return an iterator for the results.
Field Summary | |
---|---|
protected char[] |
c
Characters to tokenize. |
protected TokenCollectorStage |
tcs
A token collecting stage. |
protected UniversalTokenizer |
tok
A tokenizer. |
Constructor Summary | |
---|---|
HandyTokenizer()
Creates a new handy tokenizer. |
Method Summary | |
---|---|
java.lang.String[] |
tokenizeToArray(java.lang.String s)
Tokenize the given string and hand back an array of the tokens it produces. |
java.util.Iterator |
tokenizeToIterator(java.lang.String s)
Tokenize the given string and hand back an iterator for the tokens it produces. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected TokenCollectorStage tcs
protected UniversalTokenizer tok
protected char[] c
Constructor Detail |
---|
public HandyTokenizer()
Method Detail |
---|
public java.util.Iterator tokenizeToIterator(java.lang.String s)
s
- The string to tokenize.
Iterator
for the tokens that were
generated.public java.lang.String[] tokenizeToArray(java.lang.String s)
s
- The string to tokenize.
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |