|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectweka.core.tokenizers.Tokenizer
public abstract class Tokenizer
A superclass for all tokenizer algorithms.
Constructor Summary | |
---|---|
Tokenizer()
|
Method Summary | |
---|---|
java.lang.String[] |
getOptions()
Gets the current option settings for the OptionHandler. |
abstract java.lang.String |
globalInfo()
Returns a string describing the stemmer |
abstract boolean |
hasMoreElements()
Tests if this enumeration contains more elements. |
java.util.Enumeration |
listOptions()
Returns an enumeration of all the available options.. |
abstract java.lang.Object |
nextElement()
Returns the next element of this enumeration if this enumeration object has at least one more element to provide. |
static void |
runTokenizer(Tokenizer tokenizer,
java.lang.String[] options)
initializes the given tokenizer with the given options and runs the tokenizer over all the remaining strings in the options array. |
void |
setOptions(java.lang.String[] options)
Sets the OptionHandler's options using the given list. |
abstract void |
tokenize(java.lang.String s)
Sets the string to tokenize. |
static java.lang.String[] |
tokenize(Tokenizer tokenizer,
java.lang.String[] options)
initializes the given tokenizer with the given options and runs the tokenizer over all the remaining strings in the options array. |
Methods inherited from class java.lang.Object |
---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public Tokenizer()
Method Detail |
---|
public abstract java.lang.String globalInfo()
public java.util.Enumeration listOptions()
listOptions
in interface OptionHandler
public java.lang.String[] getOptions()
getOptions
in interface OptionHandler
public void setOptions(java.lang.String[] options) throws java.lang.Exception
setOptions
in interface OptionHandler
options
- the list of options as an array of strings
java.lang.Exception
- if an option is not supportedpublic abstract boolean hasMoreElements()
hasMoreElements
in interface java.util.Enumeration
public abstract java.lang.Object nextElement()
nextElement
in interface java.util.Enumeration
public abstract void tokenize(java.lang.String s)
s
- the string to tokenizepublic static java.lang.String[] tokenize(Tokenizer tokenizer, java.lang.String[] options) throws java.lang.Exception
tokenizer
- the tokenizer to useoptions
- the options for the tokenizer
java.lang.Exception
- if setting of options or tokenization failspublic static void runTokenizer(Tokenizer tokenizer, java.lang.String[] options)
tokenizer
- the tokenizer to useoptions
- the options for the tokenizer
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |