org.apache.lucene.analysis
Class WhitespaceTokenizer
java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.Tokenizer
org.apache.lucene.analysis.CharTokenizer
org.apache.lucene.analysis.WhitespaceTokenizer
- All Implemented Interfaces:
- Closeable
public class WhitespaceTokenizer
- extends CharTokenizer
A WhitespaceTokenizer is a tokenizer that divides text at whitespace.
Adjacent sequences of non-Whitespace characters form tokens.
Fields inherited from class org.apache.lucene.analysis.Tokenizer |
input |
Methods inherited from class org.apache.lucene.util.AttributeSource |
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString |
WhitespaceTokenizer
public WhitespaceTokenizer(Reader in)
- Construct a new WhitespaceTokenizer.
WhitespaceTokenizer
public WhitespaceTokenizer(AttributeSource source,
Reader in)
- Construct a new WhitespaceTokenizer using a given
AttributeSource
.
WhitespaceTokenizer
public WhitespaceTokenizer(AttributeSource.AttributeFactory factory,
Reader in)
- Construct a new WhitespaceTokenizer using a given
AttributeSource.AttributeFactory
.
isTokenChar
protected boolean isTokenChar(char c)
- Collects only characters which do not satisfy
Character.isWhitespace(char)
.
- Specified by:
isTokenChar
in class CharTokenizer
Copyright © 2000-2010 Apache Software Foundation. All Rights Reserved.