DocumentImpl (GATE 3.1-2270 API Specification)

Overview

Package

Class

Use

Tree

Deprecated

Index

Help

GATE
Version 3.1-2270

PREV CLASS NEXT CLASS

FRAMES NO FRAMES

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

gate.corpora
Class DocumentImpl

java.lang.Object
  gate.util.AbstractFeatureBearer
      gate.creole.AbstractResource
          gate.creole.AbstractLanguageResource
              gate.corpora.DocumentImpl

All Implemented Interfaces:: Document, CreoleListener, DatastoreListener, LanguageResource, Resource, SimpleDocument, TextualDocument, FeatureBearer, NameBearer, Serializable, Comparable, EventListener

Direct Known Subclasses:: DatabaseDocumentImpl

public class DocumentImpl
extends AbstractLanguageResource
implements TextualDocument, CreoleListener, DatastoreListener
extends AbstractLanguageResource
implements TextualDocument, CreoleListener, DatastoreListener

Represents the commonalities between all sorts of documents.

Editing

The DocumentImpl class implements the Document interface. The DocumentContentImpl class models the textual or audio-visual materials which are the source and content of Documents. The AnnotationSetImpl class supplies annotations on Documents.

Abbreviations:

DC = DocumentContent
D = Document
AS = AnnotationSet

We add an edit method to each of these classes; for DC and AS the methods are package private; D has the public method.

   void edit(Long start, Long end, DocumentContent replacement)
   throws InvalidOffsetException;

D receives edit requests and forwards them to DC and AS. On DC, this method makes a change to the content - e.g. replacing a String range from start to end with replacement. (Deletions are catered for by having replacement = null.) D then calls AS.edit on each of its annotation sets.

On AS, edit calls replacement.size() (i.e. DC.size()) to figure out how long the replacement is (0 for null). It then considers annotations that terminate (start or end) in the altered or deleted range as invalid; annotations that terminate after the range have their offsets adjusted. I.e.:

the nodes that pointed inside the old modified area are invalid now and will be deleted along with the connected annotations;
the nodes that are before the start of the modified area remain untouched;
the nodes that are after the end of the affected area will have the offset changed according to the formula below.

A note re. AS and annotations: annotations no longer have offsets as in the old model, they now have nodes, and nodes have offsets.

To implement AS.edit, we have several indices:

   HashMap annotsByStartNode, annotsByEndNode;

which map node ids to annotations;

   RBTreeMap nodesByOffset;

which maps offset to Nodes.

When we get an edit request, we traverse that part of the nodesByOffset tree representing the altered or deleted range of the DC. For each node found, we delete any annotations that terminate on the node, and then delete the node itself. We then traverse the rest of the tree, changing the offset on all remaining nodes by:

   newOffset =
     oldOffset -
     (
       (end - start) -                                     // size of mod
       ( (replacement == null) ? 0 : replacement.size() )  // size of repl
     );

Note that we use the same convention as e.g. java.lang.String: start offsets are inclusive; end offsets are exclusive. I.e. for string "abcd" range 1-3 = "bc". Examples, for a node with offset 4:

 edit(1, 3, "BC");
 newOffset = 4 - ( (3 - 1) - 2 ) = 4

 edit(1, 3, null);
 newOffset = 4 - ( (3 - 1) - 0 ) = 2

 edit(1, 3, "BBCC");
 newOffset = 4 - ( (3 - 1) - 4 ) = 6

See Also:: Serialized Form

Field Summary
`protected DocumentContent`	`content` The content of the document
`protected AnnotationSet`	`defaultAnnots` The default annotation set
`protected String`	`encoding` The encoding of the source of the document content
`protected Boolean`	`markupAware` Is the document markup-aware?
`protected Map`	`namedAnnotSets` Named sets of annotations
`protected int`	`nextAnnotationId` The id of the next new annotation
`protected int`	`nextNodeId` The id of the next new node
`protected URL`	`sourceUrl` The source URL
`protected Long`	`sourceUrlEndOffset` The end of the range that the content comes from at the source URL (or null if none).
`protected Long`	`sourceUrlStartOffset` The start of the range that the content comes from at the source URL (or null if none).

Fields inherited from class gate.creole.AbstractLanguageResource
`dataStore, lrPersistentId`

Fields inherited from class gate.creole.AbstractResource
`name`

Fields inherited from class gate.util.AbstractFeatureBearer
`features`

Fields inherited from interface gate.Document
`DOCUMENT_ENCODING_PARAMETER_NAME, DOCUMENT_END_OFFSET_PARAMETER_NAME, DOCUMENT_MARKUP_AWARE_PARAMETER_NAME, DOCUMENT_PRESERVE_CONTENT_PARAMETER_NAME, DOCUMENT_REPOSITIONING_PARAMETER_NAME, DOCUMENT_START_OFFSET_PARAMETER_NAME, DOCUMENT_STRING_CONTENT_PARAMETER_NAME`

Fields inherited from interface gate.SimpleDocument
`DOCUMENT_URL_PARAMETER_NAME`

Constructor Summary
`DocumentImpl()` Default construction.

Method Summary
`void`	`addDocumentListener(DocumentListener l)` Adds a `DocumentListener` to this document.
`void`	`cleanup()` Clear all the data members of the object.
`int`	`compareTo(Object o)` Ordering based on URL.toString() and the URL offsets (if any)
`void`	`datastoreClosed(CreoleEvent e)` Called when a `DataStore` has been closed
`void`	`datastoreCreated(CreoleEvent e)` Called when a `DataStore` has been created
`void`	`datastoreOpened(CreoleEvent e)` Called when a `DataStore` has been opened
`void`	`edit(Long start, Long end, DocumentContent replacement)` Propagate edit changes to the document content and annotations.
`protected void`	`fireAnnotationSetAdded(DocumentEvent e)`
`protected void`	`fireAnnotationSetRemoved(DocumentEvent e)`
`protected void`	`fireContentEdited(DocumentEvent e)`
`AnnotationSet`	`getAnnotations()` Get the default set of annotations.
`AnnotationSet`	`getAnnotations(String name)` Get a named set of annotations.
`Set`	`getAnnotationSetNames()` Returns a set of all named annotation sets in existence
`Boolean`	`getCollectRepositioningInfo()` Get the collectiong and preserving of repositioning information for the Document.
`DocumentContent`	`getContent()` The content of the document: a String for text; MPEG for video; etc.
`String`	`getEncoding()` Get the encoding of the document content source
`FeatureMap`	`getFeatures()` Cover unpredictable Features creation
`Boolean`	`getMarkupAware()` Get the markup awareness status of the Document.
`Map`	`getNamedAnnotationSets()` Returns a map with the named annotation sets.
`Integer`	`getNextAnnotationId()` Generate and return the next annotation ID
`Integer`	`getNextNodeId()` Generate and return the next node ID
`protected String`	`getOrderingString()` Utility method to produce a string for comparison in ordering.
`Boolean`	`getPreserveOriginalContent()` Get the preserving of content status of the Document.
`URL`	`getSourceUrl()` Documents are identified by URLs
`Long`	`getSourceUrlEndOffset()` Documents may be packed within files; in this case an optional pair of offsets refer to the location of the document.
`Long[]`	`getSourceUrlOffsets()` Documents may be packed within files; in this case an optional pair of offsets refer to the location of the document.
`Long`	`getSourceUrlStartOffset()` Documents may be packed within files; in this case an optional pair of offsets refer to the location of the document.
`String`	`getStringContent()` The stringContent of a document is a property of the document that will be set when the user wants to create the document from a string, as opposed to from a URL.
`Resource`	`init()` Initialise this resource, and return it.
`boolean`	`isValidOffset(Long offset)` Check that an offset is valid, i.e. it is non-null, greater than or equal to 0 and less than the size of the document content.
`boolean`	`isValidOffsetRange(Long start, Long end)` Check that both start and end are valid offsets and that they constitute a valid offset range, i.e. start is greater than or equal to long.
`static boolean`	`isXmlChar(char ch)` This method decide if a char is a valid XML one or not
`void`	`removeAnnotationSet(String name)` Removes one of the named annotation sets.
`void`	`removeDocumentListener(DocumentListener l)` Removes one of the previously registered document listeners.
`void`	`resourceAdopted(DatastoreEvent evt)` Called by a datastore when a new resource has been adopted
`void`	`resourceDeleted(DatastoreEvent evt)` Called by a datastore when a resource has been deleted
`void`	`resourceLoaded(CreoleEvent e)` Called when a new `Resource` has been loaded into the system
`void`	`resourceRenamed(Resource resource, String oldName, String newName)` Called when the creole register has renamed a resource.1
`void`	`resourceUnloaded(CreoleEvent e)` Called when a `Resource` has been removed from the system
`void`	`resourceWritten(DatastoreEvent evt)` Called by a datastore when a resource has been wrote into the datastore
`void`	`setCollectRepositioningInfo(Boolean b)` Allow/disallow collecting of repositioning information.
`void`	`setContent(DocumentContent content)` Set method for the document content
`void`	`setDataStore(DataStore dataStore)` Set the data store that this LR lives in.
`void`	`setDefaultAnnotations(AnnotationSet defaultAnnotations)` This method added by Shafirin Andrey, to allow access to protected member `defaultAnnots` Required for JAPE-Debugger.
`void`	`setEncoding(String encoding)` Set the encoding of the document content source
`void`	`setLRPersistenceId(Object lrID)` Sets the persistence id of this LR.
`void`	`setMarkupAware(Boolean newMarkupAware)` Make the document markup-aware.
`void`	`setNextAnnotationId(int aNextAnnotationId)` Sets the nextAnnotationId
`void`	`setPreserveOriginalContent(Boolean b)` Allow/disallow preserving of the original document content.
`void`	`setSourceUrl(URL sourceUrl)` Set method for the document's URL
`void`	`setSourceUrlEndOffset(Long sourceUrlEndOffset)` Documents may be packed within files; in this case an optional pair of offsets refer to the location of the document.
`void`	`setSourceUrlStartOffset(Long sourceUrlStartOffset)` Documents may be packed within files; in this case an optional pair of offsets refer to the location of the document.
`void`	`setStringContent(String stringContent)` The stringContent of a document is a property of the document that will be set when the user wants to create the document from a string, as opposed to from a URL.
`String`	`toString()` String respresentation
`String`	`toXml()` Returns a GateXml document that is a custom XML format for wich there is a reader inside GATE called gate.xml.GateFormatXmlHandler.
`String`	`toXml(Set aSourceAnnotationSet)` Returns an XML document aming to preserve the original markups( the original markup will be in the same place and format as it was before processing the document) and include (if possible) the annotations specified in the aSourceAnnotationSet.
`String`	`toXml(Set aSourceAnnotationSet, boolean includeFeatures)` Returns an XML document aming to preserve the original markups( the original markup will be in the same place and format as it was before processing the document) and include (if possible) the annotations specified in the aSourceAnnotationSet.

Methods inherited from class gate.creole.AbstractLanguageResource
`getDataStore, getLRPersistenceId, getParent, isModified, setParent, sync`

Methods inherited from class gate.creole.AbstractResource
`checkParameterValues, getBeanInfo, getName, getParameterValue, getParameterValue, removeResourceListeners, setName, setParameterValue, setParameterValue, setParameterValues, setParameterValues, setResourceListeners`

Methods inherited from class gate.util.AbstractFeatureBearer
`setFeatures`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Methods inherited from interface gate.LanguageResource
`getDataStore, getLRPersistenceId, getParent, isModified, setParent, sync`

Methods inherited from interface gate.Resource
`getParameterValue, setParameterValue, setParameterValues`

Methods inherited from interface gate.util.FeatureBearer
`setFeatures`

Methods inherited from interface gate.util.NameBearer
`getName, setName`

Field Detail

nextAnnotationId

protected int nextAnnotationId

The id of the next new annotation

nextNodeId

protected int nextNodeId

The id of the next new node

sourceUrl

protected URL sourceUrl

The source URL

content

protected DocumentContent content

The content of the document

encoding

protected String encoding

The encoding of the source of the document content

sourceUrlStartOffset

protected Long sourceUrlStartOffset

The start of the range that the content comes from at the source URL (or null if none).

sourceUrlEndOffset

protected Long sourceUrlEndOffset

The end of the range that the content comes from at the source URL (or null if none).

defaultAnnots

protected AnnotationSet defaultAnnots

The default annotation set

namedAnnotSets

protected Map namedAnnotSets

Named sets of annotations

markupAware

protected Boolean markupAware

Is the document markup-aware?

Constructor Detail

DocumentImpl

public DocumentImpl()

Default construction. Content left empty.

Method Detail

getFeatures

public FeatureMap getFeatures()

Cover unpredictable Features creation

Specified by:: getFeatures in interface FeatureBearer
Overrides:: getFeatures in class AbstractFeatureBearer

init

public Resource init()
              throws ResourceInstantiationException

Initialise this resource, and return it.

Specified by:: init in interface Resource
Overrides:: init in class AbstractResource

Throws:: ResourceInstantiationException

cleanup

public void cleanup()

Clear all the data members of the object.

Specified by:: cleanup in interface Resource
Overrides:: cleanup in class AbstractLanguageResource

getSourceUrl

public URL getSourceUrl()

Documents are identified by URLs

Specified by:: getSourceUrl in interface SimpleDocument

setSourceUrl

public void setSourceUrl(URL sourceUrl)

Set method for the document's URL

Specified by:: setSourceUrl in interface SimpleDocument

getSourceUrlOffsets

public Long[] getSourceUrlOffsets()

Documents may be packed within files; in this case an optional pair of offsets refer to the location of the document.

Specified by:: getSourceUrlOffsets in interface Document

setPreserveOriginalContent

public void setPreserveOriginalContent(Boolean b)

Allow/disallow preserving of the original document content. If is true the original content will be retrieved from the DocumentContent object and preserved as document feature.

Specified by:: setPreserveOriginalContent in interface Document

getPreserveOriginalContent

public Boolean getPreserveOriginalContent()

Get the preserving of content status of the Document.

Specified by:: getPreserveOriginalContent in interface Document

Returns:: whether the Document should preserve it's original content.

setCollectRepositioningInfo

public void setCollectRepositioningInfo(Boolean b)

Allow/disallow collecting of repositioning information. If is true information will be retrieved and preserved as document feature.
Preserving of repositioning information give the possibilities for converting of coordinates between the original document content and extracted from the document text.

Specified by:: setCollectRepositioningInfo in interface Document

getCollectRepositioningInfo

public Boolean getCollectRepositioningInfo()

Get the collectiong and preserving of repositioning information for the Document.
Preserving of repositioning information give the possibilities for converting of coordinates between the original document content and extracted from the document text.

Specified by:: getCollectRepositioningInfo in interface Document

Returns:: whether the Document should collect and preserve information.

getSourceUrlStartOffset

public Long getSourceUrlStartOffset()

Documents may be packed within files; in this case an optional pair of offsets refer to the location of the document. This method gets the start offset.

Specified by:: getSourceUrlStartOffset in interface Document

setSourceUrlStartOffset

public void setSourceUrlStartOffset(Long sourceUrlStartOffset)

Documents may be packed within files; in this case an optional pair of offsets refer to the location of the document. This method sets the start offset.

Specified by:: setSourceUrlStartOffset in interface Document

getSourceUrlEndOffset

public Long getSourceUrlEndOffset()

Documents may be packed within files; in this case an optional pair of offsets refer to the location of the document. This method gets the end offset.

Specified by:: getSourceUrlEndOffset in interface Document

setSourceUrlEndOffset

public void setSourceUrlEndOffset(Long sourceUrlEndOffset)

Documents may be packed within files; in this case an optional pair of offsets refer to the location of the document. This method sets the end offset.

Specified by:: setSourceUrlEndOffset in interface Document

getContent

public DocumentContent getContent()

The content of the document: a String for text; MPEG for video; etc.

Specified by:: getContent in interface SimpleDocument

setContent

public void setContent(DocumentContent content)

Set method for the document content

Specified by:: setContent in interface SimpleDocument

getEncoding

public String getEncoding()

Get the encoding of the document content source

Specified by:: getEncoding in interface TextualDocument

Returns:: a String value.

setEncoding

public void setEncoding(String encoding)

Set the encoding of the document content source

getAnnotations

public AnnotationSet getAnnotations()

Get the default set of annotations. The set is created if it doesn't exist yet.

Specified by:: getAnnotations in interface SimpleDocument

getAnnotations

public AnnotationSet getAnnotations(String name)

Get a named set of annotations. Creates a new set if one with this name doesn't exist yet. If the provided name is null then it returns the default annotation set.

Specified by:: getAnnotations in interface SimpleDocument

setMarkupAware

public void setMarkupAware(Boolean newMarkupAware)

Make the document markup-aware. This will trigger the creation of a DocumentFormat object at Document initialisation time; the DocumentFormat object will unpack the markup in the Document and add it as annotations. Documents are not markup-aware by default.

Specified by:: setMarkupAware in interface Document

Parameters:: newMarkupAware - markup awareness status.

getMarkupAware

public Boolean getMarkupAware()

Get the markup awareness status of the Document. Documents are markup-aware by default.

Specified by:: getMarkupAware in interface Document

Returns:: whether the Document is markup aware.

toXml

public String toXml(Set aSourceAnnotationSet)

Returns an XML document aming to preserve the original markups( the original markup will be in the same place and format as it was before processing the document) and include (if possible) the annotations specified in the aSourceAnnotationSet. It is equivalent to toXml(aSourceAnnotationSet, true).

Specified by:: toXml in interface Document

toXml

public String toXml(Set aSourceAnnotationSet,
                    boolean includeFeatures)

Returns an XML document aming to preserve the original markups( the original markup will be in the same place and format as it was before processing the document) and include (if possible) the annotations specified in the aSourceAnnotationSet. Warning: Annotations from the aSourceAnnotationSet will be lost if they will cause a crosed over situation.

Specified by:: toXml in interface Document

Parameters:: aSourceAnnotationSet - is an annotation set containing all the annotations that will be combined with the original marup set. If the param is null it will only dump the original markups.; includeFeatures - is a boolean that controls whether the annotation features should be included or not. If false, only the annotation type is included in the tag.
Returns:: a string representing an XML document containing the original markup + dumped annotations form the aSourceAnnotationSet

toXml

public String toXml()

Returns a GateXml document that is a custom XML format for wich there is a reader inside GATE called gate.xml.GateFormatXmlHandler. What it does is to serialize a GATE document in an XML format.

Specified by:: toXml in interface Document

Returns:: a string representing a Gate Xml document.

isXmlChar

public static boolean isXmlChar(char ch)

This method decide if a char is a valid XML one or not

Parameters:: ch - the char to be tested
Returns:: true if is a valid XML char and fals if is not.

getNamedAnnotationSets

public Map getNamedAnnotationSets()

Returns a map with the named annotation sets. It returns null if no named annotaton set exists.

Specified by:: getNamedAnnotationSets in interface Document

getAnnotationSetNames

public Set getAnnotationSetNames()

Returns a set of all named annotation sets in existence

Specified by:: getAnnotationSetNames in interface SimpleDocument

removeAnnotationSet

public void removeAnnotationSet(String name)

Removes one of the named annotation sets. Note that the default annotation set cannot be removed.

Specified by:: removeAnnotationSet in interface SimpleDocument

Parameters:: name - the name of the annotation set to be removed

edit

public void edit(Long start,
                 Long end,
                 DocumentContent replacement)
          throws InvalidOffsetException

Propagate edit changes to the document content and annotations.

Specified by:: edit in interface Document

Throws:: InvalidOffsetException

isValidOffset

public boolean isValidOffset(Long offset)

Check that an offset is valid, i.e. it is non-null, greater than or equal to 0 and less than the size of the document content.

isValidOffsetRange

public boolean isValidOffsetRange(Long start,
                                  Long end)

Check that both start and end are valid offsets and that they constitute a valid offset range, i.e. start is greater than or equal to long.

setNextAnnotationId

public void setNextAnnotationId(int aNextAnnotationId)

Sets the nextAnnotationId

getNextAnnotationId

public Integer getNextAnnotationId()

Generate and return the next annotation ID

getNextNodeId

public Integer getNextNodeId()

Generate and return the next node ID

compareTo

public int compareTo(Object o)
              throws ClassCastException

Ordering based on URL.toString() and the URL offsets (if any)

Specified by:: compareTo in interface Comparable

Throws:: ClassCastException

getOrderingString

protected String getOrderingString()

Utility method to produce a string for comparison in ordering. String is based on the source URL and offsets.

getStringContent

public String getStringContent()

The stringContent of a document is a property of the document that will be set when the user wants to create the document from a string, as opposed to from a URL. Use the getContent method instead to get the actual document content.

setStringContent

public void setStringContent(String stringContent)

The stringContent of a document is a property of the document that will be set when the user wants to create the document from a string, as opposed to from a URL. Use the setContent method instead to update the actual document content.

toString

public String toString()

String respresentation

Overrides:: toString in class Object

removeDocumentListener

public void removeDocumentListener(DocumentListener l)

Description copied from interface: Document

Removes one of the previously registered document listeners.

Specified by:: removeDocumentListener in interface Document

addDocumentListener

public void addDocumentListener(DocumentListener l)

Description copied from interface: Document

Adds a DocumentListener to this document. All the registered listeners will be notified of changes occured to the document.

Specified by:: addDocumentListener in interface Document

fireAnnotationSetAdded

protected void fireAnnotationSetAdded(DocumentEvent e)

fireAnnotationSetRemoved

protected void fireAnnotationSetRemoved(DocumentEvent e)