com.screenscraper.scraper

Class ExtractorPatternToken

java.lang.Object

com.screenscraper.scraper.ExtractorPatternToken

public class ExtractorPatternToken
extends java.lang.Object

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	BEGIN_TOKEN_DELIMITER The token delimiters.
static java.lang.String	END_TOKEN_DELIMITER
static java.lang.String	IGNORE_REGULAR_EXPRESSION The regular expression used to designate areas to ignore.
static java.lang.String	IGNORE_TOKEN The regex for matching a token in the extractor pattern text
static java.lang.String	IGNORE_TOKEN_WORD A special token identifier used to designate data that should be ignored.
static java.lang.String	LITERALIZED_BEGIN_TOKEN_DELIMITER
static java.lang.String	LITERALIZED_END_TOKEN_DELIMITER
static java.lang.String	LITERALIZED_TOKEN_REGULAR_EXPRESSION The string literalized regular expression used to extract tokens.
static org.apache.log4j.Logger	log Used for logging.
static java.lang.String	REGULAR_EXPRESSION The regular expression used to extract tokens.
static java.util.regex.Pattern	REGULAR_EXPRESSION_PATTERN
static java.lang.String	REQUIRED_REGULAR_EXPRESSION The regular expression used to extract required data.

Constructor Summary

Constructors

Constructor and Description
ExtractorPatternToken() Generates an empty extractor pattern token.
ExtractorPatternToken(ExtractorPatternToken extractorPatternToken) Creates a new token by copying another.
ExtractorPatternToken(int extractorPatternID, java.lang.String identifier, boolean excludeFromData, int sequence, boolean saveInSessionVariable, java.lang.String regularExpression, boolean compoundKey, boolean stripHTML, boolean resolveRelativeUrl, boolean replaceHTMLEntities, boolean trimWhiteSpace, boolean nullSessionVariable, int mappingSetID) Constructs the token with all of its data.
ExtractorPatternToken(java.lang.String identifier, int sequence) Constructs the token with most of its data.
ExtractorPatternToken(java.lang.String identifier, int sequence, java.lang.String leftSide, java.lang.String rightSide, java.lang.String originalText) Constructs the token, causing it to guess the regular expression to use.

Method Summary

Modifier and Type	Method and Description
void	dumpToLog() Dumps the extractor pattern token to the log.
boolean	getCompoundKey() Indicates whether or not the token should be used in the compound key of the record.
boolean	getExcludeFromData() Indicates whether or not the token is optional.
int	getExtractorPatternID() Gets the ID of the extractor pattern holding this token.
java.lang.String	getIdentifier() Gets the string that identifies the token.
MappingSet	getMappingSet() Gets the MappingSet associated with this token.
boolean	getNullSessionVariable() Indicates whether the session variable corresponding to this token should be nulled out if no matches are found.
int	getNumGroups() returns the number of groups defined in the regular expression of the token
java.lang.String	getRegularExpression() Used to specify a regular expression to do the extracting, rather than relying on the app to generate one.
boolean	getReplaceHTMLEntities() Indicates whether HTML Entities should be replaced in the pattern
boolean	getResolveRelativeURL() Indicates whether a relative url should be changed to a complete URL
boolean	getSaveInSessionVariable() Indicates whether or not the data extracted by this token should be saved in a session variable.
int	getSequence() Gets the sequence in which the token pattern should appear.
boolean	getStripHTML() Indicates whether or not HTML should be stripped
boolean	getTrimWhiteSpace() Indicates whether white space should be trimmed from the extracted value.
boolean	isDataRecord()
boolean	isOptional()
protected void	save() Saves the extractor pattern token to the database.
void	setCompoundKey(boolean compoundKey) Determines whether or not the token should be used in the compound key of the record.
void	setExcludeFromData(boolean bool) Sets if token should be excluded from matched data.
protected void	setExtractorPatternID(int extractorPatternID) Sets the ID of the extractor pattern holding this token.
void	setIdentifier(java.lang.String identifier) Sets the string that identifies the token.
void	setMappingSet(MappingSet mappingSet) Sets the MappingSet associated with this token.
void	setNullSessionVariable(boolean nullSessionVariable) Determines whether the session variable corresponding to this token should be nulled out if no matches are found.
void	setRegularExpression(java.lang.String regularExpression) Used to specify a regular expression to do the extracting, rather than relying on the app to generate one.
void	setReplaceHTMLEntities(boolean replaceHTMLEntities) Indicates whether HTML Entities should be replaced in the pattern
void	setResolveRelativeURL(boolean evaluateRelativeURL) Indicates whether a relative url should be changed to a complete URL
void	setSaveInSessionVariable(boolean saveInSessionVariable) Determines whether or not the data extracted by this token should be saved in a session variable.
void	setSequence(int sequence) Sets the sequence in which the script should appear.
void	setStripHTML(boolean stripHTML) Determines whether or not HTML should be stripped
void	setTrimWhiteSpace(boolean trimWhiteSpace) Determines whether white space should be trimmed from the extracted value.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

public static org.apache.log4j.Logger log

Used for logging.

REGULAR_EXPRESSION

public static final java.lang.String REGULAR_EXPRESSION

The regular expression used to extract tokens.

See Also:

Constant Field Values

REGULAR_EXPRESSION_PATTERN

public static final java.util.regex.Pattern REGULAR_EXPRESSION_PATTERN

BEGIN_TOKEN_DELIMITER

@NotNull
public static java.lang.String BEGIN_TOKEN_DELIMITER

The token delimiters.

END_TOKEN_DELIMITER

@NotNull
public static java.lang.String END_TOKEN_DELIMITER

LITERALIZED_BEGIN_TOKEN_DELIMITER

@NotNull
public static java.lang.String LITERALIZED_BEGIN_TOKEN_DELIMITER

LITERALIZED_END_TOKEN_DELIMITER

@NotNull
public static java.lang.String LITERALIZED_END_TOKEN_DELIMITER

REQUIRED_REGULAR_EXPRESSION

public static final java.lang.String REQUIRED_REGULAR_EXPRESSION

The regular expression used to extract required data.

See Also:

Constant Field Values

IGNORE_REGULAR_EXPRESSION

public static final java.lang.String IGNORE_REGULAR_EXPRESSION

The regular expression used to designate areas to ignore.

See Also:

Constant Field Values

IGNORE_TOKEN_WORD

public static final java.lang.String IGNORE_TOKEN_WORD

A special token identifier used to designate data that should be ignored.

See Also:

Constant Field Values

IGNORE_TOKEN

public static final java.lang.String IGNORE_TOKEN

The regex for matching a token in the extractor pattern text

LITERALIZED_TOKEN_REGULAR_EXPRESSION

@NotNull
public static java.lang.String LITERALIZED_TOKEN_REGULAR_EXPRESSION

The string literalized regular expression used to extract tokens.

Constructor Detail

ExtractorPatternToken

public ExtractorPatternToken()

Generates an empty extractor pattern token.

ExtractorPatternToken

public ExtractorPatternToken(java.lang.String identifier,
                             int sequence)

Constructs the token with most of its data.

Parameters:

identifier - The token identifier.

sequence - The sequence in which the pattern token should appear.

ExtractorPatternToken

public ExtractorPatternToken(java.lang.String identifier,
                             int sequence,
                             @Nullable
                             java.lang.String leftSide,
                             @Nullable
                             java.lang.String rightSide,
                             java.lang.String originalText)

Constructs the token, causing it to guess the regular expression to use.

Parameters:

identifier - The token identifier.

sequence - The sequence in which the pattern token should appear.

leftSide - The string left of the extractor that immediately precedes the token in its containing extractor pattern.

rightSide - The string right of the extractor that immediately follows the token in its containing extractor pattern.

originalText - The original text in the pattern this token is replacing.

ExtractorPatternToken

public ExtractorPatternToken(int extractorPatternID,
                             java.lang.String identifier,
                             boolean excludeFromData,
                             int sequence,
                             boolean saveInSessionVariable,
                             java.lang.String regularExpression,
                             boolean compoundKey,
                             boolean stripHTML,
                             boolean resolveRelativeUrl,
                             boolean replaceHTMLEntities,
                             boolean trimWhiteSpace,
                             boolean nullSessionVariable,
                             int mappingSetID)

Constructs the token with all of its data.

Parameters:

extractorPatternID - The ID of the extractor pattern containing this token.

identifier - The token identifier.

excludeFromData - Whether or not the token should be excluded from matched DataSet/DataRecord

sequence - The sequence in which the pattern token should appear.

saveInSessionVariable - Whether or not to save the value extracted by this token in a session variable.

regularExpression - Specifies a regular expression instead of having the system determine one.

compoundKey - Indicates whether or not the token helps comprise the compound key for the record.

stripHTML - Whether or not HTML should be stripped.

resolveRelativeUrl - Whether or not the relative URL should be resolved.

replaceHTMLEntities - Whether or not HTML entities should be replaced.

trimWhiteSpace - Whether or not white space should be trimmed.

nullSessionVariable - Whether or not the session variable should be nulled.

mappingSetID - The database ID of the mapping set.

ExtractorPatternToken

public ExtractorPatternToken(@NotNull
                             ExtractorPatternToken extractorPatternToken)

Creates a new token by copying another.

Parameters:

extractorPatternToken - The ExtractorPatternToken to copy.

Method Detail

getIdentifier

@Nullable
public java.lang.String getIdentifier()

Gets the string that identifies the token.

Returns:

The token identifier.

setIdentifier

public void setIdentifier(java.lang.String identifier)

Sets the string that identifies the token.

Parameters:

identifier - The token identifier.

getExcludeFromData

public boolean getExcludeFromData()

Indicates whether or not the token is optional.

Returns:

A boolean value.

isOptional

public boolean isOptional()

setExcludeFromData

public void setExcludeFromData(boolean bool)

Sets if token should be excluded from matched data.

Parameters:

bool - A boolean value.

isDataRecord

public boolean isDataRecord()

getSequence

public int getSequence()

Gets the sequence in which the token pattern should appear.

Returns:

The sequence.

setSequence

public void setSequence(int sequence)

Sets the sequence in which the script should appear.

Parameters:

sequence - The sequence.

getExtractorPatternID

public int getExtractorPatternID()

Gets the ID of the extractor pattern holding this token.

Returns:

The ID.

setExtractorPatternID

protected void setExtractorPatternID(int extractorPatternID)

Sets the ID of the extractor pattern holding this token.

Parameters:

extractorPatternID - The ID.

getSaveInSessionVariable

public boolean getSaveInSessionVariable()

Indicates whether or not the data extracted by this token should be saved in a session variable.

Returns:

A boolean.

setSaveInSessionVariable

public void setSaveInSessionVariable(boolean saveInSessionVariable)

Determines whether or not the data extracted by this token should be saved in a session variable.

Parameters:

saveInSessionVariable - A boolean.

getRegularExpression

@Nullable
public java.lang.String getRegularExpression()

Used to specify a regular expression to do the extracting, rather than relying on the app to generate one.

Returns:

The regular expression.

setRegularExpression

public void setRegularExpression(@Nullable
                                 java.lang.String regularExpression)

Used to specify a regular expression to do the extracting, rather than relying on the app to generate one.

Parameters:

regularExpression - The regular expression.

getCompoundKey

public boolean getCompoundKey()

Indicates whether or not the token should be used in the compound key of the record.

Returns:

A boolean.

setCompoundKey

public void setCompoundKey(boolean compoundKey)

Determines whether or not the token should be used in the compound key of the record.

Parameters:

compoundKey - A boolean.

getStripHTML

public boolean getStripHTML()

Indicates whether or not HTML should be stripped

Returns:

A boolean.

setStripHTML

public void setStripHTML(boolean stripHTML)

Determines whether or not HTML should be stripped

Parameters:

stripHTML - A boolean.

getMappingSet

@Nullable
public MappingSet getMappingSet()

Gets the MappingSet associated with this token.

Returns:

The MappingSet.

getResolveRelativeURL

public boolean getResolveRelativeURL()

Indicates whether a relative url should be changed to a complete URL

Returns:

The resolveRelativeURL.

setResolveRelativeURL

public void setResolveRelativeURL(boolean evaluateRelativeURL)

Indicates whether a relative url should be changed to a complete URL

Parameters:

evaluateRelativeURL - A boolean.

getReplaceHTMLEntities

public boolean getReplaceHTMLEntities()

Indicates whether HTML Entities should be replaced in the pattern

Returns:

A boolean.

setReplaceHTMLEntities

public void setReplaceHTMLEntities(boolean replaceHTMLEntities)

Indicates whether HTML Entities should be replaced in the pattern

Parameters:

replaceHTMLEntities - A boolean.

getTrimWhiteSpace

public boolean getTrimWhiteSpace()

Indicates whether white space should be trimmed from the extracted value.

Returns:

A boolean.

setTrimWhiteSpace

public void setTrimWhiteSpace(boolean trimWhiteSpace)

Determines whether white space should be trimmed from the extracted value.

Parameters:

trimWhiteSpace - A boolean.

getNullSessionVariable

public boolean getNullSessionVariable()

Indicates whether the session variable corresponding to this token should be nulled out if no matches are found.

Returns:

A boolean.

setNullSessionVariable

public void setNullSessionVariable(boolean nullSessionVariable)

Determines whether the session variable corresponding to this token should be nulled out if no matches are found.

Parameters:

nullSessionVariable - A boolean.

setMappingSet

public void setMappingSet(@Nullable
                          MappingSet mappingSet)

Sets the MappingSet associated with this token.

Parameters:

mappingSet - The MappingSet.

save

protected void save()

Saves the extractor pattern token to the database.

dumpToLog

public void dumpToLog()

Dumps the extractor pattern token to the log.

getNumGroups

public int getNumGroups()

returns the number of groups defined in the regular expression of the token

Returns:

The number of groups defined in the RegEx of the token