com.screenscraper.scraper

Interface RunningScrapeableFile

All Known Implementing Classes:

ScrapeableFile

public interface RunningScrapeableFile

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.log4j.Logger	log Used for logging.

Method Summary

Modifier and Type	Method and Description
void	addHTTPParameter(HTTPParameter httpParameter) Adds a parameter to the scrapeable file.
DataSet	extractData(java.lang.String text, java.lang.String name) Manually extracts data using an existing ExtractorPattern.
java.lang.String	extractOneValue(java.lang.String text, java.lang.String name) Manually extracts one value from data using an existing ExtractorPattern.
java.lang.String	extractOneValue(java.lang.String text, java.lang.String name, java.lang.String token) Manually extracts one value from data using an existing ExtractorPattern.
java.lang.String	getContentAsString() Gets the scraped content as a string.
java.lang.String	getCurrentPOSTData() Gets the current value of the POST data.
java.lang.String	getCurrentURL() Depending on when this value is requested it will return either the unresolved or resolved URL of the file to be scraped.
java.lang.String	getName() The name
java.lang.String	getStrippedContentAsString() Gets the scraped content as a string, with all new line characters removed
boolean	noExtractorPatternsMatched() Indicates whether or not none of the extractor patterns for this scrapeable file found a match.
void	removeAllHTTPParameters() Removes all HTTP parameters from the scrapeable file.
java.lang.String	resolveRelativeURL(java.lang.String urlToResolve) Resolve a relative URL
void	saveFileOnRequest(java.lang.String fileToSaveToOnRequest) Sets the path to save the file to just after it gets requested.
boolean	wasErrorOnRequest() Indicates whether or not an error occurred in requesting the file.

Field Detail

log

static final org.apache.log4j.Logger log

Used for logging.

Method Detail

getContentAsString

@Nullable
java.lang.String getContentAsString()

Gets the scraped content as a string.

Returns:

The content.

getStrippedContentAsString

@Nullable
java.lang.String getStrippedContentAsString()

Gets the scraped content as a string, with all new line characters removed

Returns:

The content

wasErrorOnRequest

boolean wasErrorOnRequest()

Indicates whether or not an error occurred in requesting the file.

Returns:

A boolean.

noExtractorPatternsMatched

boolean noExtractorPatternsMatched()

Indicates whether or not none of the extractor patterns for this scrapeable file found a match.

Returns:

A boolean.

getCurrentURL

@Nullable
java.lang.String getCurrentURL()

Depending on when this value is requested it will return either the unresolved or resolved URL of the file to be scraped.

Returns:

The URL.

getName

@Nullable
java.lang.String getName()

The name

Returns:

The name.

getCurrentPOSTData

@Nullable
java.lang.String getCurrentPOSTData()

Gets the current value of the POST data. Note that this is only used by RunningScrapeableFile so that the resolved POST data can be made accessible in a script. If this method is called before this scrapeable file is scraped it will return null.

Returns:

The resolved POST data.

removeAllHTTPParameters

void removeAllHTTPParameters()
                      throws java.lang.Exception

Removes all HTTP parameters from the scrapeable file.

Throws:

java.lang.Exception

addHTTPParameter

void addHTTPParameter(HTTPParameter httpParameter)

Adds a parameter to the scrapeable file.

Parameters:

httpParameter - An HTTPParameter.

resolveRelativeURL

@Nullable
java.lang.String resolveRelativeURL(java.lang.String urlToResolve)

Resolve a relative URL

Parameters:

urlToResolve - The relative URL to resolve.

Returns:

A string with the resolved URL or the original URL if it couldn't resolve

extractData

@Nullable
DataSet extractData(java.lang.String text,
                               java.lang.String name)

Manually extracts data using an existing ExtractorPattern.

Parameters:

text - The text to extract data from.

name - The name of the ExtractorPattern to be used. The ExtractorPattern must be associated with this ScrapeableFile. If more than one ExtractorPattern is found with the given name the first (by sequence) will be used.

Returns:

A DataSet containing the extracted data.

extractOneValue

@Nullable
java.lang.String extractOneValue(java.lang.String text,
                                            java.lang.String name)

Manually extracts one value from data using an existing ExtractorPattern.

Parameters:

text - The text to extract data from.

name - The name of the ExtractorPattern to be used. The ExtractorPattern must be associated with this ScrapeableFile. If more than one ExtractorPattern is found with the given name the first (by sequence) will be used.

Returns:

A String containing the extracted data. If nothing mathces null is returned.

extractOneValue

@Nullable
java.lang.String extractOneValue(java.lang.String text,
                                            java.lang.String name,
                                            java.lang.String token)

Manually extracts one value from data using an existing ExtractorPattern.

Parameters:

text - The text to extract data from.

name - The name of the ExtractorPattern to be used. The ExtractorPattern must be associated with this ScrapeableFile. If more than one ExtractorPattern is found with the given name the first (by sequence) will be used.

token - The name of the Token to return

Returns:

A String containing the extracted data. If nothing mathces null is returned.

saveFileOnRequest

void saveFileOnRequest(java.lang.String fileToSaveToOnRequest)

Sets the path to save the file to just after it gets requested.

Parameters:

fileToSaveToOnRequest - The path of the file to save to.