com.screenscraper.util.retry

Interface RetryPolicy

All Known Implementing Classes:

AbstractRetryPolicy, EmptyRetryPolicy, MultiplePolicyRetryPolicy, MultiRegexRetryPolicy, StandardRetryPolicy

public interface RetryPolicy

Policy for handling pages that load incorrectly User: Mike Date: 11/9/11 Time: 5:11 PM

Method Summary

Modifier and Type	Method and Description
java.util.Map<java.lang.String,?>	getErrorChecksMap() Returns a map that can be used to output an error message to indicate what checks failed.
int	getMaxRetryAttempts() Return the maximum number of times this policy allows for a retry before terminating in an error
boolean	isError() Checks to see if the page loaded incorrectly
boolean	resetReferrerBeforeRescrape() Returns true if the referrer should be reset before attempting to rescrape the file, if there was an error.
boolean	resetSessionVariablesBeforeRescrape() Returns true if the session variables should be reset before attempting to rescrape the file, if there was an error.
void	runOnAllAttemptsFailed() This will be called if all the retry attempts for the scrapeable file failed.
void	runOnError() Runs this code when the page had an error.
boolean	shouldLogErrors() Returns true if errors should be logged to the log/web interface when they occur

Method Detail

isError

boolean isError()
         throws java.lang.Exception

Checks to see if the page loaded incorrectly

Returns:

True on errors, false otherwise

Throws:

java.lang.Exception - If something goes wrong while executing this method

runOnError

void runOnError()
         throws java.lang.Exception

Runs this code when the page had an error. This could include things such as rotating the proxy. This will be run AFTER resetting the referrer and any session variables. In this way you can manually adjust values if needed but not have the side effects from extractors that may have modified values from this scrapeable file

Throws:

java.lang.Exception - If something goes wrong while executing this method

getErrorChecksMap

@Nullable
java.util.Map<java.lang.String,?> getErrorChecksMap()
                                                        throws java.lang.Exception

Returns a map that can be used to output an error message to indicate what checks failed. For instance, you could set a key to the value "Status Code" and the value '200', or a key with "Valid Page" and value 'false'

Returns:

Map of keys, or null if no values are indicated

Throws:

java.lang.Exception - If something goes wrong while executing this method

resetSessionVariablesBeforeRescrape

boolean resetSessionVariablesBeforeRescrape()

Returns true if the session variables should be reset before attempting to rescrape the file, if there was an error. This can be useful especially if extractors null session variables when they don't match, but the value is needed to rescrape the file. If this returns true, the session variables will be reset BEFORE calling runOnError()

Returns:

True if session variables should be reset if there was an error, false otherwise.

resetReferrerBeforeRescrape

boolean resetReferrerBeforeRescrape()

Returns true if the referrer should be reset before attempting to rescrape the file, if there was an error. This can be useful to reset so the referrer doesn't show the page you just requested. If this returns true, the referrer will be reset BEFORE calling runOnError()

Returns:

True if the referrer should be reset if there was an error, false otherwise.

shouldLogErrors

boolean shouldLogErrors()

Returns true if errors should be logged to the log/web interface when they occur

Returns:

True if errors should be logged to the log/web interface when they occur

getMaxRetryAttempts

int getMaxRetryAttempts()

Return the maximum number of times this policy allows for a retry before terminating in an error

Returns:

The maximum number of times to allow the ScrapeableFile to be rescraped before resulting in an error

runOnAllAttemptsFailed

void runOnAllAttemptsFailed()
                     throws java.lang.Exception

This will be called if all the retry attempts for the scrapeable file failed. In other words, if the policy said to retry 25 times, after 25 failures this method will be called. Note that runOnError() will be called just before this, as it is called after each time the scrapeable file fails to load correctly, including the last time it fails to load.

This should only contain code that handles the final error. Any proxy rotating, cookie clearing, etc... should generally be done in the runOnError() method, especially since it will still be called after the final error.

Throws:

java.lang.Exception - If the contents of this method throw an exception in the child class