com.screenscraper.http

Class AbstractScrapingHttpClient

java.lang.Object

com.screenscraper.http.AbstractScrapingHttpClient

All Implemented Interfaces:

ScrapingHttpClient, java.io.Closeable, java.lang.AutoCloseable

Direct Known Subclasses:

ApacheScrapingHttpClient, AsyncNingScrapingHttpClient, AsyncScrapingHttpClient2, CurlHttpClient

public abstract class AbstractScrapingHttpClient
extends java.lang.Object
implements ScrapingHttpClient

Class with method's for preparing scrapeable files to be scraped, etc... This is setup to work with the session

Version:

1.0

Field Summary

Fields

Modifier and Type	Field and Description
static java.util.concurrent.locks.ReentrantReadWriteLock	GLOBAL_REQUESTS_THREAD_LOCK A lock used globally by AbstractScrapingHttpClients so an external source can lock the write lock to block all request access temporarily.
protected ScrapingSession	session

Fields inherited from interface com.screenscraper.http.ScrapingHttpClient

REDIRECT_WITH_POST_ENTITY_STATUS_CODE

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractScrapingHttpClient(ScrapingSession session) Creates the client

Method Summary

Modifier and Type	Method and Description
ScrapingRequest.Builder	convertScrapeableFileToRequest(ScrapeableFile scrapeableFile) Converts the scrapeable file to the request.
protected ScrapingProxy	getProxy() Returns the proxy server that should be used for requests (this can change on each call)
protected boolean	isRedirectStatusCode(int statusCode) Checks if the given status code indicates a redirect
ScrapingResponse	issueRequest(ScrapingRequest request) Issues the given request
protected abstract ScrapingResponse	issueRequestInternal(ScrapingRequest request) Issues the given request.
protected void	logDebug(java.lang.String message)
protected void	logDebug(java.lang.String message, java.lang.Throwable throwable)
protected void	logError(java.lang.String message)
protected void	logError(java.lang.String message, java.lang.Throwable throwable)
protected void	logInfo(java.lang.String message)
protected void	logInfo(java.lang.String message, java.lang.Throwable throwable)
protected void	logWarn(java.lang.String message)
protected void	logWarn(java.lang.String message, java.lang.Throwable throwable)
ScrapingData	scrapeFile(ScrapeableFile scrapeableFile) Requests data for a scrapeable file.
void	setProxyServer(ScrapingProxy proxyServer) Sets a proxy server to use for future requests

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.screenscraper.http.ScrapingHttpClient

clearCookies, close, getCookies, setCookie, updateSessionClientInfo, usesHint

Field Detail

GLOBAL_REQUESTS_THREAD_LOCK

public static final java.util.concurrent.locks.ReentrantReadWriteLock GLOBAL_REQUESTS_THREAD_LOCK

A lock used globally by AbstractScrapingHttpClients so an external source can lock the write lock to block all request access temporarily. Note that requests issued by a client in the thread holding the write lock will still go through. Requests hold a readLock while being processed.

Since:

7.0.10a

session

@Nullable
protected final ScrapingSession session

Constructor Detail

AbstractScrapingHttpClient

protected AbstractScrapingHttpClient(@Nullable
                                     ScrapingSession session)

Creates the client

Parameters:

session - The session

Method Detail

issueRequest

@NotNull
public final ScrapingResponse issueRequest(@NotNull
                                                     ScrapingRequest request)

Issues the given request

Specified by:

issueRequest in interface ScrapingHttpClient

Parameters:

request - The request to be issued

Returns:

The response received

issueRequestInternal

@NotNull
protected abstract ScrapingResponse issueRequestInternal(@NotNull
                                                                   ScrapingRequest request)

Issues the given request. This method should be overridden in subclasses to do the request.

Parameters:

request - The request to be issued

Returns:

The response received

getProxy

@Nullable
protected final ScrapingProxy getProxy()

Returns the proxy server that should be used for requests (this can change on each call)

Returns:

The proxy to use

convertScrapeableFileToRequest

public ScrapingRequest.Builder convertScrapeableFileToRequest(@NotNull
                                                              ScrapeableFile scrapeableFile)

Description copied from interface: ScrapingHttpClient

Converts the scrapeable file to the request. It returns the builder type so implementors can add their own stuff. For example:

return super.convertScrapeableFileToRequest(scrapeableFile).addHeader("Cookie: " + someCookie)

Specified by:

convertScrapeableFileToRequest in interface ScrapingHttpClient

Parameters:

scrapeableFile - The scrapeable file to convert

Returns:

The builder prepared to create the request

logDebug

protected final void logDebug(@NotNull
                              java.lang.String message)

logDebug

protected final void logDebug(@NotNull
                              java.lang.String message,
                              @Nullable
                              java.lang.Throwable throwable)

logInfo

protected final void logInfo(@NotNull
                             java.lang.String message)

logInfo

protected final void logInfo(@NotNull
                             java.lang.String message,
                             @Nullable
                             java.lang.Throwable throwable)

logWarn

protected final void logWarn(@NotNull
                             java.lang.String message)

logWarn

protected final void logWarn(@NotNull
                             java.lang.String message,
                             @Nullable
                             java.lang.Throwable throwable)

logError

protected final void logError(@NotNull
                              java.lang.String message)

logError

protected final void logError(@NotNull
                              java.lang.String message,
                              @Nullable
                              java.lang.Throwable throwable)

setProxyServer

public void setProxyServer(@Nullable
                           ScrapingProxy proxyServer)

Sets a proxy server to use for future requests

Specified by:

setProxyServer in interface ScrapingHttpClient

Parameters:

proxyServer - The proxy server to use for future requests

scrapeFile

public final ScrapingData scrapeFile(@NotNull
                                     ScrapeableFile scrapeableFile)

Requests data for a scrapeable file. This should not be overridden. Instead, child classes should override the convertScrapeableFileToRequest(ScrapeableFile) method and the issue request method

Specified by:

scrapeFile in interface ScrapingHttpClient

Parameters:

scrapeableFile - The scrapeable file to request

Returns:

The response from the server

isRedirectStatusCode

protected boolean isRedirectStatusCode(int statusCode)

Checks if the given status code indicates a redirect

Parameters:

statusCode - The status code

Returns:

True if the status code is a redirect code, false otherwise