com.screenscraper.common

Class HTTPRequest

java.lang.Object

com.screenscraper.common.HTTPRequest

All Implemented Interfaces:

RunningHTTPRequest

public class HTTPRequest
extends java.lang.Object
implements RunningHTTPRequest

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.log4j.Logger	log Used for logging.
(package private) java.util.Hashtable	postParameters Holds parsed POST parameters.
(package private) java.lang.String	requestLine The Request-Line of the request.
(package private) java.lang.String	version

Constructor Summary

Constructors

Constructor and Description
HTTPRequest(org.apache.http.HttpRequest httpRequest, java.lang.String postData) Generates a new HTTPRequest based on the given HttpRequest and POST data.
HTTPRequest(java.io.InputStream is, java.lang.String protocol, java.lang.String host, int port) Generates a request based on a stream, protocol and host.
HTTPRequest(int requestID, java.lang.String requestMethod, java.lang.String url, java.lang.String postData, java.lang.String requestLine) Generates a new HTTPRequest, loads it with its basic data, then loads it up with all of its HTTP headers.
HTTPRequest(java.lang.String requestLine, java.util.HashSet requestHeaders, java.lang.String postData) Generates a new HTTPRequest based on the given parameters.
HTTPRequest(java.lang.String request, java.net.URL requestURL) Generates a new HTTPRequest, given a string.

Method Summary

Modifier and Type	Method and Description
void	addHTTPHeader(HTTPHeader newHeader)
void	addHTTPHeader(java.lang.String key, java.lang.String value) Adds a HTTP header as a key/value pair.
void	addPOSTParameter(java.lang.String key, java.lang.String value) Adds a POST parameter to the local hash.
void	deleteHeaders() Deletes the headers corresponding to this request.
void	forceRequestLineToHTTPS() Forces the request line to HTTPS.
java.util.ArrayList	getCookiesAsArrayList() Gets an ArrayList over which the cookies can be accessed as HTTPParameter objects.
java.util.ArrayList	getGETParametersAsArrayList(boolean doDecode) Gets an ArrayList over which the GET parameters can be accessed as HTTPParameter objects.
java.util.ArrayList	getHeadersAsArrayList() Gets an ArrayList over which the headers can be accessed as HTTPParameter objects.
java.lang.String	getHTTPHeader(java.lang.String key) Gets the value of a header, given its key.
java.util.Set<HTTPHeader>	getHTTPHeaders() Gets the HTTP headers.
java.util.Iterator	getHTTPHeadersIterator() Gets an Iterator over which the headers can be iterated.
int	getID() Gets the ID.
java.lang.String	getPOSTData() Gets the POST data.
java.lang.String	getPOSTDataShort() Gets the POST data truncated if it's too long.
java.lang.String	getPOSTParameter(java.lang.String key) Gets a POST parameter identified by the given key.
java.util.Hashtable	getPOSTParameters() Gets the POST data as a Hashtable of parameters.
java.util.ArrayList	getPOSTParametersAsArrayList(boolean doDecode) Gets an ArrayList over which the POST parameters can be accessed as HTTPParameter objects.
java.lang.String	getQueryString() Gets the query string portion of the URL.
java.lang.String	getRawRequest() Returns a string representing the raw HTTP request held by this object.
java.lang.String	getRequestLine() Gets the request line.
java.lang.String	getRequestMethod() Gets the request method.
java.net.URL	getURL() Gets the url.
java.lang.String	getURLAsString() Gets the url.
java.lang.String	getURLWithoutQueryString() Gets the URL without the query string.
boolean	isBinaryAccordingToURL() Indicates whether or not the file is likely binary based on the URL.
boolean	isMultiPart() Indicates whether or not this is a multipart request (most likely has a file upload).
boolean	isUsefulAsScrapeableFile() Indicates whether or not this request would likely be made into a scrapeable file.
void	removeHTTPHeader(HTTPHeader oldHeader)
void	removeHTTPHeader(java.lang.String key, java.lang.String value) Removes an HTTPHeader.
void	removePOSTParameter(java.lang.String key) Removes a POST parameter from the local hash.
boolean	requestEntityIsKeyValuePairs() Indicates whether or not the POST payload is key/value pairs or a raw entity.
void	save(int httpTransactionID) Saves the information related to this request to the database.
void	setHeader(java.lang.String header) Sets a header, given a String representing a header.
void	setHeader(java.lang.String key, java.lang.String value)
void	setHeaders(org.apache.commons.httpclient.Header[] headers) Sets the headers.
void	setID(int requestID) Sets the ID.
void	setPOSTData(java.lang.String postData) Sets the POST data.
void	setRequestLine(java.lang.String requestLine) Sets the request line.
void	setRequestLine(java.lang.String requestMethod, java.lang.String urlString, java.lang.String httpVersion) Sets the request line params.
void	setRequestLine(java.lang.String requestMethod, java.net.URL url, java.lang.String httpVersion) Sets the request line params.
void	setURL(java.net.URL url) Sets the url.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

public static org.apache.log4j.Logger log

Used for logging.

requestLine

@Nullable
java.lang.String requestLine

The Request-Line of the request.

postParameters

@Nullable
java.util.Hashtable postParameters

Holds parsed POST parameters.

version

@Nullable
java.lang.String version

Constructor Detail

HTTPRequest

public HTTPRequest(@Nullable
                   java.lang.String request,
                   @Nullable
                   java.net.URL requestURL)

Generates a new HTTPRequest, given a string.

Parameters:

request - The request, encapsulated as a string.

requestURL - The url that was initially requested, if known

HTTPRequest

public HTTPRequest(@NotNull
                   java.io.InputStream is,
                   @Nullable
                   java.lang.String protocol,
                   @Nullable
                   java.lang.String host,
                   int port)
            throws java.lang.Exception

Generates a request based on a stream, protocol and host.

Parameters:

is - The InputStream encapsulating the request.

protocol - The protocol used in the request.

host - The host corresponding to the request.

port - The port corresponding to the request.

Throws:

java.lang.Exception

HTTPRequest

public HTTPRequest(java.lang.String requestLine,
                   java.util.HashSet requestHeaders,
                   @Nullable
                   java.lang.String postData)

Generates a new HTTPRequest based on the given parameters.

Parameters:

requestLine - The first line of the request.

requestHeaders - The request headers.

postData - The POST data.

HTTPRequest

public HTTPRequest(int requestID,
                   java.lang.String requestMethod,
                   @NotNull
                   java.lang.String url,
                   java.lang.String postData,
                   java.lang.String requestLine)

Generates a new HTTPRequest, loads it with its basic data, then loads it up with all of its HTTP headers.

Parameters:

requestID - The ID.

requestMethod - The request method (e.g. GET, POST).

url - The URL requested.

postData - Any POST data sent.

requestLine - The first line of the request.

HTTPRequest

public HTTPRequest(@Nullable
                   org.apache.http.HttpRequest httpRequest,
                   java.lang.String postData)

Generates a new HTTPRequest based on the given HttpRequest and POST data.

Parameters:

httpRequest - An HttpRequest.

postData - The POST data.

Method Detail

setHeaders

public void setHeaders(@NotNull
                       org.apache.commons.httpclient.Header[] headers)

Sets the headers.

Parameters:

headers - A two-dimensional array containing the headers.

setHeader

public void setHeader(@NotNull
                      java.lang.String header)

Sets a header, given a String representing a header.

Parameters:

header - The header.

setHeader

public void setHeader(java.lang.String key,
                      java.lang.String value)

addHTTPHeader

public void addHTTPHeader(HTTPHeader newHeader)

removeHTTPHeader

public void removeHTTPHeader(HTTPHeader oldHeader)

removeHTTPHeader

public void removeHTTPHeader(java.lang.String key,
                             java.lang.String value)

Description copied from interface: RunningHTTPRequest

Removes an HTTPHeader.

Specified by:

removeHTTPHeader in interface RunningHTTPRequest

getID

public int getID()

Gets the ID.

Returns:

The ID.

setID

public void setID(int requestID)

Sets the ID.

Parameters:

requestID - The ID.

getRequestMethod

@Nullable
public java.lang.String getRequestMethod()

Gets the request method.

Returns:

The request method.

setRequestLine

public void setRequestLine(java.lang.String requestMethod,
                           @NotNull
                           java.net.URL url,
                           java.lang.String httpVersion)

Sets the request line params.

Parameters:

requestMethod - The request method POST, GET or CONNECT.

url - The url.

httpVersion - The version of http.

setRequestLine

public void setRequestLine(java.lang.String requestMethod,
                           @NotNull
                           java.lang.String urlString,
                           java.lang.String httpVersion)

Sets the request line params.

Specified by:

setRequestLine in interface RunningHTTPRequest

Parameters:

requestMethod - The request method POST, GET or CONNECT.

urlString - The url.

httpVersion - The version of http.

setRequestLine

@InternalOnly
public final void setRequestLine(java.lang.String requestLine)

Sets the request line.

Parameters:

requestLine - The request line to set.

getRequestLine

@Nullable
public java.lang.String getRequestLine()

Gets the request line.

Returns:

The request line.

forceRequestLineToHTTPS

public void forceRequestLineToHTTPS()

Forces the request line to HTTPS.

getURL

@Nullable
public java.net.URL getURL()

Gets the url.

Specified by:

getURL in interface RunningHTTPRequest

Returns:

The url.

getURLAsString

public java.lang.String getURLAsString()

Gets the url.

Specified by:

getURLAsString in interface RunningHTTPRequest

Returns:

The url.

getURLWithoutQueryString

@Nullable
public java.lang.String getURLWithoutQueryString()

Gets the URL without the query string.

Returns:

The URL without the query string.

setURL

public void setURL(java.net.URL url)

Sets the url.

Parameters:

url - The url.

getQueryString

@Nullable
public java.lang.String getQueryString()

Gets the query string portion of the URL.

Returns:

The query string, if one exists; null, otherwise.

getPOSTData

@Nullable
public java.lang.String getPOSTData()

Gets the POST data.

Returns:

The POST data.

getPOSTDataShort

@Nullable
public java.lang.String getPOSTDataShort()

Gets the POST data truncated if it's too long.

Returns:

The POST data.

setPOSTData

public void setPOSTData(java.lang.String postData)

Sets the POST data.

Parameters:

postData - The POST data.

getPOSTParameters

@Nullable
public java.util.Hashtable getPOSTParameters()

Gets the POST data as a Hashtable of parameters.

Returns:

A Hashtable of key/value pairs.

getPOSTParameter

@NotNull
public java.lang.String getPOSTParameter(@NotNull
                                                   java.lang.String key)

Gets a POST parameter identified by the given key.

Parameters:

key - The key associated with the desired value.

Returns:

A value, if it exists; null, if not.

requestEntityIsKeyValuePairs

public boolean requestEntityIsKeyValuePairs()

Indicates whether or not the POST payload is key/value pairs or a raw entity.

Returns:

A boolean.

getGETParametersAsArrayList

@NotNull
public java.util.ArrayList getGETParametersAsArrayList(boolean doDecode)

Gets an ArrayList over which the GET parameters can be accessed as HTTPParameter objects.

Parameters:

doDecode - Whether or not parameters should be URL-decoded before returning them.

Returns:

An ArrayList.

getPOSTParametersAsArrayList

@NotNull
public java.util.ArrayList getPOSTParametersAsArrayList(boolean doDecode)

Gets an ArrayList over which the POST parameters can be accessed as HTTPParameter objects.

Parameters:

doDecode - Whether or not parameters should be URL-decoded before returning them.

Returns:

An ArrayList.

isMultiPart

public boolean isMultiPart()

Indicates whether or not this is a multipart request (most likely has a file upload). This is determined by iterating over all of the HTTP headers, looking for the "Content-Type" header, which should indicate if it is.

Returns:

A boolean.

addPOSTParameter

public void addPOSTParameter(@NotNull
                             java.lang.String key,
                             @NotNull
                             java.lang.String value)

Adds a POST parameter to the local hash.

Specified by:

addPOSTParameter in interface RunningHTTPRequest

Parameters:

key - The key in the parameter.

value - The value in the parameter.

removePOSTParameter

public void removePOSTParameter(@NotNull
                                java.lang.String key)

Removes a POST parameter from the local hash.

Specified by:

removePOSTParameter in interface RunningHTTPRequest

Parameters:

key - The key in the parameter.

getCookiesAsArrayList

@NotNull
public java.util.ArrayList getCookiesAsArrayList()

Gets an ArrayList over which the cookies can be accessed as HTTPParameter objects.

Returns:

An ArrayList.

getHeadersAsArrayList

@NotNull
public java.util.ArrayList getHeadersAsArrayList()

Gets an ArrayList over which the headers can be accessed as HTTPParameter objects.

Returns:

An ArrayList.

getHTTPHeaders

public java.util.Set<HTTPHeader> getHTTPHeaders()

Gets the HTTP headers.

Returns:

The HTTP headers as a set.

getHTTPHeadersIterator

@NotNull
public java.util.Iterator getHTTPHeadersIterator()

Gets an Iterator over which the headers can be iterated.

Returns:

An Iterator pointing to HTTPHeader objects.

addHTTPHeader

public void addHTTPHeader(java.lang.String key,
                          java.lang.String value)

Adds a HTTP header as a key/value pair.

Specified by:

addHTTPHeader in interface RunningHTTPRequest

getHTTPHeader

@Nullable
public java.lang.String getHTTPHeader(java.lang.String key)

Gets the value of a header, given its key.

Parameters:

key - The key of the header.

Returns:

The value (or null if there is no such header)

getRawRequest

@NotNull
public java.lang.String getRawRequest()

Returns a string representing the raw HTTP request held by this object.

Returns:

The request.

save

public void save(int httpTransactionID)

Saves the information related to this request to the database.

Parameters:

httpTransactionID - The ID of the transaction holding this request.

deleteHeaders

public void deleteHeaders()

Deletes the headers corresponding to this request.

isUsefulAsScrapeableFile

public boolean isUsefulAsScrapeableFile()

Indicates whether or not this request would likely be made into a scrapeable file.

Returns:

A boolean.

isBinaryAccordingToURL

public boolean isBinaryAccordingToURL()

Indicates whether or not the file is likely binary based on the URL.

Returns:

A boolean.