com.screenscraper.http

Class AsyncScrapingHttpClient2

java.lang.Object

com.screenscraper.http.AbstractScrapingHttpClient

com.screenscraper.http.AsyncScrapingHttpClient2

All Implemented Interfaces:

ScrapingHttpClient, java.io.Closeable, java.lang.AutoCloseable

public class AsyncScrapingHttpClient2
extends AbstractScrapingHttpClient

Async HTTP client. This is the newer version of the ning HTTP client which uses a new API. There are some things that make it incompatible with the other client, but there are also things we need to keep the old one for

Version:

7.0.6a

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AsyncScrapingHttpClient2.SharedAsyncHttpClient Class to wrap an async client so it can be shared (save on resources) but still closed like normal

Field Summary

Fields inherited from class com.screenscraper.http.AbstractScrapingHttpClient

GLOBAL_REQUESTS_THREAD_LOCK, session

Fields inherited from interface com.screenscraper.http.ScrapingHttpClient

REDIRECT_WITH_POST_ENTITY_STATUS_CODE

Constructor Summary

Constructors

Constructor and Description
AsyncScrapingHttpClient2(ScrapingSession session) Creates the client
AsyncScrapingHttpClient2(ScrapingSession session, boolean allowSharingUnderlyingClient) Creates the client
AsyncScrapingHttpClient2(ScrapingSession session, boolean followRedirects, boolean autoSetCookies, org.asynchttpclient.AsyncHttpClient client, boolean showActualRequestHeaders) Forcibly sets the underlying client.
AsyncScrapingHttpClient2(ScrapingSession session, boolean followRedirects, boolean autoSetCookies, boolean allowSharingUnderlyingClient, java.util.Set<java.lang.String> acceptOnlyCiphers, java.lang.String... acceptOnlyProtocols) Creates the client
AsyncScrapingHttpClient2(ScrapingSession session, boolean followRedirects, boolean autoSetCookies, java.util.Set<java.lang.String> acceptOnlyCiphers, java.lang.String... acceptOnlyProtocols) Creates the client

Method Summary

Modifier and Type	Method and Description
void	clearCookies() Clears all cookies from this client
void	close()
java.util.List<ScrapingCookie>	getCookies() Returns a list of all the cookies for the current client, for any domain
static java.util.Map<HttpClientKey,SharedHttpClient>	getCurrentOpenClients() Returns a map of the currently open clients (for debugging purposes)
protected ScrapingResponse	issueRequestInternal(ScrapingRequest request) Issues the given request
void	setCookie(ScrapingCookie cookie) Sets the cookie
java.lang.String	toString()
void	updateSessionClientInfo() Tells the client the session http info value has changed.
boolean	usesHint(ScrapingHint hint) Returns whether or not this client implementation uses the given hint.

Methods inherited from class com.screenscraper.http.AbstractScrapingHttpClient

convertScrapeableFileToRequest, getProxy, isRedirectStatusCode, issueRequest, logDebug, logDebug, logError, logError, logInfo, logInfo, logWarn, logWarn, scrapeFile, setProxyServer

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

AsyncScrapingHttpClient2

public AsyncScrapingHttpClient2(@Nullable
                                ScrapingSession session)

Creates the client

Parameters:

session - The session for logging (and converting scrapeable files)

AsyncScrapingHttpClient2

public AsyncScrapingHttpClient2(@Nullable
                                ScrapingSession session,
                                boolean allowSharingUnderlyingClient)

Creates the client

Parameters:

session - The session for logging (and converting scrapeable files)

allowSharingUnderlyingClient - True if this can share a client with other scrapes (not cookies though), false to require this scrape to have a unique client reference and connection pool

AsyncScrapingHttpClient2

public AsyncScrapingHttpClient2(@Nullable
                                ScrapingSession session,
                                boolean followRedirects,
                                boolean autoSetCookies,
                                @NotNull
                                org.asynchttpclient.AsyncHttpClient client,
                                boolean showActualRequestHeaders)

Forcibly sets the underlying client. This allows using various providers supported by the AsyncHttpClient library

Parameters:

session - The session for logging and converting scrapeable files

followRedirects - Whether or not to follow redirects automatically. Set to false if using for proxying

autoSetCookies - True if cookies should be tracked automatically, false otherwise

client - The client to use. Please note that if you want all cookies to be properly set you should not have this client configured to follow

showActualRequestHeaders - True if we should show the actual headers sent, false to try and guess. Generally this should be set to true, but some providers don't allow it, so it can be set here if the manual configuration won't allow it redirects, but should instead tell it not to but then pass in true for the followRedirects flag

AsyncScrapingHttpClient2

public AsyncScrapingHttpClient2(@Nullable
                                ScrapingSession session,
                                boolean followRedirects,
                                boolean autoSetCookies,
                                @NotNull
                                java.util.Set<java.lang.String> acceptOnlyCiphers,
                                java.lang.String... acceptOnlyProtocols)

Creates the client

Parameters:

session - The session for logging and converting scrapeable files

followRedirects - Whether or not to follow redirects automatically. Set to false if using for proxying

autoSetCookies - True if cookies should be tracked automatically, false otherwise

acceptOnlyCiphers - A set of ciphers to accept (or empty set to use defaults)

acceptOnlyProtocols - An array of protocols to accept. Note that empty means use the default (and on that failing to try all protocols)

AsyncScrapingHttpClient2

public AsyncScrapingHttpClient2(@Nullable
                                ScrapingSession session,
                                boolean followRedirects,
                                boolean autoSetCookies,
                                boolean allowSharingUnderlyingClient,
                                @NotNull
                                java.util.Set<java.lang.String> acceptOnlyCiphers,
                                java.lang.String... acceptOnlyProtocols)

Creates the client

Parameters:

session - The session for logging and converting scrapeable files

followRedirects - Whether or not to follow redirects automatically. Set to false if using for proxying

autoSetCookies - True if cookies should be tracked automatically, false otherwise

allowSharingUnderlyingClient - True if this can share a client with other scrapes (not cookies though), false to require this scrape to have a unique client reference and connection pool

acceptOnlyCiphers - A set of ciphers to accept (or empty set to use defaults)

acceptOnlyProtocols - An array of protocols to accept. Note that empty means use the default (and on that failing to try all protocols)

Method Detail

getCurrentOpenClients

public static java.util.Map<HttpClientKey,SharedHttpClient> getCurrentOpenClients()

Returns a map of the currently open clients (for debugging purposes)

Returns:

A map of the currently open clients

Since:

7.0.11a

updateSessionClientInfo

public void updateSessionClientInfo()

Description copied from interface: ScrapingHttpClient

Tells the client the session http info value has changed. If the client is built from values at the start of the session rather than grabbing them per request, it may need to update something internally. This method should be called to indicate a change has occurred, and all the session.getXXX methods should return the updated values

setCookie

public void setCookie(@NotNull
                      ScrapingCookie cookie)

Sets the cookie

Parameters:

cookie - The cookie to set

issueRequestInternal

@NotNull
protected ScrapingResponse issueRequestInternal(@NotNull
                                                          ScrapingRequest request)

Issues the given request

Specified by:

issueRequestInternal in class AbstractScrapingHttpClient

Parameters:

request - The request to be issued

Returns:

The response received

usesHint

public boolean usesHint(@NotNull
                        ScrapingHint hint)

Returns whether or not this client implementation uses the given hint. This method is for utility purposes, as hints may be ignored regardless of what this method returns. This is instead a clue about whether a given hint will be used. Developers may forget to return the correct value here and it won't cause any failures (just helps determine the client to use)

Parameters:

hint - The hint to test

Returns:

True if the client tries to use the given hint type, false if it doesn't. This is a clue, and is not guaranteed to always hold

getCookies

@NotNull
public java.util.List<ScrapingCookie> getCookies()

Returns a list of all the cookies for the current client, for any domain

Returns:

A list of all cookies in the current client

clearCookies

public void clearCookies()

Clears all cookies from this client

close

public void close()

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object