public class ScrapeStateLogger
extends java.lang.Object
Constructor and Description |
---|
ScrapeStateLogger(ScrapingSession scrapingSession)
Create a scrape logger for the given session
|
ScrapeStateLogger(ScrapingSession scrapingSession,
boolean includeCurrentSessionDetails,
boolean includeTimestamp)
Create a scrape logger for the given session
|
Modifier and Type | Method and Description |
---|---|
AutoProgressBar |
addAutoProgressBar(java.lang.String key,
java.util.Collection<java.lang.String> values)
Adds an auto progress bar to this loggers progress bars
|
void |
addMonitoredPostfix(java.lang.String postfix)
Watches for all session variables whose keys end with the postfix specified, and will output their values when monitored variables
are logged.
|
void |
addMonitoredPrefix(java.lang.String prefix)
Watches for all session variables whose keys begin with the prefix specified, and will output their values when monitored variables
are logged.
|
void |
addMonitoredVariable(java.lang.String key)
Watches the value of a session variable, and will output it each time monitored variables are output
|
ProgressBar |
addProgressBar(java.lang.String title,
double total)
Adds a progress bar to the progress bars
|
java.lang.Object |
addValue(java.lang.String key,
java.lang.Object value)
Adds an arbitrary value to be logged with this logger.
|
java.lang.String |
buildWebMessage(java.lang.String mainDivStyle,
java.lang.Object messageHeader,
java.util.Map<java.lang.String,java.util.List<LoggableValue>> loggableValues)
Build a standard web message, with the formatting for the main <div> as given.
|
ScrapeStateLogger |
duplicate(ScrapingSession scrapingSession)
Creates a duplicate of this logger, but using the given scraping session instead of the current one.
|
java.lang.String |
getCustomHTMLHeader()
Returns the HTML header that has been set.
|
java.util.Map<java.lang.String,java.lang.Object> |
getGeneralInfoVariables()
Returns the general information values, or an empty map if they shouldn't be logged
|
int |
getHighestLevelLogged()
Returns the highest level this logger has logged at.
|
java.util.Date |
getLastRecordCountIncrementTime()
Returns the last time a record was saved, as determined by the number of records scraped value being changed
|
java.lang.String |
getMainWebHeader()
Returns the HTML to display the main header that is always present
|
java.util.Set<java.lang.String> |
getMonitoredPostfixes()
Returns a set of monitored postfixes
|
java.util.Set<java.lang.String> |
getMonitoredPrefixes()
Returns a set of monitored prefixes
|
java.util.Map<java.lang.String,java.lang.Object> |
getMonitoredVariables()
Gets all the currently monitored session variables and values
|
int |
getNumMessagesToSave()
Gets the number of messages that the web interface will save.
|
int |
getNumProgressBars()
Returns the number of progress bars currently available
|
ProgressBar |
getProgressBar(int index)
Returns the progress bar at the given index, with 0 being the highest level bar, and lower-level bars being sequentially larger in index
|
ProgressBar |
getProgressBar(java.lang.String title)
Returns the first progress bar with the given title, or null if none is found.
|
java.lang.String |
getSavedMessages()
Gets a string containing all the currently saved messages, sorted in order from newest to oldest
|
java.lang.String |
getWebDivider(java.lang.String value)
Gets the HTML needed to insert a custom divider in the web message
|
java.lang.String |
getWebProgress()
Get the HTML for the current progress of the scrape, taken from the current progress bars.
|
java.lang.String |
getWebTimestamp()
Get an HTML formatted message indicating the time this method was called, and an html element with the id age[id] that can be
dynamically updated with javascript to show the age of the message
|
boolean |
isLogLowerCaseDataRecordKeys()
Returns whether or not full lower case keys will be logged in a data record
|
boolean |
isLogMixedCaseDataRecordKeys()
Returns whether or not keys with both upper and lower case characters will be logged in a data record
|
boolean |
isLogUpperCaseDataRecordKeys()
Returns whether or not full upper case keys will be logged in a data record
|
void |
removeAllMonitoredVariables()
Stops watching all currently monitored variables, prefixes, postfixes, and any manually set values
|
void |
removeMonitoredPostfix(java.lang.String postfix)
Stops watching for a postfix in session variables
|
void |
removeMonitoredPrefix(java.lang.String prefix)
Stops watching for a prefix in session variables
|
void |
removeMonitoredVariable(java.lang.String key)
Stops watching the specified variable
|
void |
removeProgressBar(ProgressBar progressBar)
Removes the specified progress bar
|
java.lang.Object |
removeValue(java.lang.String key)
Removes a value from the additional values being output by this monitor (doesn't remove session variable values)
|
void |
reset()
Resets all internal values, so that this becomes as if just created.
|
void |
saveMessage(java.lang.String message,
int mapping)
Saves a message under a specific category, as given by the mapping.
|
void |
setCustomHTMLHeader(java.lang.String customHTMLHeader)
Sets the HTML header that will be logged at the top of every message in the web interface.
|
void |
setLogLowerCaseDataRecordKeys(boolean logLowerCaseDataRecordKeys)
Sets whether or not all lowercase keys of a datarecord should be logged
|
void |
setLogMixedCaseDataRecordKeys(boolean logMixedCaseDataRecordKeys)
Sets whether or not keys of a datarecord with both upper and lowercase characters should be logged
|
void |
setLogUpperCaseDataRecordKeys(boolean logUpperCaseDataRecordKeys)
Sets whether or not all uppercase keys of a datarecord should be logged
|
void |
setNumMessagesToSave(int numMessagesToSave)
Sets the number of messages that the web interface will save.
|
void |
updateLastRecordCountIncrementTime()
Updates the time value when the last record saved value was last modified
|
java.lang.String |
webMessage(java.lang.Object messageHeader,
int level,
boolean saveMessage)
Creates an HTML message representing the scrape state and current scrape progress, and outputs it to the web message area.
|
void |
writeToLog(java.lang.Object message,
int logLevel)
Writes the values in this logger to the log
|
public ScrapeStateLogger(ScrapingSession scrapingSession)
scrapingSession
- The scraping session to log values forpublic ScrapeStateLogger(ScrapingSession scrapingSession, boolean includeCurrentSessionDetails, boolean includeTimestamp)
scrapingSession
- The scraping session to log values forincludeCurrentSessionDetails
- True to include details about the session (variables)includeTimestamp
- True to include timestamps in the logged messages@Nullable public java.util.Date getLastRecordCountIncrementTime()
@NotNull public java.util.Set<java.lang.String> getMonitoredPrefixes()
@NotNull public java.util.Set<java.lang.String> getMonitoredPostfixes()
public boolean isLogLowerCaseDataRecordKeys()
public boolean isLogUpperCaseDataRecordKeys()
public boolean isLogMixedCaseDataRecordKeys()
public final void reset()
public void updateLastRecordCountIncrementTime()
public java.lang.Object addValue(java.lang.String key, @Nullable java.lang.Object value)
key
- The key to usevalue
- The value to log as the value for the given keypublic java.lang.Object removeValue(java.lang.String key)
key
- The key to usepublic void writeToLog(@Nullable java.lang.Object message, int logLevel)
message
- The message to log at the top of this writelogLevel
- The level at which to output the log messagespublic void setNumMessagesToSave(int numMessagesToSave)
numMessagesToSave
- How many messages of a given type to save total.public int getNumMessagesToSave()
@NotNull public java.lang.String webMessage(java.lang.Object messageHeader, int level, boolean saveMessage)
messageHeader
- A header to include at the top of the messagelevel
- The level at which to log this messagesaveMessage
- Should this message be saved (so it can be viewed later)@NotNull public java.lang.String buildWebMessage(java.lang.String mainDivStyle, @Nullable java.lang.Object messageHeader, @NotNull java.util.Map<java.lang.String,java.util.List<LoggableValue>> loggableValues)
mainDivStyle
- The CSS Style for the message bodymessageHeader
- The header to display at the top of the messageloggableValues
- A mapping of the values to log this time@NotNull public java.lang.String getWebProgress()
@NotNull public java.lang.String getWebTimestamp()
public java.lang.String getWebDivider(@Nullable java.lang.String value)
value
- An optional value to put inside the divider@NotNull public java.lang.String getMainWebHeader()
@NotNull public java.util.Map<java.lang.String,java.lang.Object> getGeneralInfoVariables()
@NotNull public java.util.Map<java.lang.String,java.lang.Object> getMonitoredVariables()
public java.lang.String getCustomHTMLHeader()
public void setCustomHTMLHeader(java.lang.String customHTMLHeader)
customHTMLHeader
- The custom HTML header to log before each message, or null to clear it@NotNull public java.lang.String getSavedMessages()
public void saveMessage(java.lang.String message, int mapping)
message
- Text for the message to save (if used with web messages, this should include the HTML)mapping
- Category this message goes underpublic void addMonitoredVariable(java.lang.String key)
key
- The key in the session corresponding to a valuepublic void addMonitoredPrefix(java.lang.String prefix)
prefix
- The prefix to monitorpublic void addMonitoredPostfix(java.lang.String postfix)
postfix
- The postfix to monitorpublic void removeMonitoredVariable(java.lang.String key)
key
- Key for the variable to stop watchingpublic void removeMonitoredPrefix(java.lang.String prefix)
prefix
- Prefix to remove from monitoringpublic void removeMonitoredPostfix(java.lang.String postfix)
postfix
- Postfix to remove from monitoringpublic void removeAllMonitoredVariables()
@NotNull public ScrapeStateLogger duplicate(ScrapingSession scrapingSession)
scrapingSession
- The scraping session to use for session variables, cookies, etc...@NotNull public ProgressBar addProgressBar(java.lang.String title, double total)
title
- The title of the progress bartotal
- The total number of records allowed in this progress bar@NotNull public AutoProgressBar addAutoProgressBar(java.lang.String key, @NotNull java.util.Collection<java.lang.String> values)
key
- The key in the session to usevalues
- The values the progress bar can have. These should iterate in the same order they will be searchedpublic void removeProgressBar(@Nullable ProgressBar progressBar)
progressBar
- The progress bar to remove@Nullable public ProgressBar getProgressBar(java.lang.String title)
title
- The title to search for@Nullable public ProgressBar getProgressBar(int index)
index
- The desired ProgressBar's indexpublic int getHighestLevelLogged()
public int getNumProgressBars()
public void setLogUpperCaseDataRecordKeys(boolean logUpperCaseDataRecordKeys)
logUpperCaseDataRecordKeys
- True to log all uppercase keys, false otherwisepublic void setLogLowerCaseDataRecordKeys(boolean logLowerCaseDataRecordKeys)
logLowerCaseDataRecordKeys
- True to log all lowercase keys, false otherwisepublic void setLogMixedCaseDataRecordKeys(boolean logMixedCaseDataRecordKeys)
logMixedCaseDataRecordKeys
- True to log keys with both upper and lowercase characters, false otherwise