Version: 17.07
UltraESB-X facilitate the collection of diagnostic details of itself and its container (JVM), such as logs, configurations, heap dumps and thread dumps, via a set of troubleshoot reporting tasks.
Each UltraESB-X instance may expose a set of troubleshoot reporting tasks. The Management API facilitates the invocation of a subset of these tasks sequentially, as a task queue, bundling the results of all tasks into an archive on the remote system, which can be fetched manually or downloaded via the API.
Some tasks, such as thread dumps and heap dumps, can be lengthy and consume high amounts of system resources (e.g. CPU and memory). Care should be taken when invoking them, as they may further de-stabilize the target system, e.g. in high-load situations. Such tasks are marked as intensive by the framework. During invocation of a task queue containing such intensive tasks, UXTerm will issue a warning and ask for confirmation before submitting the queue to the server.
Tasks that support the credential masking feature are marked as maskable.
Some tasks allow customization via parameters.
Such parameters can be specified during invocation by prepending the task name with a comma-separated list of
key=value
pairs, following the syntax
task1(param1=value1,param2=value2[,…])[,…]
where key
corresponds to a parameter name and value
is its value.
If any of the parameters are not specified, a default value defined by the task will be utilized.
For example,
thread-dump(period=5,count=3),conf-dir,detailed-logs(period=300,logger=org.adroitlogic.x.transport.http.nio)
will invoke the tasks
thread-dump
, taking 3 (period
) thread dumps with a delay of 5 seconds (period
) between each pair
conf-dir
detailed-logs
, increasing the log level of org.adroitlogic.x.transport.http.nio
(logger
) over 300 seconds
(period
), i.e. 5 minutes
Currently the following in-built tasks are provided in the UltraESB-X distribution:
Task Name | Description | Intensive? | Maskable? | Arguments (Name | Description | Default) | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
archives the UltraESB projects directory, optionally with credential masking |
No |
Yes |
||||||||||
|
archives the UltraESB configuration directory, optionally with credential masking |
No |
Yes |
||||||||||
|
archives the UltraESB custom libraries directory |
No |
No |
||||||||||
|
produces thread dumps of the JVM running the UltraESB instance |
Yes |
No |
|
|||||||||
|
takes a detailed log sample over a given time by escalating log levels of specified loggers |
Yes |
Yes |
|
|||||||||
|
archives the log files currently residing in the UltraESB instance, optionally with credential masking |
No |
Yes |
||||||||||
|
produces a heap dump of the JVM running the UltraESB instance |
Yes |
No |
||||||||||
|
Lists the files reside inside the library directory and its main sub directories |
No |
No |
||||||||||
|
archives the UltraESB patch libraries directory |
No |
No |
||||||||||
|
produces a detailed version and status information report of the UltraESB instance |
No |
Yes |
||||||||||
|
produces a detailed report on the system and JVM running the UltraESB instance |
No |
Yes |
Additionally, the UXTerm configuration allows one to define several task bundles for quick invocation of groups of tasks. A task bundle is basically a predefined queue of tasks, denoted by a comma-separated list of task names, referred to by a single name (bundle name); for example,
ultra.terminal.troubleshoot.bundles.info=esb-info, sys-info
in the UXTerm configuration denotes a task bundle named info
, including the two tasks esb-info
and sys-info
.
Note that the bundle name should be added to the name list under ultra.terminal.troubleshoot.bundles
property for it
to be identified correctly by the commmand; for example,
ultra.terminal.troubleshoot.bundles=full, dump,
info
Instead of a list of tasks, one can specify a task bundle name when invoking the troubleshoot report command, in which case the associated list of tasks will be submitted to the server.
Some tasks, such as configuration archival, offer an option to run their output through a masking logic, encrypting
portions of sensitive content (e.g. passwords or keys) using a user-specified password.
This can be useful if you plan to submit the troubleshoot report to the original developers for further analysis, and
in case the credentials need to be unmasked during a later stage, the password can be disclosed separately.
The parameters -p
, -m
, -k
and -c
in the tbsrstr
command govern the masking process.
The same masking parameters are applied to all applicable tasks (having the maskable flag) in the submitted queue.
Tasks are picked using the -t
and -b
parameters of the
tbsrstr
command.
Bundles specified in -b
will be expanded into the respective task names, and merged with task names specified in -t
.
If -i
(ignore) arguments are also specified, they would be removed from the generated task list, to derive the final
list (queue) of tasks for execution.
You can refer to a task or bundle name by a part of its name, as long as it is unambiguous.
For example, when invoking tbsrstr
with -b
, instead of specifying the full name of a bundle named
configuration-archival
you can simply enter config
, as long as no other task bundle name contains the word config
.
These commands facilitate the collection of diagnostic details from the connected UltraESB-X instance, via reporting tasks.
troubleshoot-report-option-list
, tbsrol
Lists the troubleshoot reporting tasks available on the connected UltraESB-X instance.
Syntax: tbsrol [-v]
|
lists only the task bundles (defined in the UXTerm configuration) |
|
lists only the available tasks (retrieved from the connected instance) |
|
displays all available details of the listed tasks or bundles |
troubleshoot-report-start
, tbsrstr
Starts a troubleshoot reporting session on the connected instance.
Syntax: tbsrstr [-a] [-b <bundle-list>] [-c <cipher>] [-f <target-file>] [-i <tasks-to-ignore>] [-k <key-factoy>]
[-m <mask>] [-p <password>] [-t <task-list>]
E.g. : tbsrstr -f /tmp/threads.zip -t heap,thread(period=5,count=3) -m password=[^\s]* -p ultraesb
|
asynchronous invocation (return to shell after submitting the reporting configuration to server) |
|
comma-separated list of names of task bundles to be executed (tasks will be merged, and duplicates removed on a retain-first-found basis) |
|
cipher name used for encryption under masking mode, applicable with |
|
desired output path (must be absolute) where the report should be saved on the instance, optionally including file name |
|
comma-separated list of tasks to be ignored when running the specified bundles (generally for use along with |
|
key factory name used for encryption, applicable with |
|
regular expression for matching credentials to be masked or encrypted |
|
password for encrypting matches of |
|
comma-separated list of tasks to be executed, in the format |
troubleshoot-report-status-check
, tbsrstchk
Checks the status of a previous asynchronous troubleshoot reporting invocation.
Syntax: tbsrstchk
The output includes the execution state of each executed task, along with the completion status (success/failure) and the time taken by each, and the overall status and elapsed time for the whole invocation.