Advance Monitoring

This document describes the concept of Detailed MXBeans in the UltraESB and the Zabbix monitoring capabilities. The detailed MXBeans are used by Zabbix to extract data via JMX Zabbix agent running on UltraESB.

While the UTerm gives the user some notion of monitoring, for a complete monitoring solution UltraESB recommends the use of Zabbix to comprehensively monitor the system as well as to issue alerts on critical conditions to be notified to the users etc. This guide consist of the following sections.

At the end of this guide the user should be able to do advance monitoring of the UltraESB via Zabbix and should be able to configure alters/notifications for the critical events.

Detailed MXBeans

UltraESB has a concept called "Detailed MXBeans" which mainly helps the Open Source Zabbix monitoring solution to be integrated with the UltraESB.

Introduction

The UltraESB utilizes Java Management Extensions (JMX) for monitoring and managing the server instance/s. To be more precise, all administration and monitoring tasks are carried out via JMX, regardless of whether you use the Java JConsole, the JSON REST management services, or the command line client (UTerm). Apart from that we have been introduced to Zabbix by one of our customers and as a community driven open source project we have explored the possibilities of enabling Zabbix monitoring in UltraESB. Thus the v1.4.x onwards now ships a native Zabbix agent within the UltraESB for easier management and monitoring using Zabbix. This article will walk you through how you can use the UltraESB detailed MXBeans, over the usual MXBeans to monitor the UltraESB. The normal MXBeans of the UltraESB are optimized for the JSON/REST API and the command line client usage, while the detailed MXBeans are optimized to show values always as bean attributes - so that monitoring via Zabbix would be much easier.

Note
The Detailed MXBeans feature is available in 1.5.x release onwards of the UltraESB
Detailed MXBeans

To get started with, lets first have a look at "What are these detailed MXBeans?".

The normal MXBeans that are published by the UltraESB uses something called a Management Class to expose management of a certain type. For example endpoint management is being exposed as the EndpointManagement MXBean. This gives the list of endpoints as a brief composite data list as described in the MXBean Specification, where those composite data objects representing the endpoints (we call them the EndpointView’s) are not detailed - but rather holds a basic set of information to give you a list view. While the above approach gives much more advantages when it comes to web administration console and management services, to monitor a given endpoint you have to call a JMX operation which is getEndpoint() on the EndpointManagement MXBean.

With Zabbix calling an JMX operation to monitor an object is not directly supported and not recommended. Because of the above issue, you can configure the UltraESB to expose detailed MXBeans for the individual configuration elements as attributes. If we look at the previous example the endpoint itself will be registered as an MXBean to give its EndpointView under the endpoint identifier, so that it is easy to configure Zabbix to monitor the UltraESB server. This also allows system administrators to also use the Java JConsole and manage each endpoint more easily - as each is presented as a separate bean with the unique identifier. Yet, if you look at the implementation all these methods fall back to the same view objects which are composite data objects described as in the MXBean API.

Enabling Detailed MXBeans in UltraESB

The above described detailed MXBeans are not registered with the JMX platform by default, due to the obvious reasons. Actually these detailed MXBeans duplicate the existing normal MXBean functionality and if you can live with the MXBeans which are there by default, you do not need to worry about these detailed MXBeans. This however is going to be important in Zabbix monitoring.

Enabling these detailed MXBeans are easy as un-commenting few lines in the conf/ultra-root.xml file of the UltraESB installation. Open up the ultra-root.xml from your favourite editor and find the following XML fragment, inside the "ultra-config" spring bean configuration, which is commented out by default.

Enabling Detailed MXBeans

<property name="configurationWatchers">
  <bean class="java.util.ArrayList">
    <constructor-arg>
      <list>
        <bean name="detailedMXBeanRegistrar" class="org.adroitlogic.ultraesb.core.mgt.DetailedMXBeanRegistrar"/>
      </list>
    </constructor-arg>
  </bean>
</property>

Just un-comment the above section in the XML file and save it before starting the UltraESB with the ultraesb.sh or the ultraesb.bat files depending on your operating system. After starting the ESB, you can use the jconsole tool comes with the JDK to inspect these detailed MXBeans, the following snapshot shows you the enabled detailed MXBeans, which are under the "org.adroitlogic.ultraesb.detail" domain name while the normal default MXBeans under the "org.adroitlogic.ultraesb" domain name.

detailed jmx jconsole

Now if you look at the above snapshot of the jconsole, you can see there are 2 domains and the detailed domain has MXBeans for each and every configuration element, like "echo-proxy-inDestination" and so forth, while the default domain has a single MXBean for monitoring all the endpoints.

Selectively Enabling for Artifact Types

The above configuration enables detailed MXBeans for all the configuration elements, but that may not be necessary for your requirement and you might want to minimize the duplicated information. This is possible with UltraESB by selectively enabling only the configuration elements that you want to monitor with an additional property configuration of the above the DetailedMXBeanRegistrar as follows.

Enabling Detailed MXBeans Selectively

<property name="configurationWatchers">
  <bean class="java.util.ArrayList">
    <constructor-arg>
      <list>
        <bean name="detailedMXBeanRegistrar" class="org.adroitlogic.ultraesb.core.mgt.DetailedMXBeanRegistrar">
          <property name="enabledFor">
            <bean class="java.util.ArrayList">
              <constructor-arg>
                <list>
                  <value>ELEMENT_TYPE_ENDPOINT</value>
                  <value>ELEMENT_TYPE_TRANSPORT_SENDER</value>
                </list>
              </constructor-arg>
            </bean>
          </property>
        </bean>
      </list>
    </constructor-arg>
  </bean>
</property>

This configuration registers detailed MXBeans for endpoints and transport senders but not for the other element types. If you compare the below snapshot with the previous one you will be able to see that the detailed JMX domain of UltraESB contains just the endpoints and transport senders listed, but not others.

detailed jmx selective jconsole

The Enumeration values for all the types are listed below for your reference, so that you can selectively enable detailed MXBeans for any element type or any combination of those using the respective enumeration values.

Element Artifact Type Enumeration Value

Endpoints

ELEMENT_TYPE_ENDPOINT

Proxy services

ELEMENT_TYPE_PROXY_SERVICE

Sequences

ELEMENT_TYPE_SEQUENCE

Transport senders

ELEMENT_TYPE_TRANSPORT_SENDER

Transport listeners

ELEMENT_TYPE_TRANSPORT_LISTENER

Work managers

ELEMENT_TYPE_WORK_MANAGER

File caches

ELEMENT_TYPE_FILE_CACHE

That concludes the documentation on the detailed MXBeans and next lets briefly look at how we can use Zabbix to monitor the UltraESB using these detailed MXBeans.

Monitoring, Reporting and Alerting with Zabbix

This document describes the Zabbix monitoring solution and the use of it to advance monitoring use-cases of the UltraESB.

Introduction

Zabbix is an enterprise-class open source distributed monitoring solution for networks and applications. It could be used for monitoring data sets, SLA reporting, alerting and notification and graphing of the history data. UltraESB integrates seamlessly with Zabbix for its advance monitoring reporting and alerting requirements.

Installing Zabbix

Installing Zabbix is a two step process where you first have to install the Zabbix server and then the zabbix web console, which runs on top of the Apache web server. A detailed guide on installing Zabbix is available with the Zabbix documentation here. Or if you are on Ubuntu or any other Unix operating system which has the package installer, you may take the liberty of using that to install Zabbix.

Here is what you need to do on Ubuntu, assuming that you want to install the zabbix server on MySQL. Just follow the instructions of the installer and get it installed.

Install Zabbix server on Ubuntu

$ sudo apt-get install zabbix-server-mysql

Once the Zabbix server is installed, we want the Zabbix web console front end installed as well. You may do this by installing the front end on Apache.

Installing the Zabbix Frontend

$ sudo apt-get install zabbix-frontend-php

The Zabbix front-end is required for the UltraESB to automatically register its element for monitoring with a well defined set of templates covering almost all the monitoring of items, graphs and triggers.

Once the font end installation is complete, just start the Zabbix server with the following command.

Starting the Zabbix Server

$ sudo zabbix_server

Well, we are now done with the installation, so lets move onto configuring Zabbix to monitor the UltraESB.

Configuring Zabbix to Monitor UltraESB

Configuring the Zabbix to Monitor UltraESB requires a bit of configuration, however the Zabbix integration module of the UltraESB simplifies this to just a single command via the UTerm. So the available options for configuring Zabbix to monitor UltraESB are as follows.

  1. UTerm Zabbix Registration Command

  2. Custom Configuration from the Zabbix Console

While the first 2 options are very straight forward the third option is available for any custom configuration.

Use the one of the first 2 options
Once again it is emphasised that one of the first 2 options are very convenient over the 3rd option, but the following section will help in-order to understand how the configuration is done on Zabbix.

First of all point your favourite web browser to the Zabbix URL http://localhost/zabbix/ where you will get to the login page of Zabbix. Use the default username "admin" and password "zabbix" to login to the Zabbix console.

Add Zabbix Host

Once you are logged in go to the Configuration > Hosts from the top navigation bar and click on the "Create Host" as shown below to create a new host to be monitored.

zabbix add host

Then you will be taken into a page with a form to enter details about the Host that you need to monitor. Please follow the snapshot of the form shown below to create a host in Zabbix for UtraESB

Note the values in the fields which are, name "UltraESB" in a new group named "ESB servers", monitoring the locally running UltraESB server so the IP address will be intact which is "0.0.0.0" (you may use the domain name in which case you need to change the "Connect to" option from "IP address" to "DNS name") and specify the "Zabbix agent port" as "11819" which is the default port at which the Zabbix agent is running on UltraESB (you may change this port by un-commenting the port property in the "zabbix-agent" spring bean in the ultra-root.xml configuration), before saving the form by clicking on the "Save" button on the bottom of the form.

zabbix add host form

If you do not see the UltraESB host listed in the hosts table after clicking on the save button, change the "Group" option box on the top right corner to have the value "all" or "ESB servers" which is the group in which we have created our host. Now you will be able to see a table with our host.

Add Zabbix Item

Click on the "Items" link in the UltraESB row, Items column of the host table, to navigate to monitored items. Now we are going to add an item to be monitored for our UltraESB host via the Zabbix agent running in the UltraESB server. For that matter click on the "Create Item" button on the top right corner of the content pane.

With that you will be able to see an item form, which should be filled as shown below to monitor the processed message count by the endpoint echo-proxy-inDestination which is available by default in the UltraESB.

zabbix add item form

If you look at this item form for monitoring this simple statistical data item you need to just fill the description and the key that you are monitoring via the Zabbix agent running in the UltraESB. Out of these two, "key" is the important field as that describes the entity that we are monitoring. So lets look at this key in detail.

JMX Key of a Zabbix Item

jmx[org.adroitlogic.ultraesb.detail:Type=Endpoints,Name=echo-proxy-inDestination][Details.processedMessageCount]

This is a Zabbix agent key describing the item being monitored, where this item is being looked up by the Zabbix agent via the detailed MXBean that we have been describing in the first half of this article. In particular we are interested about an endpoint here named "echo-proxy-inDestination" which has the JMX ObjectName "org.adroitlogic.ultraesb.detail:Type=Endpoints,Name=echo-proxy-inDestination". Now, have a look at the above key where you have this exact same ObjectName within square brackets, just after the characters "jmx" and after that the remaining bit of this key describes the attribute of the MXBean described by the given JMX ObjectName, again within square brackets.

As we have noted in the first half of this article, we have been exposing a composite data object named EndpointView in the detailed endpoint MXBean, which contains the "processedMessageCount" as a field variable. So the notation for traversing the composite data is to use the character '.' to separate the JMX attribute name and the composite data field. Following this we could form a general formula or all the information exposed by the UltraESB detailed MXBeans as follows.

General Formula for Zabbix Item JMX Key

jmx[org.adroitlogic.ultraesb.detail:Type=$ELEMENT_TYPE,Name=$ELEMENT_IDENTIFIER][Details.$FIELD_NAME_OF_VIEW]

where, the $ELEMENT_TYPE is the exposed type name ("Endpoints" in our previous case), $ELEMENT_IDENTIFIER is the identifier of the element being monitored ("echo-proxy-inDestination" in our previous case), and the $FIELD_NAME_OF_VIEW is the name of the field that we are monitoring in the respective View object ("processedMessageCount" of the EndpointView in our previous case). You can get to know how these keys work and advance concepts of these keys from the Zabbix documentation on Items.

Well, now lets look at what this simple configuration gives back to us.

Monitoring with Zabbix

This is time to fill your coffee up as you will have to wait a bit (30 seconds by default) to let Zabbix collect data from the agent running in the UltraESB as instructed with the previous setup. Then if you go to the Monitoring > Latest data you will be able to see a data table as follows; after expanding the Descriptions column header by clicking on the (+) sign.

zabbix latest data

Now use the UltraESB built in toolbox to flow through few messages via the echo-proxy at random times and monitor the Zabbix console back again to see few more data points collected (note that you may reduce the data collection frequency by editing the item configuration) and click on the Graph link on the right most column aligned to our item to view a graphical analysis of this item as shown below.

zabbix graph

What we have seen with Zabbix in this article is just one drop of water in the sea, and the sea is actually very deep and contains a lot more stuff than even water. In the same analysis Zabbix has many more features that you can use it to really monitor the UltraESB with the collected data items. For example like triggers, graphs, error cases, warnings/alerts to reporting and many more with a very sophisticated dashboard to monitor the health and many more. These advance features adds a lot to UltraESB monitoring. Try it out it is free and open source.

In this topic
In this topic
(c) Copyright 2016-2017 AdroitLogic. All rights reserved.
Terms of Use
Contact Us