UltraESB-X Monitoring with Kibana

We at AdroitLogic recently released UltraESB-X developed on top of the flexible, extensible and lightweight Project-X platform. UltraESB-X comes with a GUI developer tool, UltraStudio, to develop, test and package your integration flows easily. Along with this we released Integration Platform Server (IPS), our PaaS solution to manage the deployment, maintenance and scaling of UltraESB-X clusters. When it comes to these enterprise integration products, one thing you can’t keep away is the requirement for statistics and monitoring. AdroitLogic Integration Platform comes with inbuilt monitoring and statistics support and we are in the process of developing our enterprise level monitoring solution IMonitor for UltraESB-X. Since UltraESB-X publishes metrics to the famous, distributed RESTful search and analytics engine Elasticsearch, we can use the open source visualization tool Kibana for monitoring as well. So today we will go through the steps to setup UltraESB-X monitoring with Kibana.

Step 1 –  Installing Kibana and Elasticsearch

Current UltraESB-X release, v17.01, uses Elasticsearch v5.1.1. In this case I am going to assume that you have downloaded and familiarized yourself with UltraESB; if not, please go over to our developer portal to have a quick look. Let’s first download and install Kibana and Elasticsearch from the following links:

https://www.elastic.co/downloads/past-releases/elasticsearch-5-1-1

https://www.elastic.co/downloads/past-releases/kibana-5-1-1

Elasticsearch Installation Guide

https://www.elastic.co/guide/en/elasticsearch/reference/5.1/_installation.html

Kibana Installation Guide

https://www.elastic.co/guide/en/kibana/5.1/install.html

Step 2 – Installing X-Pack to Kibana and Elasticsearch

X-Pack is an Elastic Stack extension that bundles security, alerting, monitoring, reporting, and graphing capabilities. Follow the guide in the following link for X-Pack installation to both Elasticsearch and Kibana. Note that this extension comes as a 30 day trial.

https://www.elastic.co/guide/en/x-pack/current/installing-xpack.html

Step 3 – Changing UltraESB-X metrics configuration

UE-X by default is configured to publish to an embedded Elasticsearch server. However, it is recommended to use an standalone Elasticsearch server in production. Therefore in this case we will change the metrics configuration to publish to an standalone ES server. Metrics configuration is in the XML file in UltraESB-X_HOME/conf/metrics/metrics.xml.

There, comment the bean,

<bean id="es-server" class="org.adroitlogic.x.metrics.EmbeddedElasticsearchServer"/>

And uncomment the bean,

<bean id="es-server" class="org.adroitlogic.x.metrics.ExternalElasticsearchServer">
    <constructor-arg>
        <map>
            <entry key="localhost" value="9300"/>
        </map>
    </constructor-arg>
    <constructor-arg value="elasticsearch" type="java.lang.String"/>
</bean>

Also for this case we’ll disable stats expiration. Change the rollover configuration section in es-publisher bean as follows.

<!--RollOver Configuration-->
<property name="indexRollOverEnabled" value="false"/>
<property name="maxDocs" value="1000"/>
<property name="maxAgeInMinutes" value="15"/>
<property name="rollOverTimePeriodInMinutes" value="10"/>
<property name="deleteOldestIndexAfterRollOver" value="false"/>
<property name="deleteInactiveIndicesAtInit" value="false"/>
<property name="deleteInactiveIndicesAtDestroy" value="false"/>

Final updated configuration is in the metrics.xml file.

Step 4 – Starting Elasticsearch, UltraESB-X and Kibana

Start Elasticsearch by running the elasticsearch script in ES_HOME/bin directory. Start Kibana by running kibana script in KIBANA_HOME/bin directory. Then start UltraESB-X. Kibana by default runs at http://localhost:5601. Default credentials will be username:elastic, password:changeme.

Note: X-Pack comes with a security extension. You can disable it by adding xpack.security.enabled: false to the elasticsearch.yml file in ES_HOME/conf directory. When the security extension is disabled, login will not be required.

Step 5 – Adding index template to Kibana

In Kibana we first need to add the index templates we are interested in. To do that, let’s go to Management tab in left pane and add an index template pattern ultraesb_x*. Once index pattern is entered Kibana will automatically pick the time-field name as recordTime as shown below.

Adding_index_template_ultraesb_x

Step 6 – Visualizations

Now we can add visualizations. In order to add visualizations it is better to have an understanding of the ultraesb_x index mapping first. Therefore please take a look and Elasticsearch mapping used is in the full-mapping.json file.

Now let’s add a line chart for received messages. Go to visualizations tab in the left pane and add a line chart by selecting Line Chart and ultraesb_x-* index pattern.

selecting_pattern_ultraesb_x

Now let’s configure x-axis as shown below.

configure_x

Then we can configure y-axis as follows.

configure_y

Finally we can apply the changes using the Play button. There you go, our first visualization is a success!

final_line_chart

Note that we can pick the time ranges from the top right button.

Stay tuned for more. Cheers! 🙂

~ Rajind Ruparathna

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s