Monitoring JMX through http

Monitoring JMX data has been always a painful process, especially in production systems where the ability to open ports and/or allow certain kind of traffic is kind of "complicated".

Thanks to Jolokia we have a lightweight layer on top of the MBean Server infrastructure which allows to query our JMX data over the http protocol. The architecture is very simple:

I have borrowed the previous image from this presentation where you can see a great overview of the Jolokia's benefits.

How can I use it in Liferay?

The Jolokia guys have an OSGi bundle ready to be deployed inside an OSGi container. The only dependency of the previous bundle is that an Http Service implementation should be available within the container, but this is not a problem since we already provide our own implementation fo the previous specification (it will be bundled by default from 6.2.0 and above). The provided bundle needs a small modification which I have already done for this blog entry and you can get here ( I need to do it configurable, but for the purpose of this post we are fine with this artifact).

Get the previous bundle and copy it to your deploy folder; in a few seconds your bundle will be deployed into the OSGi container and you will have a fully working http layer on top of your JMX data.

Querying some data

There are a few clients you can use to query the data through this Http layer (even if you want, you can write your own :) ): 

• http://hawt.io/ Nice HTML5 extensible dashboard with many available plugins

• j4psh: Unix like shell with command completion

• jmx4perl: CLI tool for scripting

• check_jmx4perl: Nagios plugin

• AccessJ: and iPhone client

Let's see an small example of how to use the j4psh shell:

First of all we need to connect to our server

As I have told you before the current bundle is not configurable and you should use the /o/jolokia/jolokia path (hope this is not a big deal for you). Once we are connected we can navigate through our JMX data (j4psh structures the info using a a "file system approach").

For example, if we type ls, we can see something like this:

Or we can search for a certain info and get the current value, for example the number of total loaded classes into the JVM

As you can see this is an extremely powerful and lightweight tool we can use to monitor our data in a very simple way.

I will try to clean-up a little bit the bundle used in this post in order to make it configurable and I will try to publish it in a more accessible place.

See you soon!

[Denis Signoretto]

Hi Miguel,

thanks for your interesting article !!!.

I've tryed to copy your version of jolokia osgi bundle to $LR_HOME\deploy but during deploy I get the following error:

15:31:26,764 INFO [com.liferay.portal.kernel.deploy.auto.AutoDeployScanner][ModuleAutoDeployListener:69] Module for C:\Users\denis\liferay\liferay-portal-6.2.0-ce-ga1\deploy\jolokia-osgi-1.1.1.jar copied successfully. Deployment will start in a few seconds.
15:31:26,765 ERROR [com.liferay.portal.kernel.deploy.auto.AutoDeployScanner][AutoDeployDir:217] Auto deploy failed to remove jolokia-osgi-1.1.1.jar
15:31:26,765 INFO [com.liferay.portal.kernel.deploy.auto.AutoDeployScanner][AutoDeployDir:224] Add jolokia-osgi-1.1.1.jar to the blacklist

I can found jolokia-osgi-1.1.1.jar in data\osgi\modules but at the same time it's not removed from deploy dir and deployment fails.
I've seen the file in deploy dir it's locked by JVM (I'm running Liferay 6.2.0 GA1 with Oracle/SUN JVM 1.6.0_45 on Windows 8.1 64-bit) and it's not deleted.

Do you have any suggestion ?

Thanks,
Denis.

[Andrey Filippov]

Hi Miguel,
Thanks for this article. Although I put jolokia.jar into deploy and I could see it in a Felix as active: 7|Active | 1|Jolokia Agent (1.1.1) I could not do j4psh http://localhost:8080/o/jolokia/jolokia - it returns 503 Service Unavailable. Getting http://localhost:8080/o/jolokia/jolokia in the browser also failed with HTTP Status 503 - Module framework is unavailable. Do you happen to know what could it be?

[Andrey Filippov]

Ping...