2 minutes

Need:

We had a need to expose JVM parameters and other application specific metrics for monitoring.
Altering the code was not an option as we needed a systematic solution to deploy on all our servers without rewriting all projects.

Solution:

As we were already using Prometheus the logical path was to search for a tool which we would easily integrate with an existing solution.
I stumbled upon Prometheus JMX_EXPORTER, a process for exposing JMX Beans via HTTP for Prometheus consumption. Official git page of jmx_exporter can be found here.
With this solution I was able to export any JMX bean on a specific JVM (database info, bean info, JVM parameters, memory, heap, http statistics)

Problems:

The whole project had very little documentation. There is some information on the official git page but unfortunately out of the box setup and configuration didn’t work on Jboss EAP at all.
Even when searching outside the git you could find non info/guilds on how to approach this matter. Eventually after merging few solutions I found on the internet I was able to run and example on Wildfly 12.0.0 in standalone.
Unfortunately this was not enough as all our production servers are running in domain mode. After a few more hours and a little bit of luck (thanks to my college Aleksandar Brankovic) I was done.

• First step:

All the files that we are going to be used should be located on host servers and placed in the matching folders. You can also place the files on a shared folder so you don’t need to copy/paste them on all of your servers.
Place jmx_prometheus_javaagent.jar that you can find on the official git page and copy it to the $JBOSS_HOME/bin
Place jmx_exporter_conf.yaml $JBOSS_HOME/domain/configuration/
Here is the most simple configuration file that will just be used for this test:

• Second step:

We need to propagate the -javaagent configuration to all server registered on the domain controller and we need to add other specific configuration that needs to be added in order to fix class loader problems that are present when providing a custom javaagent. Pass these configuration to your domain.xml of domain controller located at $JBOSS_HOME/domain/configuration/domain.xml

At the time of the writing most recent version of WF had jboss-logmanager-2.0.9.Final so in your case check which version is present and change the configuration appropriately. With this configuration metrics will be available on all interfaces on port 58080 as jmx_exporter bind to all interfaces if not provided otherwise. Here is the syntax:

• Third step:

We need to start our host server and pass the one last argument. In most plain example you start you server via domain.sh file located on your host.

After this you should be able to access your metrics on localhost:58080. Later you can pass this location to your Prometheus at tell it to gather the data.