How do I use jConsole to test and troubleshoot connectivity issues?

[Vishaka.Balasubramanian Sekar]

Using the jConsole utility for connectivity testing and troubleshooting

jConsole is a utility that comes with the Java Development Kit (JDK). It helps to test the connection between the extension and the JMX artifact being monitored. You can use jConsole to troubleshoot connection issues in the extension. You can also use it to configure the MBeans the extension collects.

This article walks you through using jConsole with the artifact that you want to monitor.

Prerequisite

The JDK needs to be installed on the machine where the extension is installed.

Table of Contents

How do I get started with jConsole?
How do I troubleshoot JMX connection fails?
How do I view and configure the JMX MBeans?

How do I get started with jConsole?

To start using jConsole with a JMX artifact, you'll need to launch jConsole, and then connect the artifact to it.

Step 1: Launch jConsole

1. Open a terminal from the machine where the extension is installed.
2.  If JMX monitoring over SSL is not enabled, type jconsole in the terminal.

3. If you have enabled JMX monitoring via SSL on the artifact, you need to run jConsole as follows.

jconsole -J-Djavax.net.ssl.trustStore=/path/to/truststore \

-J-Djavax.net.ssl.trustStorePassword=trustword 

4. This will open a  jConsole window.

Step 2: Connect to the artifact.

1. On the jConsole window, select “Remote Process”.
   
   
2. You can enter the JMX service URL from either the config.yml or a host JMX-port pair to connect to the artifact.
   
   If you are using the JMX service URL, the format is:
   service:jmx:rmi:///jndi/rmi://host:JMXport/jmxrmi

jConsole Window, Remote Process, JMX service URL format

If you're choosing the host and JMX port to connect, the format of the Host JMX-port pair is
<IP address of the machine on which the artifact is running>:<JMX-port>jConsole window, Remote Process, host and JMX-port pair format

For example: 

34.222.32.114:9999
localhost:9999

4. Only provide a Username and Password if authentication is enabled on the artifact you’re installing. 
   
   The JMX username and JMX password are not the same as the credentials used to connect to the artifact via REST API, terminal. If you don’t have password authentication enabled on the artifact that you’re trying to monitor, please leave the fields blank.
   
   
5. Click the “Connect” button.
   
   If monitoring via SSL is not enabled, you will see a “Secure connection failed” pop-up window. Click the "Insecure Connection" button.
   

If SSL monitoring is not enabled, a "Secure connection failed" pop-up window will appear. Click the "Insecure Connection" button.
If the configuration is correct, the connection between the extension and the artifact will be successful, and metrics will be reported. 

If you see any error messages, please refer to the  How do I troubleshoot JMX connection fails? section below. 

Step 3: Setting up the Machine Agent after a successful jConsole Connection

1. If your connection was successful, you can now set up the Machine Agent. 
   
2. If there is a problem with your connection, please go to How do I troubleshoot JMX connection fails?
   below
3. If you are using JMX over SSL, then you need to add the same SSL parameters to your Machine Agent startup command:

java -Djavax.net.ssl.trustStore=/path/to/truststore \ -Djavax.net.ssl.trustStorePassword=trustword <other parameters go here> -jar machine-agent.jar

How do I troubleshoot JMX connection fails?

If you see a jConsole connection failing, as shown with the “connection refused” message, it might be caused by one or more of the following reasons: 

• JMX monitoring is not enabled. See the extension documentation for the steps to enable JMX monitoring.
• The serviceUrl may be incorrect.
• The hostname/port pair may be incorrect.
• SSL may be required, and the jConsole was launched in non-SSL mode.
• JMX port might not be configured on the artifact.
• The username or password may be incorrect.
• A proxy or firewall between the machine on which jConsole was launched and the artifact may be blocking incoming JMX connections.

How do I view and configure the JMX MBeans?

Once the connection is successful (starting from the Java Monitoring and Management Console), navigate to the MBeans tab.

MBeans tab in the Java monitoring and management console

The extension is configured to fetch some metrics by default. But if you find that the metrics that you want aren’t listed in config.yml by default, you can configure config.yml to add them. 

Once you have followed How do I get started with jConsole steps 1 and 2, the jConsole should be up. You should see the MBean Explorer on the left pane of the jConsole window. Each MBean has a few metrics listed underneath it: navigate through the MBeans down to the metric(s) you're interested in.

Examples of metrics you can track

As an example, let's consider using the Kafka monitoring extension to track metrics:

• How do I use Kafka to see the number under-replicated partitions?
• How do I use Kafka with multiple metrics under a single object?
• How do I use wildcards with Kafka?

How do I use Kafka to see the number of under-replicated partitions?

You can use Kafka to see the number of UnderReplicatedPartitions in your partition.

Add the MBean in your config.yml MBeans:

<<Other MBeans>>
<<add your MBean here>>
<<Example of adding UnderReplicatedPartitions.Value here>>
 - objectName: 'kafka.server:type=ReplicaManager,name=UnderReplicatedPartitions'
   metrics:
     - Value:
         alias: 'Value'
         multiplier: ''
         delta: false
         aggregationType: 'OBSERVATION'
         timeRollUpType: 'CURRENT'
clusterRollUpType: 'INDIVIDUAL'

How do I use Kafka with multiple metrics under a single object?

Let’s consider another case where there are multiple metrics under one object:FailedIsrUpdatesPerSec. Those metrics are listed under kafka.server -> Replica Manager ->object:FailedIsrUpdatesPerSec -> Attributes.

For example, to fetch all metrics(stats) about FailedIsrUpdatesPerSec (Count, MeanRate, OneMinuteRate, etc.), list each of them in the metrics section of the FailedIsrUpdatesPerSec. The Value metric is listed as an example.

How do I use wildcards with Kafka?

If you want to add all the MBeans under ReplicaManager, you can add a wildcard (a *) in your config.yml MBeans:

<<Other MBeans>>
<<add your MBean here>>
<<Example of adding UnderReplicatedPartitions.Value here>>
 - objectName: 'kafka.server:type=ReplicaManager,name=*'
   metrics:
     - Value:
         alias: 'Value'
         multiplier: ''
         delta: false
         aggregationType: 'OBSERVATION'
         timeRollUpType: 'CURRENT'
         clusterRollUpType: 'INDIVIDUAL'

The extension can only collect metrics if the JMX connection is successful.