cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Bhuvnesh.Kumar
Moderator
Moderator

AppDynamics Extensions Troubleshooting Guide

Before proceeding with this document, go through the Extensions Prerequisites Guide and verify whether the environment and agents being used have been set up correctly.

PLEASE NOTE | Not all steps may be applicable for your extension.

 

Contents


 

1. How do I build and place an extension?

To build the extension artifact from the source, follow the prerequisite and installation steps from individual extension documentation. 

Once the artifact is generated, extract it to the <MachineAgentHome>/monitors directory. It is important to verify the supported Controller, Machine Agent, and artifact versions from the extension's documentation.

 

2. How do I configure an extension? 

Every monitoring extension is shipped with a file called config.yml. The following steps must be verified for the config.yml:

A. Metric Prefix

Most config.yml files contain two metricPrefix sections which may look like the following (Considering the Redis Monitoring Extension as an example):

metricPrefix Prefix Recommended use
"Custom Metrics|Redis"

This metric prefix is recommended when SIM is enabled.

"Server|Component:<COMPONENT_ID>|Custom Metrics|Redis"

This metric prefix is recommended when you would like to publish metrics to the Machine Agent configured tier.


This metric prefix points to one specific component (tier). The
COMPONENT_ID can be found by using these steps:

  1. Go to your Controller
  2. Select the application associated with your Machine Agent
  3. Select Tiers and Nodes, then select the exact tier corresponding to your Machine Agent
  4. Once you select the tier, click the Details button to open the Dashboard
  5. Once you reach the Dashboard, take a look at the URL and find the word Component and the value corresponding to it
  6. That value should be a number and is the tier ID or Component ID that you need to update as your Component_ID value in the config

NOTE | The component ID, or component name, is the same as the tier ID, or tier name. To get the component ID, go to the URL under Applications > Tiers & Nodes > Dashboard, as shown in the image, below.
Applications > For your component ID URL, go to Applications > Tiers & Nodes > Dashboard, then copy the URL fieldApplications > For your component ID URL, go to Applications > Tiers & Nodes > Dashboard, then copy the URL field

IMPORTANT | The component ID (or component name) specified in the metric prefix of the extension should match the corresponding tier of the Machine Agent. The extension cannot report to a different tier than its Machine Agent's tier. 

LEARN MORE | To read about advanced metric prefix configuration scenarios, please refer to the Advanced Metric Prefix Configuration Scenarios for AppDynamics Monitoring Extensions Knowledge Base article.

 

B. How do I format the config.yml?

After verifying the metric prefix and configuring the details of the artifact monitored by the extension, ensure that the config.yml is correctly formatted. The use of Tabs is not supported by YML files. You can validate the file using an online YML validator like https://codebeautify.org/yaml-validator.  

 

C. How do I modify the monitor.xml file?

Another file present in the extracted extension's directory is the monitor.xml. In most cases, this file does not need any changes. If the monitor.xml does need to be modified, complete the following steps:

Execution frequency An extension ships with a default execution frequency of 60 seconds. The highest execution frequency supported by monitoring extensions is 5 minutes (300 seconds). It is important to ensure that this limit is not exceeded, as it will lead to inconsistencies in metric values.
Path to config

Ensure that the path to the config.yml is correctly specified under the following tag:

<task-arguments> <argument name="config-file" is-required="true" default-value="monitors/RedisMonitor/config.yml" /> </task-arguments>

An absolute path can also be used here.

 

3. What do I need to know about the Machine Agent restart?

At this point, ensure that the Machine Agent has restarted. 

Deploying new extensions and making changes to the monitor.xml for previously deployed extensions require a Machine Agent restart. However, simply changing the contents of a previously deployed extension's config.yml does not require a Machine Agent restart.

 

4. Where do I look for custom metrics?

Once everything has been configured correctly, follow the relevant steps below to look for Custom Metrics in the Controller:

A. When the SIM is enabled

  1. If SIM is enabled, navigate to the Servers tab in the main menu bar.
  2. Click on Metric Browser at the bottom-left. A new window should open up.
  3. Expand the 'Root' server, or alternatively, the server configured under the 'Machine Path' section of the <MachineAgentHome>/conf/controller-info.xml.
  4. Select Custom Metrics. Metrics from the configured extensions should show up here, in their respective directories.

B. Machine Agent (When the SIM is disabled)

  1. Find the Application associated with the Machine Agent under the Applications tab in the main menu bar.
  2. Click on Metric Browser at the bottom-left. A new window should open up.
  3. Select Application Infrastructure Performance, and look for the Tier associated with the Machine Agent
  4. Under this tier, look for Custom Metrics. Metrics from the configured extensions should show up here, in their respective directories.
  5. The data in the previous point is a tier-level value, an aggregation of data represented by each node under this tier. To get metrics pertaining to a specific node, the Individual Nodes section must be used.

 

5. Metric Limits

What if you followed all the steps up to this point, but still don’t see all the required metrics in the Metric Browser? 

It could be a case of the Machine Agent's metric limit being hit. In this case, you should restart it with a higher metric limit. For example, to increase the limit to 5000, you can use the following command while starting the Machine Agent:

 

java -Dappdynamics.agent.maxMetrics=5000 -jar machineagent.jar

 

However, this limit should not be an arbitrarily large number. It is a factor of the number of metrics generated by any installed extensions, heap memory provided to the Machine Agent, etc. Check the hardware and sizing requirements for Machine Agents and extensions in the Machine Agent Requirements and Supported Environments documentation.

For more information, see the latest Metric Limits documentation.

 

Increasing metric limits in Windows

For Windows, where the Machine Agent is running as a service:

  1. Uninstall/delete the already installed service
  2. Use this command to increase metric limit
    cscript <machine_agent_home>\InstallService.vbs -Dappdynamics.agent.maxMetrics=5000


If at this point the extension’s metrics are still not visible, please refer to the extension-specific troubleshooting steps (if any) found on the respective extension documentation pages.

 

Additional troubleshooting resources

Comments
Anjana.Mohan
Producer

Very informative. 

 I can still not see the metrics from our MQ servers in the metric browser.

I currently do not have enough rights to raise an appdynamics ticket even though i raised this concern several times with the support team.(info-appdynamics)

How do i send across the logs and details so that somebody from help support can analyse and give a solution ? Please help.

Bhuvnesh.Kumar
Moderator
Moderator

HI Anjana, 

Please contact your Account manager to get access to file tickets. Once you have that, please feel free to file tickets about the issues you are facing. You can also send an email to help@appdynamics.com if you would like assistance on any issues as well.

Thanks,

Bhuvnesh

Mark.Kelly-Smith
AppDynamics Team

Section 4 needs an update - its implication is contrary to the expected effect.

 

Apache Extension sums the actual effect nicely;

 

#This will create this metric in all the tiers, under this alias
metricPrefix: Custom Metrics|Apache|

 

#This will create it in specific Tier/Component.
metricPrefix: Server|Component:<COMPONENT_ID>|Custom Metrics|Apache|

The issue with the first option above - which is the articles suggested option B, for SIM agents - is that the custom metrics folder appears EVERYWHERE, not just under the Servers individual node (which is the implication)

Abhishek.Malviya
Wanderer

Hi Guys

 

Please help us in resolving the Zabbix integration issue :

We are getting user authentication issue in connecting AppD with the Zabbix server

We are getting the below log in the machine agent:-

 

[Agent-Monitor-Scheduler-2] 26 Jul 2019 02:43:39,052 INFO PathResolver - Install dir resolved to /opt/machineagent-bundle-64bit-linux-4.5.9.2096
[Agent-Monitor-Scheduler-2] 26 Jul 2019 02:43:39,071 ERROR ZabbixMonitor - Error while connecting to Zabbix
com.zabbix4j.ZabbixApiException: com.zabbix4j.ZabbixApiException: HTTP Request Error
at com.zabbix4j.user.User.login(User.java:53)
at com.zabbix4j.ZabbixApi.login(ZabbixApi.java:89)
at com.appdynamics.monitors.zabbix.ZabbixMonitor.createZabbixAPI(ZabbixMonitor.java:135)
at com.appdynamics.monitors.zabbix.ZabbixMonitor.populateStats(ZabbixMonitor.java:116)
at com.appdynamics.monitors.zabbix.ZabbixMonitor.execute(ZabbixMonitor.java:64)
at com.singularity.ee.agent.systemagent.components.monitormanager.managed.MonitorTaskRunner.runTask(MonitorTaskRunner.java:148)
at com.singularity.ee.agent.systemagent.components.monitormanager.managed.PeriodicTaskRunner.runTask(PeriodicTaskRunner.java:86)
at com.singularity.ee.agent.systemagent.components.monitormanager.managed.PeriodicTaskRunner.run(PeriodicTaskRunner.java:47)
at com.singularity.ee.util.javaspecific.scheduler.AgentScheduledExecutorServiceImpl$SafeRunnable.run(AgentScheduledExecutorServiceImpl.java:122)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.singularity.ee.util.javaspecific.scheduler.ADFutureTask$Sync.innerRunAndReset(ADFutureTask.java:335)
at com.singularity.ee.util.javaspecific.scheduler.ADFutureTask.runAndReset(ADFutureTask.java:152)
at com.singularity.ee.util.javaspecific.scheduler.ADScheduledThreadPoolExecutor$ADScheduledFutureTask.access$101(ADScheduledThreadPoolExecutor.java:119)
at com.singularity.ee.util.javaspecific.scheduler.ADScheduledThreadPoolExecutor$ADScheduledFutureTask.runPeriodic(ADScheduledThreadPoolExecutor.java:206)
at com.singularity.ee.util.javaspecific.scheduler.ADScheduledThreadPoolExecutor$ADScheduledFutureTask.run(ADScheduledThreadPoolExecutor.java:236)
at com.singularity.ee.util.javaspecific.scheduler.ADThreadPoolExecutor$Worker.runTask(ADThreadPoolExecutor.java:694)
at com.singularity.ee.util.javaspecific.scheduler.ADThreadPoolExecutor$Worker.run(ADThreadPoolExecutor.java:726)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.zabbix4j.ZabbixApiException: HTTP Request Error
at com.zabbix4j.ZabbixApiMethod.sendRequest(ZabbixApiMethod.java:75)
at com.zabbix4j.user.User.login(User.java:49)
... 17 more
[Agent-Monitor-Scheduler-2] 26 Jul 2019 02:43:39,072 ERROR ZabbixMonitor - Metrics Collection Failed:
com.singularity.ee.agent.systemagent.api.exception.TaskExecutionException: Error while connecting to Zabbix
at com.appdynamics.monitors.zabbix.ZabbixMonitor.createZabbixAPI(ZabbixMonitor.java:139)
at com.appdynamics.monitors.zabbix.ZabbixMonitor.populateStats(ZabbixMonitor.java:116)
at com.appdynamics.monitors.zabbix.ZabbixMonitor.execute(ZabbixMonitor.java:64)
at com.singularity.ee.agent.systemagent.components.monitormanager.managed.MonitorTaskRunner.runTask(MonitorTaskRunner.java:148)
at com.singularity.ee.agent.systemagent.components.monitormanager.managed.PeriodicTaskRunner.runTask(PeriodicTaskRunner.java:86)
at com.singularity.ee.agent.systemagent.components.monitormanager.managed.PeriodicTaskRunner.run(PeriodicTaskRunner.java:47)
at com.singularity.ee.util.javaspecific.scheduler.AgentScheduledExecutorServiceImpl$SafeRunnable.run(AgentScheduledExecutorServiceImpl.java:122)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.singularity.ee.util.javaspecific.scheduler.ADFutureTask$Sync.innerRunAndReset(ADFutureTask.java:335)
at com.singularity.ee.util.javaspecific.scheduler.ADFutureTask.runAndReset(ADFutureTask.java:152)
at com.singularity.ee.util.javaspecific.scheduler.ADScheduledThreadPoolExecutor$ADScheduledFutureTask.access$101(ADScheduledThreadPoolExecutor.java:119)
at com.singularity.ee.util.javaspecific.scheduler.ADScheduledThreadPoolExecutor$ADScheduledFutureTask.runPeriodic(ADScheduledThreadPoolExecutor.java:206)
at com.singularity.ee.util.javaspecific.scheduler.ADScheduledThreadPoolExecutor$ADScheduledFutureTask.run(ADScheduledThreadPoolExecutor.java:236)
at com.singularity.ee.util.javaspecific.scheduler.ADThreadPoolExecutor$Worker.runTask(ADThreadPoolExecutor.java:694)
at com.singularity.ee.util.javaspecific.scheduler.ADThreadPoolExecutor$Worker.run(ADThreadPoolExecutor.java:726)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.zabbix4j.ZabbixApiException: com.zabbix4j.ZabbixApiException: HTTP Request Error
at com.zabbix4j.user.User.login(User.java:53)
at com.zabbix4j.ZabbixApi.login(ZabbixApi.java:89)
at com.appdynamics.monitors.zabbix.ZabbixMonitor.createZabbixAPI(ZabbixMonitor.java:135)
... 15 more
Caused by: com.zabbix4j.ZabbixApiException: HTTP Request Error
at com.zabbix4j.ZabbixApiMethod.sendRequest(ZabbixApiMethod.java:75)
at com.zabbix4j.user.User.login(User.java:49)
... 17 more
[Agent-Monitor-Scheduler-2] 26 Jul 2019 02:43:39,073 ERROR PeriodicTaskRunner - Error creating environment task
com.singularity.ee.agent.systemagent.api.exception.TaskExecutionException: Zabbix Monitor completed with failures
at com.appdynamics.monitors.zabbix.ZabbixMonitor.execute(ZabbixMonitor.java:75)
at com.singularity.ee.agent.systemagent.components.monitormanager.managed.MonitorTaskRunner.runTask(MonitorTaskRunner.java:148)
at com.singularity.ee.agent.systemagent.components.monitormanager.managed.PeriodicTaskRunner.runTask(PeriodicTaskRunner.java:86)
at com.singularity.ee.agent.systemagent.components.monitormanager.managed.PeriodicTaskRunner.run(PeriodicTaskRunner.java:47)
at com.singularity.ee.util.javaspecific.scheduler.AgentScheduledExecutorServiceImpl$SafeRunnable.run(AgentScheduledExecutorServiceImpl.java:122)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.singularity.ee.util.javaspecific.scheduler.ADFutureTask$Sync.innerRunAndReset(ADFutureTask.java:335)
at com.singularity.ee.util.javaspecific.scheduler.ADFutureTask.runAndReset(ADFutureTask.java:152)
at com.singularity.ee.util.javaspecific.scheduler.ADScheduledThreadPoolExecutor$ADScheduledFutureTask.access$101(ADScheduledThreadPoolExecutor.java:119)
at com.singularity.ee.util.javaspecific.scheduler.ADScheduledThreadPoolExecutor$ADScheduledFutureTask.runPeriodic(ADScheduledThreadPoolExecutor.java:206)
at com.singularity.ee.util.javaspecific.scheduler.ADScheduledThreadPoolExecutor$ADScheduledFutureTask.run(ADScheduledThreadPoolExecutor.java:236)
at com.singularity.ee.util.javaspecific.scheduler.ADThreadPoolExecutor$Worker.runTask(ADThreadPoolExecutor.java:694)
at com.singularity.ee.util.javaspecific.scheduler.ADThreadPoolExecutor$Worker.run(ADThreadPoolExecutor.java:726)
at java.lang.Thread.run(Thread.java:748)

 

 

Please help us in resolving the issue

 

Thanks

 

Regards 

Abhishek

Rafi.Shaik
Explorer

Hi Team,

i receiving below error for 'Resource Manger'. but, 'Ambari Metrics' are working fine for same user credentials.. all values are inplace. 

Please help to resolve the issue..

 

[Monitor-Task-Thread1] 10 Oct 2019 08:07:32,217 ERROR HttpClientUtils - The status line for the url [http://rm1.example.com:8088/ws/v1/cluster/apps?applicationTypes=MAPREDUCE&startedTimeBegin=157069095...] is [HTTP/1.1 401 Authentication required] and the headers are [[Cache-Control: must-revalidate,no-cache,no-store, Date: Thu, 10 Oct 2019 07:07:32 GMT, Pragma: no-cache, Date: Thu, 10 Oct 2019 07:07:32 GMT, Pragma: no-cache, Content-Type: text/html; charset=iso-8859-1, WWW-Authenticate: Negotiate, Set-Cookie: hadoop.auth=; Path=/; Domain=example.com; HttpOnly, Content-Length: 1411, Server: Jetty(6.1.26.hwx)]]
[Monitor-Task-Thread1] 10 Oct 2019 08:07:32,217 ERROR HttpClientUtils - The contents are <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 401 Authentication required</title>
</head>
<body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /ws/v1/cluster/apps. Reason:
<pre> Authentication required</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>

</body>
</html>

Vikrant.Malik
New Member

Hello,

 

We are trying to set up MQ monitoring in AppDynamics. 

In Appd, MQ channel running status has value= 5, we are trying to set up alerts like, if the value is more than 5, ie if the channel goes to Retrying mode then only its trigger alert.

 

But we getting alerts even if the value is 0, ie channels is INACTIVE. 

 

Can you advise which option should we choose? 

Hi Vikrant, 

 

Thanks for posting in the AppDynamics Community!

 

Can you please open a ticket with AppDynamics support by going to www.appdynamics.com/support and clicking Support Portal? We will make sure that it gets assigned to the right team and is resolved ASAP. 

 

Thanks, 

Vishaka Sekar

 

revised method of opening Support ticket
C. Landivar, Community Manager

Yakov.Keranchuk
AppDynamics Team

It's good to automate getting the COMPONENT_ID on the extension. 

Claudia.Landivar
Community Manager

Hello, @Yakov.Keranchuk 

Thank you for pointing out that detail!

Claudia Landivar

Community Manager & Editor