cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

JMX ActiveSession metric collection deactivated after Tomcat restart for few instances

Hello,

we have a Tiers application with 4 nodes corresponding to Tomcat applications. We restart these applications every evening at 22:00. For 2 of 4 nodes, metrics correspond to JMX.ActiveSession is no longer collected after startup until around 10:00 the day after. It is fixed without any action but comes again the day after.

4 Nodes are managed the same with chef deployment.

Here are versions of the components :

  • Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0
  • AppDynamics Controller build 22.10.1-611

Here is a screenshot extracted from Metrics Browser :

Missing_ActiveSeesion.pngMissing_ActiveSession_2nodes.png

Thanks for your suggestion and analysis of the potential root-cause.

3 REPLIES 3

Here are screenshot extracted from Metrics Browser :

Missing_ActiveSeesion.pngMissing_ActiveSession_2nodes.png

Here is part of agent log file with error raised :

[AD Thread-Metric Reporter1] 28 Nov 2022 22:21:27,700 WARN AgentErrorProcessor - Agent error occurred, [name,transformId]=[com.singularity.JMX.JMXDomainFinder - java.lang.ClassNotFoundException,2147483647]
[AD Thread-Metric Reporter1] 28 Nov 2022 22:21:27,700 WARN AgentErrorProcessor - 4 instance(s) remaining before error log is silenced
[AD Thread-Metric Reporter1] 28 Nov 2022 22:21:27,700 ERROR JMXDomainFinder - Error occurred while trying to query MBean Server during version check -
java.lang.ClassNotFoundException: com.ibm.ws.security.core.SecurityContext
at com.sun.jmx.mbeanserver.ClassLoaderRepositorySupport.loadClass(ClassLoaderRepositorySupport.java:232) ~[?:1.8.0_232]
at com.sun.jmx.mbeanserver.ClassLoaderRepositorySupport.loadClass(ClassLoaderRepositorySupport.java:144) ~[?:1.8.0_232]
at com.sun.jmx.mbeanserver.SecureClassLoaderRepository.loadClass(SecureClassLoaderRepository.java:52) ~[?:1.8.0_232]
at com.singularity.ee.agent.appagent.services.jmxservice.appserver.JMXDomainsFinder.tryLoadingClassFromMBeanServerClassLoaderRepository(JMXDomainsFinder.java:347) ~[appagent.jar:Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0]
at com.singularity.ee.agent.appagent.services.jmxservice.appserver.JMXDomainsFinder.getAttributeFromWebSphereMBeanServer(JMXDomainsFinder.java:328) ~[appagent.jar:Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0]
at com.singularity.ee.agent.appagent.services.jmxservice.appserver.JMXDomainsFinder.checkIfMBeanServerSupportsMBeanServerInfo(JMXDomainsFinder.java:254) ~[appagent.jar:Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0]
at com.singularity.ee.agent.appagent.services.jmxservice.appserver.JMXDomainsFinder.loadAndGetDomainMapping(JMXDomainsFinder.java:111) ~[appagent.jar:Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0]
at com.singularity.ee.agent.appagent.services.jmxservice.appserver.common.ServerMBeanManager.run(ServerMBeanManager.java:236) ~[appagent.jar:Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0]
at com.singularity.ee.util.javaspecific.scheduler.AgentScheduledExecutorServiceImpl$SafeRunnable.run(AgentScheduledExecutorServiceImpl.java:122) ~[appagent.jar:Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_232]
at com.singularity.ee.util.javaspecific.scheduler.ADFutureTask$Sync.innerRun(ADFutureTask.java:320) ~[appagent.jar:Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0]
at com.singularity.ee.util.javaspecific.scheduler.ADFutureTask.run(ADFutureTask.java:140) ~[appagent.jar:Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0]
at com.singularity.ee.util.javaspecific.scheduler.ADScheduledThreadPoolExecutor$ADScheduledFutureTask.access$301(ADScheduledThreadPoolExecutor.java:119) ~[appagent.jar:Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0]
at com.singularity.ee.util.javaspecific.scheduler.ADScheduledThreadPoolExecutor$ADScheduledFutureTask.run(ADScheduledThreadPoolExecutor.java:238) ~[appagent.jar:Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0]
at com.singularity.ee.util.javaspecific.scheduler.ADThreadPoolExecutor$Worker.runTask(ADThreadPoolExecutor.java:694) ~[appagent.jar:Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0]
at com.singularity.ee.util.javaspecific.scheduler.ADThreadPoolExecutor$Worker.run(ADThreadPoolExecutor.java:726) ~[appagent.jar:Server Agent #22.5.0.33845 v22.5.0 GA compatible with 4.4.1.0 rd9770530415f19f4c5154a80772b833db8dd7cee release/22.5.0]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]

Ryan.Paredez
Community Manager

Hi @Stephane.Babouhot,

I found this info. I hope it helps out or at leaves gives you a lead.

The JMX metrics are published by the JVM and javaagent fetches are reports those metrics (without any computation).

Could you please:

  • has a custom JMX rule configured for the activeSession?
  • check mbean browser (https://docs.appdynamics.com/display/PRO45/Configure+JMX+Metrics+from+MBeans - access mbean browser) to see what is the value of active session being reported there
  • check with the application team as to what is the exact mbean pattern where the application/container/server is publishing the activeSession. It could be possible the actual activeSession count is being published under some other mbean. In that case, we will need to configure custom JMX rule on that mbean and attribute.

Thanks,

Ryan, Cisco AppDynamics Community Manager




Found something helpful? Click the Accept as Solution button to help others find answers faster.

Liked something? Click the Thumbs Up button.



Check out Observabiity in Action

new deep dive videos weekly in the Knowledge Base.

Join Us On December 10
Learn how Splunk and AppDynamics are redefining observability


Register Now!

Observe and Explore
Dive into our Community Blog for the Latest Insights and Updates!


Read the blog here