How do I troubleshoot high CPU usage?

[Preet.Gaur]

Use the following recommendations if high CPU usage occurs after attaching the AppDynamics agent:

1. Disable aggressive slow snapshot collection
2. Exclude specific hotspot interceptors
3. Async Instrumentation
4. Disable unwanted instrumentation from transaction detection
5. Turn off Turbo custom exit point interceptor

1. Disable aggressive slow snapshot collection

If aggressive slow snapshot collection is enabled, the agent will retain selected call graph segments that precede the detection of a slow, stall or error transaction condition. Disabling the aggressive slow snapshot collection will significantly reduce overhead.

1. In the Controller UI, click on Configuration in the left navigation bar.
2. Click the double arrows in the top right to expand the menu.
3. Click on Call Graph Settings.
4. Uncheck the checkbox next to "Enable Aggressive Slow Snapshot Collection."
5. Click the save button. 

2. Exclude specific hotspot interceptors 

Editing the node level property "exclude-interceptors" to exclude hotspots will reduce CPU usage.

To determine if your agent has enabled hotspots, search the agent.* log files for "Enabled hotspots" and search the ByteCodeTransformer*.log for "Applying method interceptor diag.snapshot.BoundHotspot."

Edit a node property:

1. In the controller UI, click on Tiers and Nodes in the left navigation bar.
2. Double click on the node you'd like to configure, and click Agents > App Server Agent > Configure. This will open the App Server Configuration window. 
3. At the node level, click on the "Use Custom Configuration" button.
4. Either search for the exclude-interceptors property and double click on it, or click on the gray plus sign to create a new agent property if the exclude-interceptors property does not already exist.
5. Click the save button after adding the new configuration. A "saved successfully" message will appear in the top right corner of the window. 

Name - exclude-interceptors
Description - exclude-interceptors
Type - String
Value - com.singularity.BoundHotspotInterceptor

Note: If there are already some class names added to this property, use a comma between them.

Example:

3. Async Instrumentation

Too many async interceptors can cause high CPU utilization.

You may have unnecessary application framework classes instrumented that are not needed for visibility or monitoring. Get a list of classes/methods which are applying async interceptor by looking at the ByteCodeTransformer*.log or search "async.handoffAsyncHandOffExecutionTracker." Exclude the packages 1-1 to improve CPU utilization.

The package/class can be excluded either from fork-config OR using node property.

Fork-config changes (recommended) - Requires JVM restart.

1. Navigate to <agent_install_dir>/<ver4.X.x>/conf/app-agent-config.xml.
2. Search for <fork-config> keyword.
3. Add the desired packages to be excluded. 

Example:

<excludes filter-type="STARTSWITH" filter-value="<package/>"/> or fully qualified class name

Few Examples:
<excludes filter-type="STARTSWITH" filter-value="com.arjuna/"/>
<excludes filter-type="STARTSWITH" filter-value="com.netflix/"/>
<excludes filter-type="STARTSWITH" filter-value="com.bea/,com.weblogic/,weblogic/,com.ibm/,net/sf/,com/mchange"/>

Node Property - See step 2 to create or edit a node property.

Name: thread-correlation-classes-exclude
Description: thread-correlation-classes-exclude
Value:Type: <fully.qualified.package.ClassName>,<package.name>.*

Note: Excluding packages and classes could result in loss of visibility. 

 
4. Disable unwanted instrumentation from transaction detection

Spring Bean and EJB interceptors are CPU intensive. If you are not looking for any spring or EJB entry points, disable Transaction Detection. 

1. From the controller UI, click on Applications.
2. Click Configuration in the left navigation.
3. Under the Transaction Detection tab, uncheck both of the checkboxes next to Spring Bean and EJB to disable them.

5. Turn off Turbo custom exit point interceptor 

Turbo exit points are special interceptors that handle a high volume of calls. To determine if these interceptors are causing high CPU usage, search the ByteCodeTransformer*.log for "exit.TurboCustom." If the classes are not adding visibility, exclude them and disable the interceptors by creating node-level properties (see "Edit a node property" in step 2).

Name - exclude-interceptors
Description - exclude-interceptors
Type - String
Value - com.singularity.TurboCustomExitPointInterceptor

Name - disable-ootb-turbo-interceptors
Description - disable-ootb-turbo-interceptors
Type - boolean
Value - true

If these troubleshooting recommendations do not improve your high CPU usage, contact our support team for further assistance.