Updated on 7/30/18
This article describes common issues that can explain why expected snapshots might be missing in the Controller. The issues are usually specific to the monitored application, its business transaction characteristics, and the load on the application.
There is a 10,000 UI limit for displaying snapshots. You cannot fetch additional snapshots when more than 10,000 will be displayed.
Be sure the time range selected for display matches the time range of the expected snapshots.
By default, AppDynamics collects a snapshot every ten minutes. You can modify the default values for slow, very slow, and stalled requests to generate snapshots more frequently.
However, changing any threshold or limit from the default value can cause additional overhead on both application and Controller performance. The predefined limits help ensure minimal impact to the application being monitored, so we don’t normally recommend adjusting them. See the documentation: Configure Snapshot Periodic Collection Frequency
Disabling Stall Transaction Detection can prevent incomplete transactions from being reported. If disabled, please re-enable it and select the checkbox to apply Stall Detection on existing business transactions.
For long-running transactions, the threshold values for slow and very slow might need to be set lower than the baseline response times.
We have seen cases where snapshots are abandoned when execution time is taking six times the default stall threshold. If snapshots are not captured for requests taking more than X amount of time, increase the stall threshold.
To maximize the value of snapshots or to avoid hitting agent or Controller limits, tune the threshold settings for specific business transactions (BTs) on the Configure -> "Slow Transaction Thresholds" screen.
The default snapshot thresholds might not be appropriate for BTs in some scenarios where:
By default, AppDynamics stores snapshots for up to two weeks (336 hours). This is configurable using the snapshots.retention.period setting. See the documentation: Database Size and Data Retention.
There are node properties that limit the number of snapshots per minute and the number of elements (calls) in the snapshot call graph:
The max-snapshots-per-minute limit is distributed across the BTs as needed and is not strictly divided evenly among them. This is a per-minute limit, not a per-BT limit.
You will see the log message below when max-call-elements-per-snapshot is exceeded. If seen, try increasing the limit by setting the property max-call-elements-per-snapshot on the node.
---------- [AD Thread Pool-CGG0] 11 Jul 2014 10:14:06,198 WARN Snapshot - Aborting snapshot started from org.jboss.web.tomcat.filters.ReplyHeaderFilter:doFilter [line -1] because the number of calls  exceeded threshold  -------------
The CONTROLLER_RSD_UPLOAD_LIMIT_REACHED event type indicates that your Controller has reached the limit for the number of RSDs (Request Segment Data) that can be uploaded per minute from this account. Once the limit is reached no more RSDs — other than certain key ones — are uploaded for that minute. This is a safety feature to avoid overhead on the agent nodes and overloading of the Controller. The counts (normal/slow...etc) for the requests are incremented irrespective of this warning.
You can tune the Controller settings using the rsds.upload.limit.per.min property. This value may be adjusted in the field as appropriate (without bouncing the Controller) and the new limits become effective within the next minute. However, we do not recommend doing this without first reviewing your threshold settings.
There are other limits at the Controller for the Data Buffers. If any snapshots are missing and the agent trace logs don't show any indication of dropping snapshots on the agent side, please enable the loggers below in the Controller (On-Prem) in the
logging.properties file located at
This message in the Controller server logs indicates that the data buffer limits have been reached in the Controller:
ExitCall Data Buffer full. Dropping data. Buffer Size: 2097152 bytes|#]
You can increase the Controller property values below from the Admin Console. The default value is 8 MB for each property. Please note that higher values will impact the Controller performance.
These changes should allow the snapshots to show in the UI—but if the snapshot size is huge, then the snapshot may not open and will show the query TIMEOUT exception in the UI. Please note that this query execution time can not be changed and this is the limitation at the Controller side to avoid the Controller performance issues.
To avoid the situation above, enable agent side tracing to figure out how much data a BT snapshot is pushing to the Controller and what data it is pushing. Based on the data, you can fine-tune your application or agent instrumentation to block a few of the less important data segments to the Controller.
Several node properties relate to transaction snapshots. Confirm that the settings are not affecting snapshot generation in your specific scenario. You can view a list of node properties here: App Agent Node Properties Reference
If you have reached the business transaction limit, confirm that you exclude the business transactions that you are not interested in. This is to make sure that snapshot/callgraph limits are not reached due to load on overflow BTs on the affected node.
Search the log files for strings such as "WARN Snapshot - Aborting snapshot" or error messages to see if there are related messages that can provide insight on the issue.
If you find the following error in the logs, there could be an issue with agent directory file permissions.
httpSSLWorkerThread-8080-12 20 Dec 2012 03:02:24,082 ERROR TransactionSnapshotService - Error in starting snapshot for Sampled error request java.lang.UnsupportedOperationException at com.singularity.ee.agent.appagent.services.transactionmonitor.common.uf.c(uf.java:260) at com.singularity.ee.agent.appagent.services.transactionmonitor.common.ye.w(ye.java:362) at com.singularity.ee.agent.appagent.services.snapshot.ac.<init>(ac.java:99) at com.singularity.ee.agent.appagent.services.transactionmonitor.common.ye.a(ye.java:195) at com.singularity.ee.agent.appagent.services.snapshot.u.a(u.java:423) at com.singularity.ee.agent.appagent.services.snapshot.jc.a(jc.java:437) at com.singularity.ee.agent.appagent.services.transactionmonitor.common.cg.a(cg.java:523) at com.singularity.ee.agent.appagent.services.transactionmonitor.common.c.a(c.java:391) at com.singularity.ee.agent.appagent.services.transactionmonitor.common.c.a(c.java:278) at com.singularity.ee.agent.appagent.services.bciengine.a.onMethodEnd(a.java:60) at com.singularity.ee.agent.appagent.entrypoint.bciengine.FastMethodInterceptorDelegator.safeOnMethodEndNoReentrantCheck(FastMethodInterceptorDelegator.java:204) at com.singularity.ee.agent.appagent.entrypoint.bciengine.FastMethodInterceptorDelegator.safeOnMethodEnd(FastMethodInterceptorDelegator.java:156) at com.sun.xml.ws.server.sei.EndpointMethodHandler.invoke(EndpointMethodHandler.java:275)
No snapshots are collected for 401 Exceptions because this is an HTTP Error 401: unauthorized access error. This error is handled before the entry point for any protocol (ASP/.NET/Web services/etc.). We capture it with its details as displayed but it is not possible to capture a snapshot since the error occurs too early in the IIS pipeline.
IIS rejects a request and returns 400 before we start instrumenting, so we are unable to capture the snapshot in the case of a 400 error. Review more details about debugging 400 errors here.