Symptoms
Transaction Analytics stops getting data and you see the types of errors listed below in the analytics-agent.log
file:
[2018-04-23T21:54:17,448+01:00] [WARN ] [pipeline-thread-14 (bt-publish-6)] [c.a.a.a.p.e.EventServicePublishStage] Transient error encountered due to the following cause: [Message could not be delivered because the REST resource rejected it]com.appdynamics.analytics.client.common.exceptions.NotAcceptableRestException: Status code: [406], Message: For action [EVENT_UPSERT], you have reached the events limit of [25000000] for account [xyz_bc1420f4-b665-4393-a4bb-e866dc948682] and event type group [biz_txn_v.*|custom_.*]
[2018-04-23T21:54:17,448+01:00] [WARN ] [pipeline-thread-14 (bt-publish-6)] [c.a.a.a.p.e.EventServicePublishStage] This message was throttled [13] times: Transient error encountered due to the following cause: [Message could not be delivered because the REST resource rejected it]
com.appdynamics.analytics.client.common.exceptions.NotAcceptableRestException: Status code: [406], Message: For action [EVENT_UPSERT], you have reached the events limit of [25000000] for account [xyz_bc1420f4-b665-4393-a4bb-e866dc948682] and event type group [biz_txn_v.*|custom_.*]
Diagnosis
Solution
To get past this issue, you need to choose one of the options below:
1) Review the traffic and remove the unimportant BTs from reporting into the Events Service. Essentially, you need to remove additional BTs from the Analytics configuration in order to bring down the total to within your daily limit. You can choose other business transactions to remove, but most effective ones to remove are the transactions that have high loads.
2) As a long term solution, you can increase the number of Transaction Analytics units in the license to allow for more events to be ingested. If you opt for this option, we suggest that you get in touch with your sales rep and work on this as this is going to be a license entitlement thing. Perhaps they can get you temporary units for a couple days to see what is the more realistic amount needed, and then you can decide how to proceed.