Discussion Feed

Problem: Discovery of the Events Service node using Enterprise Console (EC) fails. Upon checking the EC logs, we see the below error: Events Service discovery failed.  Failure occurred at stage: Enable transient allocations on masters  Error message:  Task failed: Enable transient allocation in the api store node    Analysis: This is a very generic error that occurs if the Events Service is failing on its respective node. To get the actual cause of issue, we need to check the events-service-api-store.log . In the Events Service log for this instance, we see the below error: [2018-08-07T07:08:42,738+02:00]  [ERROR]  [ProcessQueue0]  [c.a.a.p.e.e.ElasticSearchExecutor]  Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000004e32c0000, 12294815744, 0) failed; error='Cannot allocate memory' (errno=12)   The system did not have enough memory to allocate the process. Hence, the start of the process failed so the discovery process failed, as well.   Solution: Ensure that there is enough space available for the Events Service and elastic search process combined so that the process can start and the discovery process will complete successfully. ... View more

Question  Can I enable multi-factor authentication for AppDynamics mobile applications?   Answer Currently, there is no way to configure multi factor authentication for an AppDynamics mobile application. We only have direct username/password-based or SAML login for the mobile app.  ... View more

Problem: Analytics data being collected is a lot less than the actual load on the application and you are seeing repetitive messages like the one below in the Analytics Agent log: [2018-05-21T11:24:38,137+01:00]  [WARN ]  [pipeline-thread-12 (bt-publish-4)]  [c.a.a.a.p.e.EventServicePublishStage]  The message at index [0] failed due to the following permanent error: [Failed [failed to parse [segments.userData.xxxx]] with id [484xxx-63xx-4xxx-axxx-47d08xxxxx] index [customer1_xxxxxxx___biz_txn_v1___2018-05-19_16-38-20] type [customer1_xxxxx___biz_txn_v1]]   Analysis: The same MIDC is defined for multiple applications. We observed that the field that is registered with the Events Service has a different type than the field that is being collected for the current application. The field type can be retrieved using the below query:   curl -XGET http://localhost:9200/<global_account_name>___biz_txn_v1*/_mapping?pretty=true   If same field name is being used across multiple MIDC, the Events Service will use the first registered field type. If the field type for a new application is different from the registered one, the Analytics Agent will not parse the record and will discard it.    It is not discarding all the requests because the MIDC field is not being collected for all events. Hence, the events that do not have this field value are parsed properly and published to the Events Service. For this reason, you will see less events data compared to actual load on the application.   Solution: You can approach this in two ways:   1. You can change the type of data being returned by the data collector at the application level to the type that is registered with Events Service.   2. You can change the field name in the MIDC that is defined for the current application so that it registers with the Events Service with its own type under a new field. ... View more

Problem: EUM browser data is not correlating to the backend application, even when the correlation cookie is coming as a part of the response from the application server.   Analysis: We see that the cookie information is present in the response and there is no httpOnly flag tagged with the cookie, but the same data is not being sent in the beacons. The guid we see in the beacon is self-generated and not the one sent by the application server. This is causing the correlation to break. We can see that there are resource files in the page like .png that also have the correlation cookies. These correlation cookies on resource files cause the actual page correlation to break.     Solution: The transactions serving the resource files should not be enabled for correlation. Identify the transactions that serves the resource file on the page and disable EUM correlation for those transactions. ... View more

Question  Why do I need to run the eum-schema update after importing the MySQL eum_db dump from the Controller MySQL to the EUM MySQL database?   Answer Starting 4.4, EUM has its own MySQL database in production mode deployment. If we are upgrading from a lower version to 4.4.x, we need to take a dump from the Controller MySQL for the  eum_db schema and import the dump in the EUM server's MySQL database. In 4.4.x, the schema of the EUM server has additional changes from its previous version, which are required for the EUM processor to function. When we are importing the schema level dump, it drops the current eum_db schema in the EUM database and then restores the older version of the schema from dump. To bring it back to a compatible state for the 4.4.x version, we need to update the schema using the eum-schema update  command. ... View more

Problem: EUM upgrade from a version lower than 4.4 fails in the "schema update" step with the below error: Unexpected error running Liquibase: Error executing SQL CREATE TABLE eum_db.crash_alert_events (hash_key VARCHAR(128) NOT NULL, range_key VARCHAR(128) NOT NULL, application_key VARCHAR(128) NOT NULL, alert_minute BIGINT(19) NOT NULL, group_id VARCHAR(128) NOT NULL, threshold INT(19) NOT NULL): Table 'crash_alert_events' already exists   Analysis: For versions prior to 4.4, EUM has its database schema with Controller MySQL. Starting 4.4, EUM has its own database for a production mode. Hence, for a production mode installation, it is required to export the schema from the Controller database to the EUM MySQL database.    The EUM schema in 4.4 has some additional tables. When we import the dump as per the documented process, it drops the eum_db schema in MySQL of EUM and then imports the dump from the previous version. The update-schema then creates the tables that were missing in the previous version of schema.   If we are not using the recommended command to take the export, the tables that are different from the dump are not dropped during the import. Therefore, when we run the eum-schema update  command, it fails with the above error. For example, we can take the database export using the below command as well: ./mysqldump -u root --password --protocol=TCP -P 3388 eum_db --add-drop-database > eum_db.sql   However, when we use this dump to import, it does not have drop syntax for the eum_db schema, which poses a problem during the update-schema command.    Solution: To avoid any anomalies during the upgrade process, we need to use the specified commands for taking the dump like below: ./mysqldump -u root --password --protocol=TCP -P [Controller-MySQL-Port] --databases eum_db --add-drop-database > eum_db.sql   This will ensure that the eum_db schema is cleared before importing the dump and the eum-schema update  will run without issues. ... View more

Problem: Values for EURT show considerable differences between the Pages and Ajax Request view and the Analyze view.    Analysis: By design, the EUM does not include outlier requests. However, the outlier requests are sent to the Events Service. If there are more outlier requests, the EURT in Analyze can get spiked.    Solution: Your outlier threshold is five times the stall threshold. If you do not want to see outliers in the Analyze view, you can add a filter to the displayed results limiting the value to filter out the outlier requests.   ... View more