Problem:
Analytics data being collected is a lot less than the actual load on the application and you are seeing repetitive messages like the one below in the Analytics Agent log:
[2018-05-21T11:24:38,137+01:00] [WARN ] [pipeline-thread-12 (bt-publish-4)] [c.a.a.a.p.e.EventServicePublishStage] The message at index [0] failed due to the following permanent error: [Failed [failed to parse [segments.userData.xxxx]] with id [484xxx-63xx-4xxx-axxx-47d08xxxxx] index [customer1_xxxxxxx___biz_txn_v1___2018-05-19_16-38-20] type [customer1_xxxxx___biz_txn_v1]]
Analysis:
The same MIDC is defined for multiple applications. We observed that the field that is registered with the Events Service has a different type than the field that is being collected for the current application. The field type can be retrieved using the below query:
curl -XGET http://localhost:9200/<global_account_name>___biz_txn_v1*/_mapping?pretty=true
If same field name is being used across multiple MIDC, the Events Service will use the first registered field type. If the field type for a new application is different from the registered one, the Analytics Agent will not parse the record and will discard it.
It is not discarding all the requests because the MIDC field is not being collected for all events. Hence, the events that do not have this field value are parsed properly and published to the Events Service. For this reason, you will see less events data compared to actual load on the application.
Solution:
You can approach this in two ways:
1. You can change the type of data being returned by the data collector at the application level to the type that is registered with Events Service.
2. You can change the field name in the MIDC that is defined for the current application so that it registers with the Events Service with its own type under a new field.
