Discussion Feed

Table of Contents What are derived metrics? Points to remember Configuring “derivedMetrics" section in config.yml   What are derived metrics? Derived metrics are the metrics created by using the existing metrics (let’s call them base metrics). For example, we have two metrics, namely, cache hits and cache misses.  We can calculate the derived metric "cache hit ratio" using the following formula:  cache hits / (cache hits + cache misses)   Rolled-up metrics at any level are also derived metrics since they can be obtained by aggregating all the base level metrics.   Points to remember These are points to remember while configuring derived metrics and rolled-up metrics through config.yml in AppDynamics extensions:   1. Specifying base metric path pattern First and foremost, the derived metrics calculator uses the base metric paths and metric values. Since these paths are dynamically created in AppDynamics, it is required to have insight into these metric paths. While configuring the derived metrics in the config.yml , the base metrics have to be specified with their metric path pattern.   For example, let’s take the following metric hierarchy: Server1 Queue Queue1 Cache hits : 10 Cache misses : 20 Cache ratio Queue2 Cache hits : 15 Cache misses : 30 Cache ratio Server2 Queue Queue1 Cache hits : 10 Cache misses : 20 Cache ratio Queue2 Cache hits : 15 Cache misses : 30 Cache ratio   Let’s consider the base metric Cache hits. In the given metric hierarchy, we have 4 Cache hits. Server1|Queue|Queue1|Cache hits Server1|Queue|Queue2|Cache hits Server2|Queue|Queue1|Cache hits Server2|Queue|Queue2|Cache hits   If we analyze the 4 paths, we can see that all of the paths have a common pattern with static and dynamic parts. Using the above metric paths, let’s build a common pattern.   The static parts in the above paths are Queue and Cache hits. The dynamic parts are Server1 (or) Server2 and Queue1 (or) Queue2. While building the common pattern for the base metrics, the static part remains the same in the base metric pattern whereas the dynamic parts have to be specified by place holders like {x}. Since we have two dynamic parts in this example, two place holders have to be used. Let’s use {x} for Server1 (or) Server2 and {y} for Queue1 (or) Queue2. The final common base metric path pattern for Cache hits is:  {x}|Queue|{y}|Cache hits   2. Using multiple base metric path patterns The important thing to remember when multiple base metrics need to use the same dynamic value is that the placeholder variable should be the same.  For example, let’s consider the same hierarchy specified above. Suppose we want to calculate the Cache ratio for each queue in every server. We have the following base metric pairs where each pair represents the base metrics required for calculating the Cache ratio of a specific queue in a specific server.    Server1|Queue|Queue1|Cache hits  ,  Server1|Queue|Queue1|Cache misses Server1|Queue|Queue2|Cache hits  ,  Server1|Queue|Queue2|Cache misses   Server2|Queue|Queue1|Cache hits  ,  Server2|Queue|Queue1|Cache misses Server2|Queue|Queue2|Cache hits  ,  Server2|Queue|Queue2|Cache misses In each pair, we can observe that the dynamic parts have the same value so the metric path patterns of both the base metrics should have same place holder as follows: {x}|Queue|{y}|Cache hits {x}|Queue|{y}|Cache misses   This means that while calculating the derived metric, the same values will be used for the dynamic parts in both of the base metrics as shown in the base metric pairs above. If the same place holders are not used, there will be a mix up of values.   3. Specifying derived metric path pattern The base metrics have a metric path in the metric hierarchy. In the same way, the derived metrics should also have a metric path that infers where in the metric tree the derived metrics fit.   For example, in the example tree stated earlier, we want to calculate the Cache ratio for each queue in every server as Cache hits / (Cache hits + Cache misses). The metric path of the derived metric Cache ratio will be {x}|Queue|{y}|Cache ratio and the formula to calculate it will be {x}|Queue|{y}|Cache hits / ( {x}|Queue|{y}|Cache hits + {x}|Queue|{y}|Cache misses ) . This means that for every queue in all the servers, the corresponding Cache hits and Cache misses are used to calculate Cache ratio.   If the place holder {y} is missed from the derived metric path (i.e., {x}|Queue|Cache ratio), then the metric tree will look like this: Server1 Queue Queue1 Cache hits : 10 Cache misses : 20 Queue2 Cache hits : 15 Cache misses : 30 Cache ratio Server2 Queue Queue1 Cache hits : 10 Cache misses : 20 Queue2 Cache hits : 15 Cache misses : 30 Cache ratio   This metric tree implies that the Cache ratio is calculated by using the Cache ratios of both queues in a server. Although it does not make sense in this case, this technique can be used to get rolled-up metrics at any level. For example, consider the following:   Server1 Queue Queue1 RAM ops : 0 Hdd ops : 0 Total ops:0 Queue2 RAM ops : 15 Hdd ops: 4 Total ops:19 Total ops: (0 + 19 = 19) Server2 Queue Queue1 RAM ops : 9 Hdd ops: 3 Total ops:12 Queue2 RAM ops : 2 Hdd ops: 6 Total ops:8 Total ops: (12 + 8 = 20)    In this tree we have 2 base metrics: RAM ops and Hdd ops (4 occurrences each) Server1|Queue|Queue1|RAM ops , Server1|Queue|Queue1|hdd ops Server1|Queue|Queue2|RAM ops , Server1|Queue|Queue2|hdd ops Server2|Queue|Queue1|RAM ops , Server2|Queue|Queue1|hdd ops Server2|Queue|Queue2|RAM ops , Server2|Queue|Queue2|hdd ops   The base metric patterns can be {x}|Queue|{y}|RAM ops and {x}|Queue|{y}|Hdd ops. If we want to calculate the Total ops at queue level, then we can use the following formula: {x}|Queue|{y}|Total ops =  {x}|Queue|{y}|RAM ops + {x}|Queue|{y}|Hdd ops   But, if we want Total ops at the server level, we can derive it with the Server Total ops derived metric, with the metric path {x}|Queue|Server Total ops and the formula {x}|Queue|{y}|RAM ops + {x}|Queue|{y}|hdd ops.   Since we omitted one dynamic path level({y}) from derived metric path, we can achieve aggregation at its previous dynamic path level({x}). Basically what will happen at this level is that Total ops will be calculated at queue level but the derived metric path is required at server level, so all the queue level Total ops for a specific server will be aggregated at server level. Likewise, if we omit the next dynamic path level({x}) from the derived metric path, we can get Total ops at the cluster level. The aggregation type can be sum, average or last observed value based on the aggregation qualifier of the metric.     Configuring “derivedMetrics" section in config.yml  To initiate derived metrics in an AppDynamics extension, the config.yml should contain a separate section called derivedMetrics and it should have the list of derived metrics that need to be calculated. A sample of the “derivedMetrics” section is listed below:   derivedMetrics: - derivedMetricPath: "{x}|Queue|{y}|Cache ratio" formula: “{x}|Queue|{y}|Cache hits / ({x}|Queue|{y}|Cache hits + {x}|Queue|{y}|Cache misses)” - derivedMetricPath: “Cluster|Total ops" formula: “{x}|Total ops” aggregationType: “SUM" timeRollUpType: “SUM" clusterRollUpType: “COLLECTIVE” - derivedMetricPath: “{x}|Queue|Server Total ops” formula: “{x}|Queue|{y}|RAM ops + {x}|Queue|{y}|hdd ops"   If the expression has only one operand like the second metric in the   sample listed above, it means that the derived metric is actually a rolled-up metric calculated by aggregating base metric values. If the expression has multiple operands, but the derivedMetricPath has fewer dynamic path levels like the third metric in the example above, it also means that the derived metric is a rolled-up metric. In the list of derived and rolled-up metrics, each derived metric should have a derivedMetricPath (th path of the derived metric in the metric hierarchy) and a formula (the expression used for calculating the derived metric). Both these fields are required. In case of rolled-up metrics (see second metric in the example), aggregation type depends on the aggregation qualifier ( aggregationType ) specified. If the aggregationType field is not present (see third metric in the sample) by default, the values are averaged. Other optional fields are timeRollUpType  (default value is AVERAGE), clusterRollUpType  (default value is INDIVIDUAL), alias  (gives an alias name to the derived metric for use in the metric hierarchy), multiplier, delta and convert .   Note : If the base metrics have aliases that are used in the metric tree, the alias names have to be used in the derived metric formula to represent the base metrics . Using  the third metric of the above sample as an example, if the base metric RAM ops has an alias RAM operations, then the formula should be {x}|Queue|{y}|RAM operations + {x}|Queue|{y}|hdd ops ” instead of “ {x}|Queue|{y}|RAM ops + {x}|Queue|{y}|hdd ops .   Please find the assumptions and limitations of Derived Metrics Calculator and Cluster Metrics Processor here.   ... View more

How do I configure Monitoring Extensions to report metrics to the Controller?   Table of Contents Overview Server Infrastructure and Monitoring (SIM) Machine Agent   Overview AppDynamics Monitoring Extensions require either a Server Infrastructure and Monitoring (SIM) Agent or a stand-alone Machine Agent (MA) to report metrics to the AppDynamics Controller. For more details on SIM, check Server Visibility . For more details on MA, check Machine Agent .   IMPORTANT:   AppDynamics Extensions are supported only if there is a single instance of either a Machine Agent with APM Agents or a SIM Agent on a physical host. Multiple instances of Machine Agent or SIM on a single physical host is not supported for extensions installation.   Server Infrastructure & Monitoring (SIM)      In this mode, it is imperative to purchase the Server Visibility license and provide the SIM-enabled flag to the Agent. It can be provided as a JVM argument to the Machine Agent, or can be specified in the controller-info.xml . For more details, refer to Enable Server Visibility .   In this case, the Agent’s metrics will be reported in the Metric Browser under the "Servers" tab in the controller as shown below : Machine Agent In this mode, there are two installation scenarios   When using the APM Machine Agent installation scenario , the application name, tier name, and node name should not be provided to the MA as it will piggyback on the APM Agents running on the same server. In this case, the Agent’s metrics will be reported under the Metric Browser of all the APM apps. In the Independent Machine Agent installation scenario , a Server Visibility License (SIM License) is mandatory. Please refer to SIM section above. Once the Agent setup is complete, the Agent should be started and the hardware metrics reported by the Agent should be verified in the Controller. IMPORTANT: AppDynamics Extensions should be installed on the Agent only after verifying that the Agent’s hardware metrics are being reported consistently in the Controller.      ... View more

Assumptions There are the assumptions made before the derived metrics calculation is initiated: All the base metrics (in all different levels) are available (i.e, the metrics have been reported) for derived metrics calculation. For a derived metric to be calculated, all of its base metrics should have reported a valid Number.   Limitations The limitation when using the derived metrics calculator is:  A derived metric cannot be reused in another derived metric   If a derived metric has to be calculated at different levels, each metric has to be specified separately in the list of metrics under the derivedMetrics  section. For example, in the following metric tree:                      Server1               Queue                   Queue1                         RAM ops : 9                         Hdd ops: 3                          Total ops:12                   Queue2                         Total ops : 2                         Hdd ops: 6                          Total ops:8                     Total ops: (12 +  8 = 20)              If the "Total ops" metric is required both at the queue level and server level, then the derivedMetrics  section looks as follows:   - derivedMetricPath: “{x}|Queue|{y}|Total ops" //queue level Total ops formula: “{x}|Queue|{y}|RAM ops + {x}|Queue|{y}|Total ops” - derivedMetricPath: “{x}|Total ops" //server level Total ops formula: “{x}|Queue|{y}|RAM ops + {x}|Queue|{y}|Total ops”   The queue level “Total ops” derived metric cannot be used for calculating the server level or any other level derived metric. ... View more