Baseline Deviation vs Baseline Percentage

[Sebastian.Perna]

We are trying to create a health rule self service mode for our developers and we have some doubts on how to explain the difference between using standard deviations and baseline percentage.

Which is the difference between the calculation of both of them? And which are the best scenarios to use one or another?

Regards!

[Mark.Byrne]

Hiya,

Standard deviations will work better if you have periods where the range of values is wider than other period - an example might be more varying response times during busy times, as opposed to quieter times.

Comparing against a Standard Deviation will reduce false alarms in those scenarios compared with a comparison against baseline percentages.

If the range of values is constant over time, always distributed over the same range, then a Baseline Percentage would work just as well.

To demonstrate this, I would recommend finding an example in your Metric Browser of a series where the baseline changes by a small amount from hour to hour, but the range varies widely - this should also have larger shaded Standard Deviation values. I've attached an example - we don't want the spike at 7:50am to generate an alarm as we know performance can be a little erratic first thing in the morning. It is well over the baseline though.

Mark

[Ujjwal.chhabra]

Hi Mark,

Can you please explain, what is the difference between standard deviation and baseline calculation such that it leads to this varying behavior? We need to set up a health rule using standard deviation.

Can you share what values for standard deviation to be used, and how can they be determined.

Regards,

Ujjawal. 

[Ryan.Paredez]

Hi @Ujjwal.chhabra,

You can also check out our AppD Docs page that talks about both of these: https://docs.appdynamics.com/21.10/en/application-monitoring/business-transactions/monitor-the-perfo...

Let me know if they are helpful.

[Ujjwal.chhabra]

Hi Ryan,

I have checked the pages which you have shared, I am not able to find the relevant information.

I am still not clear with these 2 questions:

1. Difference between standard deviation and baseline calculation such that it leads to this varying behavior?
2. Can you share what values for standard deviation to be used, and how can they be determined?

Regards, 

Ujjwal.

[Ryan.Paredez]

Let's see if @Mark.Byrne can jump in and help out.

[Mark.Byrne]

Hi @Ujjwal.chhabra 

In order to determine what values of standard deviations I would recommend looking at the metric in the Metric Browser as shown in the attached picture.

Turn on the Shade option (bottom right) and Configure the range (top left). This shows the calculated standard deviations above the baseline for the time period - note that this can change every hour. Review the data across a wider timeframe - we often different performance early in the morning as systems are warming up.

With the example attached I would pick 2 standard deviations as a warning, 3 standard deviations as critical. I would also define the health rule so that it looks for multiple occurrences in a time period e.g. 5 in 30 minutes, so as to avoid a single peak generating an alert.

The majority of our Health Rules are defined with 2sd for warning, 3sd for critical, however we do have a set that are set at 3sd and 4sd respectively as they were triggering too frequently

Regards

Mark