Why doesn't AppD have time settings for health rules?
Examples:
"trigger alert only if threshold is breached every polling minute for X minutes"
"trigger alert if threshold is breached X number of times in the last X minutes"
For business transactions, I'm using the standard baseline settings for average response time health rules, but my business transactions are filling up my inbox with violations that are not really useful. I can't turn these rules off or I'll miss any serious issues that show up ( a continuously sick node or worse ).
I need health rules to only trigger if average response time, for a specific node, crosses a threshold AND also stays above a threshold for x minutes/seconds. There is no need to alert out for a single transaction that succeeds but takes a long time, when 99.9% are within limits. I can't find a simple way to filter this noise out without watering down detection accross the larger pool of nodes.
The only way I can see to do this is to play around with the baseline, which then raises the ART and reduces effectiveness for large scale problems. Please enlighten me if I'm missing something.
This time based option is standard on most other APMs I've used. I see you can build custom rules, but there seems to be little documentation on how to do this (no examples that I can find).
TIA.
... View more
