I understand that Agents add 2% overhead to an application server (CPU, response time, throughput, network IO, etc.), and will consume about 10mb-100mb of the internal memory of the application server. How do you measure the memory consumption and make sure the Agent consumes 10mb-100mb?
Java Agents have various safety checks built-in internally, such as max number of business transaction (50), max number of snapshots per min (20), metrics reported (5000) and many others, which keep the Agents' overhead under the limit.
Additionally, there are other factors which can contribute to overhead, including:
Nature of the business application that you are monitoring (technology stack).
The number of business transactions being monitored.
The number of snapshots taken by an Agent per minute.
The number of requests/load handled by the application server.
Async nature of the application.
And much more.
So the overhead factor can fluctuate depending on these factors.
To understand the heap usage of an Agent, find the memory consumption with and without the Agent, and the difference will give an accurate estimate of heap consumption of agent. The right tool will depend on the JDK version and operating system in use.