How IIS app pools represent App Agent Status on the controller

[Alex.Riley]

Contents: 

• What is the App Agent vs Coordinator?
  • App Agent status vs Machine Agent status
• Why is my app agent status 0% on IIS applications?
• What are the options for having 100% app agent status?
• What if I cannot modify IIS settings?

What is the App Agent vs Coordinator?

The AppDynamics.Agent.Coordinator is the orchestration on when to inject the app agent's DLLs into an application as well as collecting machine metrics (CPU, Memory, Performance Counters, etc). The Coordinator does not monitor any application on the server has this is the responsibility of the app agent.

In an environment where the profiler environment variables are defined, any .NET runtime at startup will check if the application should be profiled and what profiler to inject. As part of the installation process of the MSI package, it will create the necessary profiler environment variables. 

https://learn.microsoft.com/en-us/dotnet/framework/unmanaged-api/profiling/setting-up-a-profiling-en...

Profiler environment variables:

• COR_PROFILER
  • Full framework profiler to be injected into the application
• COR_ENABLE_PROFILING
  • Boolean value on whether or not full framework profiling is enabled
• COR_PROFILER_PATH
  • Path to where the full framework profiler resides
• CORECLR_PROFILER
  
  • .NET Core profiler to be injected into the application
• CORECLR_ENABLE_PROFILING
  • Boolean value on whether or not .NET Core profiling is enabled
• CORECLR_PROFILER_PATH
  • Path to where the .NET Core profiler resides

If the .NET application is a full framework, it will write a message to the Event Viewer's Application logs. Sample of a successful instrumentation:

.NET Runtime version 4.0.30319.0 - The profiler was loaded successfully.  Profiler CLSID: 'AppDynamics.AgentProfiler'.  Process ID (decimal): 110060.  Message ID: [0x2507].

When the application does not match an application to be monitored in the config.xml of the Coordinator it will not inject the agent DLLs:

.NET Runtime version 4.0.30319.0 - The profiler has requested that the CLR instance not load the profiler into this process.  Profiler CLSID: 'AppDynamics.AgentProfiler'.  Process ID (decimal): 111500.  Message ID: [0x2516].

Both messages are at the level of Information. Neither message is a cause for alarm and is only informational.

App Agent status vs Machine Agent status

The AppDynamics.Agent.Coordinator reports to the controller and one of the metrics it reports is [Availability]. This metric represents the Machine Agent status on the Controller's Tiers & Nodes page. 

The App Agent status is the app agent that is injected into your application. If your application is not running then neither is the app agent. This leads us to the next point regarding IIS applications.

Why is my app agent status 0% on IIS applications?  

The app agent is injected into your application and shares the application's lifecycle. For IIS, this means the app agent's DLLs are injected into the w3wp process on .NET startup. This can only happen at the startup of the process. 

However, app pools are managed by IIS, and the default settings do the following:

• App pools are not started by default. Traffic must be sent to the application first
• App pools that have not received any traffic for 20 minutes will be terminated

As mentioned earlier, the app agent shares the application's lifecycle, so you can see how these default settings might affect the app agent status that is displayed on the controller. 

Two possible scenarios with the default IIS settings can cause the app agent status to show 0%. 

• App pool was killed by IIS because there was no activity on the application. On the controller, you will see a downward trend in the app agent status during periods of idle activity. 
• The server was restarted and no traffic is currently being sent to the application. Therefore, no w3wp process has been started so the controller shows a 0% on app agent status. 

What are the options for having 100% app agent status?

Three settings must be changed to ensure that the app pool is running and remains running regardless of traffic or server restart. 

1. Idle timeout https://learn.microsoft.com/en-us/previous-versions/iis/6.0-sdk/ms525537(v=vs.90)
2. Start Mode https://learn.microsoft.com/en-us/iis/configuration/system.applicationhost/applicationpools/applicat...
3. IIS Application Initialization (requires IIS 8.0) https://learn.microsoft.com/en-us/iis/get-started/whats-new-in-iis-8/iis-80-application-initializati...

The Idle Timeout property is responsible for terminating an app pool that has not received traffic after some time (default is 20 minutes). Setting this property to 0 will prevent IIS from terminating the app pool regardless of how long the app pool is idle. 

Start Mode set to AlwaysRunning instead of the default value of OnDemand. 

IIS Application Initialization requires IIS 8.0. When the server starts, IIS will invoke a fake request to the specified page to start the app pool. Follow the instructions listed in the link above for the detailed steps.

What if I cannot modify IIS settings?

You can modify the config.xml to monitor the performance counter "Current Application Pool State" which is part of the APP_POOL_WAS category for your particular app pool and create a health rule to trigger in the event that the app pool is in a stopped state.

"Current Application Pool State" possible values:

• Starting
• Started
• Stopping
• Stopped
• Unknown

However, you need to be aware of the following:

• An app pool can be assigned to multiple sites and applications. There is no way to get a granular scope to a single application unless each IIS application/site uses a unique app pool
• There are really only three states for the "Current Application Pool State" - Started, Stopped, and Unknown. The in-between states are too quick to capture and report on. 
• The difference between an app pool and worker process. Having an app pool in a started state does not mean your application and, by extension, the agent is running. 
  • In addition, an app pool in the started state does not mean your application is able to start. For example, .NET runtime errors at startup can prevent the application from starting even though the app pool is started. 

I strongly recommend modifying the IIS settings to get a true app agent status and then rely on the "Current Application Pool State" performance counter but this option is available if your circumstances prevent modification of the IIS settings and the limitations above are not a concern. 

With the caveats out of the way, let's discuss how to make this change. 

Config.xml:

<machine-agent>
    <perf-counters>
      <perf-counter cat="APP_POOL_WAS" name="Current Application Pool State" instance="MY_APP_POOL_NAME" />
    </perf-counters>
</machine-agent>

Then create a new health to trigger if the app pool state is not in a Started state.