How do I troubleshoot a Machine Agent on Windows showing intermittent Controller gaps?

[Vijay.Venkatarajulu]

A Machine Agent running on Windows OS shows gaps on the Controller due to collector script timeout 

This article describes an issue where the Machine Agent on Windows OS shows gaps on the Controller, presents the likely cause, and provides steps for troubleshooting as well as two test scripts. The scripts can be found below the text portion of this article.

In this article... 

• Machine Agent functionality
  Machine Agent on Windows OS
• Observed gaps in the Machine Agent  
• Troubleshooting steps 
  Observations 

Machine Agent functionality 

By default, the Machine Agent invokes collector scripts to gather hardware stats once every 30 seconds (this limit can be increased to a maximum of 60 seconds). At the end of every minute, the Machine Agent aggregates the data received and publishes it to the Controller. Hence, the Controller expects data every minute, and if not received, it will show a gap for that respective minute.  

Machine Agent on Windows OS 

The Machine Agent uses open source OSHI libraries and collector scripts to gather hardware statistics. CPU, Memory and Process statistics are collected via OSHI libraries, and the rest of the metrics are gathered via collector script. The collector script Machine Agent executes on Windows OS is sim-collector.vbs (added to this article as an attachment). The collector scripts get the data directly via Windows WMI calls and OSHI, as well relying internally on similar Windows WMI java APIs to get the data. This Machine Agent functionality is also applicable for Windows OS. 

 Back to top

Observed gaps in the Machine Agent 

The administrator observes intermittent gaps from a Machine Agent installed on Windows OS. This can happen across single or multiple servers.  

Gaps appear as follows: 

 Back to top

Troubleshooting steps we performed 

We created collect-sample.vbs to mimic the functionality of the Machine Agent.  

NOTE | You can find the collect-sample.vbs and sim-collector.vbs scripts below the text of this article, just above and to the left of the comment buttons.
 

1. The collect-sample.vbs script ensures that sim-collector.vbs is invoked once every 30 seconds. Before invoking the sim-collector.vbs script, collect-sample.vbs writes a time stamp in the log file as shown below: 
   

   objFile.WriteLine("Script Start Time") 
   objFile.WriteLine(Now())

2. Immediately after writing the above time stamp, collect-sample.vbs file invokes the sim-collector.vbs script. 
   

   Set WshShellExec = objShell.Exec("cscript sim-collector.vbs")  

3. The first module invoked as part of the sim-collector.vbs script is "runCollector". 
   

   Function runCollector() 

   
   
4. The very first operation performed as part of the "runCollector" module is writing the time stamp onto the same log file again. 
   

   Wscript.Echo "runCollector start", Now()  

    
   The sample output of the collector script is attached here in the ticket. Here are the first few lines from the collector script sample output:
   

Explanation of the collector script’s sample output :

SCRIPT	LINES CAPTURED
collect-sample.vbs	Script Start Time  2023-05-17 12:34:37 PM
sim-collector.vbs	Microsoft (R) Windows Script Host Version 5.8  Copyright (C) Microsoft Corporation. All rights reserved.    runCollector start 2023-05-17 12:34:38 PM  {

The expected difference in the two timestamps described above should be a few seconds—ideally less than 5 - 10 seconds— since it is just a timestamp print operation happening on both sides, and no other operations that would consume more time are taking place between them. 

Observations after performing troubleshooting

• In general, the scripts run fine with an expected delay of less than 5 seconds. During this time, metrics are also published to the Controller as expected. 
   
• During other times, the delay between both the time stamps explained above goes beyond 2-3 minutes. The Controller also shows gaps during the same period of time as the delay. 
   
  Here is a sample output snippet from this scenario: 
  

  Script Start Time 
  2023-05-17 12:39:01 PM 
  Microsoft (R) Windows Script Host Version 5.8 
  Copyright (C) Microsoft Corporation. All rights reserved. 
   
  runCollector start 2023-05-17 12:43:31 PM 

  The gap in the snippet above shows that the script is getting stuck during this time, and that something on the OS is blocking the sim-collector.vbs script invoked by collect-sample.vbs.  
  

• A number of things could block this call from the OS. The specifics must be determined by a Windows administrator. In our experience with similar issues in the past, customers’ Windows administrators were able to identify the OS tasks blocking the vbs scripts, and to fix the issue. 

 Back to top