cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Erin
AppDynamics Team (Retired)

Question:

Our logs are showing the following error message which says that the thread pool task queue is full. We restarted the Controller and the problem went away. Is this normal, and should we increase our thread pool as a result?

 

[#|2017-01-07T04:52:34.017-0500|SEVERE|glassfish3.1.2|grizzly|_ThreadID=34;_ThreadName=Thread-5;|doSelect exception 
java.util.concurrent.RejectedExecutionException: The thread pool's task queue is full, limit: 4096
at com.sun.grizzly.util.AbstractThreadPool.onTaskQueueOverflow(AbstractThreadPool.java:493)
at com.sun.grizzly.util.SyncThreadPool.execute(SyncThreadPool.java:191)
at com.sun.grizzly.util.GrizzlyExecutorService.execute(GrizzlyExecutorService.java:162)
at com.sun.grizzly.http.StatsThreadPool.execute(StatsThreadPool.java:131)
at com.sun.grizzly.NIOContext.execute(NIOContext.java:510)
at com.sun.grizzly.NIOContext.execute(NIOContext.java:488)
at com.sun.grizzly.SelectorHandlerRunner.handleSelectedKey(SelectorHandlerRunner.java:374)
at com.sun.grizzly.SelectorHandlerRunner.handleSelectedKeys(SelectorHandlerRunner.java:265)

 

 

Answer: 

 

As of version 4.2.x, for large profiles, our team recommends 24 CPU Cores. This machine only had 8 Cores.

To resolve this, first backup the existing domain.xml configuration file.

 

Then make the following changes to the domain.xml file:

 

From:
<thread-pool max-thread-pool-size="32" name="http-thread-pool" min-thread-pool-size="16"></thread-pool>

Change to:
<thread-pool max-thread-pool-size="64" name="http-thread-pool" min-thread-pool-size="64" max-queue-size="-1"></thread-pool>

 

And restart the App Server:

  1. cd <ControllerDir>/bin
  2. ./controller.sh stop-appserver
  3. ./controller.sh start-appserver   [This will take some time, depending on the size]

 

Comments
Rahul.Jangid
Builder

HI Team,

 

Made the name="http-thread-pool" to 16 from 8 but after trying to stop the controller server, it fails. 

 

I am attaching the sbapshots of domain.xml with the change and then trying to stop the controller server. Can you please suggest what can be the issue?

 

domain.xml post changedomain.xml post changeFailed to stop controllerFailed to stop controller

Best Regards,

Saurabh

 

Mark.Early
Creator

Since we have 16 processes on this box, I've also increased the thread pool sizes from 8 to 16.  these are the changes: 

 

Original: 

<thread-pool max-thread-pool-size="8" min-thread-pool-size="8" name="http-thread-pool"/>

 

New: 
<thread-pool max-thread-pool-size="16" min-thread-pool-size="16" name="http-thread-pool" max-queue-size="-1"/>

 

 

Testing now. 

Claudia.Landivar
AppDynamics Team (Retired)

Hi, @Mark.Early 

How did the thread pool size increase work out for you?

 

Claudia Landivar

Community Manager & Editor

Version history
Last update:
‎06-15-2017 09:38 AM
Updated by:
On-Demand Webinar
Discover new Splunk integrations and AI innovations for Cisco AppDynamics.


Register Now!

Observe and Explore
Dive into our Community Blog for the Latest Insights and Updates!


Read the blog here
Contributors