Events Service Unhealthy

[Salah.Tarek Khattab]

Hi,

I just started to implement a Cluster of 3 nodes of events service with a load balancer,
every node is started to be healthy and then unhealthy for two properties, and then go down

[Elasticsearch] unhealthy...retrying it the same error in every node and then go down

 I tried to build this cluster on Windows Environment

I tried to make a single node on the test environment also on windows, and it's running successfully.

[Ryan.Paredez]

Hi @Salah.Tarek Khattab,

What is the actual error you are getting? Can you share it as a reply?

[Salah.Tarek Khattab]

Hi Ryan, 

The Error that I have is 

"events-service-api-store / Connection to ElasticSearch: clusterName=[appdynamics-events-service-cluster]" : {
"healthy" : false,
"message" : "None of the configured nodes are available: [{#transport#-1}{10.13.2.33}{10.13.2.33:9300}, {#transport#-2}{10.13.2.36}{10.13.2.36:9300}, {#transport#-3}{10.13.2.37}{10.13.2.37:9300}]",
"error" : {
"message" : "None of the configured nodes are available: [{#transport#-1}{10.13.2.33}{10.13.2.33:9300}, {#transport#-2}{10.13.2.36}{10.13.2.36:9300}, {#transport#-3}{10.13.2.37}{10.13.2.37:9300}]",
"stack" : [ "org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:290)", "org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:207)", "org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:55)", "org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:288)", "org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359)", "org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:348)", "org.elasticsearch.client.support.AbstractClient$ClusterAdmin.execute(AbstractClient.java:848)", "org.elasticsearch.client.support.AbstractClient$ClusterAdmin.nodesStats(AbstractClient.java:943)", "com.appdynamics.analytics.processor.elasticsearch.node.admin.ElasticSearchAdminFacade.getNodesStatsResponse(ElasticSearchAdminFacade.java:410)", "com.appdynamics.analytics.processor.elasticsearch.node.provider.ElasticSearchClientFacade.getNodesStatsResponse(ElasticSearchClientFacade.java:368)", "com.appdynamics.analytics.processor.elasticsearch.clusters.ClusterHealthStateReporter.isPartOfCluster(ClusterHealthStateReporter.java:120)", "com.appdynamics.analytics.processor.elasticsearch.clusters.ClusterHealthStateReporter.updateHistogram(ClusterHealthStateReporter.java:110)", "com.appdynamics.analytics.processor.elasticsearch.clusters.ClusterHealthStateReporter.updateHealthCheckResult(ClusterHealthStateReporter.java:99)", "com.appdynamics.common.util.health.AsynchronousHealthCheckable.runUpdateHealthCheckResult(AsynchronousHealthCheckable.java:122)", "com.appdynamics.common.util.health.AsynchronousHealthCheckable.access$100(AsynchronousHealthCheckable.java:46)", "com.appdynamics.common.util.health.AsynchronousHealthCheckable$2.run(AsynchronousHealthCheckable.java:146)", "java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)", "java.util.concurrent.FutureTask.runAndReset(Unknown Source)", "java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)", "java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)", "java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)", "java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)", "java.lang.Thread.run(Unknown Source)" ]
} 

[Ryan.Paredez]

Hi @Salah.Tarek Khattab,

I found this existing post where a member shared a similar error message. Can you check it out and let me know if it helps.

https://community.appdynamics.com/t5/Java-Java-Agent-Installation-JVM/Event-Service-health-keeps-tur...

[Salah.Tarek Khattab]

there is no solution in this link,

it's only a question 

[Ryan.Paredez]

Hi @Salah.Tarek Khattab,

"we should scale up the events service as long as we run it as a single node especially in the dev & test environment however it's recommended to use a cluster of multi-nodes in a production environment 

https://docs.appdynamics.com/display/PRO21/Events+Service+Deployment"