cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Trying to startup Controller after changing the Cert and creating a new one

Beshoy.Shaher
Builder

After modifying the Controller's certificate and creating a new one, then after I tried to start the Controller again, but it did not start, nor did the event service.

ERROR [2024-01-06 14:15:56,850] com.appdynamics.orcha.extensions.es.health.StoreNodeHealth: Failed to connect to welcomeb/2001:0:2851:782c:1451:2eab:3b33:ff4d:9080 ERROR [2024-01-06 14:16:00,870] com.appdynamics.orcha.extensions.es.health.StoreNodeHealth: Connection failure while checking for number of data nodes on the host: welcomeB ERROR [2024-01-06 14:16:02,866] com.appdynamics.platformadmin.core.job.JobProcessor: Platform/Job [1/a6631ee1-e56a-4689-8d9a-4da5127b1e01]: Stage [es_cluster_health_stage] failed due to [Unable to check health of Events Service hosts [welcomeB] through port 9080.] INFO [2024-01-06 14:16:12,208] com.appdynamics.platformadmin.resources.VersionResource: Found Enterprise Console version 23.9.0-10017, 14:17:24,850] com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage: Failed to reach URL [http://welcomeB:9080/v1/store/report] ! java.net.ConnectException: Connection refused: connect ! at 14:17:38,858] com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage: Failed to reach URL [http://welcomeB:9080/v1/store/report] ! java.net.ConnectException: Connection refused: connect ! at 4da5127b1e01]: Stage [es_cluster_health_stage] failed due to [Unable to check health of Events Service hosts [welcomeB] through port 9080.] INFO [2024-01-06 14:18:12,207] com.appdynamics.platformadmin.resources.VersionResource: Found Enterprise Console version 23.9.0-10017, build INFO [2024-01-06 14:18:43,400] com.appdynamics.orcha.modules.modules.UriExec: Sending request to: http://localhost:8090/controller/rest/serverstatus INFO [2024-01-06 14:18:56,827] com.appdynamics.orcha.extensions.es.health.StoreNodeHealth: Connection failure while checking node health status on the host: welcomeB ERROR [2024-01-06 14:18:56,827] com.appdynamics.orcha.extensions.es.health.StoreNodeHealth: Failed to connect to welcomeb/2001:0:2851:782c:1451:2eab:3b33:ff4d:9080 ERROR [2024-01-06 14:18:56,839] com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage: Failed to reach URL [http://welcomeB:9080/v1/store/report] ! java.net.ConnectException: Connection refused: connect ! at

 

BeshoyShaher_0-1704580629061.pngBeshoyShaher_1-1704580669267.png

 

ES

BeshoyShaher_2-1704581003069.png

BeshoyShaher_3-1704581176835.png

 

3 REPLIES 3

Beshoy.Shaher
Builder


ERROR [2024-01-06 15:37:24,915] com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage: Failed to reach URL [http://welcomeB:9080/v1/store/report]
! java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
! at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
! at java.net.Socket.connect(Socket.java:613)
! at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:117)
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:283)
! ... 29 common frames omitted
! Causing: java.net.ConnectException: Failed to connect to welcomeb/2001:0:2851:782c:1451:2eab:3b33:ff4d:9080
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:285)
! at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:195)
! at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:249)
! at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:108)
! at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:76)
! at okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:245)
! at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:32)
! at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
! at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:82)
! at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
! at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
! at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
! at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
! at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
! at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:197)
! at okhttp3.internal.connection.RealCall.execute(RealCall.kt:148)
! at com.appdynamics.platformadmin.util.HealthCheckWrapper.healthCheckRequestOkHttp3(HealthCheckWrapper.java:99)
! ... 13 common frames omitted
! Suppressed: java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
! at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
! at java.net.Socket.connect(Socket.java:613)
! at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:117)
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:283)
! ... 29 common frames omitted
! Causing: java.net.ConnectException: Failed to connect to welcomeb/192.168.27.207:9080
! ... 30 common frames omitted
! Suppressed: java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
! at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
! at java.net.Socket.connect(Socket.java:613)
! at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:117)
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:283)
! ... 29 common frames omitted
! Causing: java.net.ConnectException: Failed to connect to welcomeb/fe80:0:0:0:280e:a326:367e:6dfb%5:9080
! ... 30 common frames omitted
! Suppressed: java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
! at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
! at java.net.Socket.connect(Socket.java:613)
! at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:117)
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:283)
! ... 29 common frames omitted
! Causing: java.net.ConnectException: Failed to connect to welcomeb/fe80:0:0:0:1451:2eab:3b33:ff4d%11:9080
! ... 30 common frames omitted
! Causing: java.lang.RuntimeException: Health check request failed to connect with exception
! at com.appdynamics.platformadmin.util.HealthCheckWrapper.healthCheckRequestOkHttp3(HealthCheckWrapper.java:102)
! at com.appdynamics.platformadmin.util.HealthCheckWrapper.healthCheckRequest(HealthCheckWrapper.java:71)
! at com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage.getClusterHealth(ESClusterHealthCheckStage.java:164)
! at com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage.getDetailedClusterHealth(ESClusterHealthCheckStage.java:102)
! at com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage.execute(ESClusterHealthCheckStage.java:76)
! at com.appdynamics.platformadmin.core.job.JobProcessor.run(JobProcessor.java:260)
! at com.appdynamics.platformadmin.core.job.JobProcessor.call(JobProcessor.java:114)
! at com.appdynamics.platformadmin.core.service.JobExecutionServiceImpl.lambda$registerRecurringJobHelper$0(JobExecutionServiceImpl.java:204)
! at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
! at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1596)
! at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
! at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
! at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
! at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
ERROR [2024-01-06 15:37:38,924] com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage: Failed to reach URL [http://welcomeB:9080/v1/store/report]
! java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
! at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
! at java.net.Socket.connect(Socket.java:613)
! at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:117)
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:283)
! ... 29 common frames omitted
! Causing: java.net.ConnectException: Failed to connect to welcomeb/2001:0:2851:782c:1451:2eab:3b33:ff4d:9080
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:285)
! at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:195)
! at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:249)
! at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:108)
! at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:76)
! at okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:245)
! at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:32)
! at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
! at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:82)
! at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
! at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
! at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
! at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
! at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
! at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:197)
! at okhttp3.internal.connection.RealCall.execute(RealCall.kt:148)
! at com.appdynamics.platformadmin.util.HealthCheckWrapper.healthCheckRequestOkHttp3(HealthCheckWrapper.java:99)
! ... 13 common frames omitted
! Suppressed: java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
! at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
! at java.net.Socket.connect(Socket.java:613)
! at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:117)
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:283)
! ... 29 common frames omitted
! Causing: java.net.ConnectException: Failed to connect to welcomeb/192.168.27.207:9080
! ... 30 common frames omitted
! Suppressed: java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
! at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
! at java.net.Socket.connect(Socket.java:613)
! at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:117)
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:283)
! ... 29 common frames omitted
! Causing: java.net.ConnectException: Failed to connect to welcomeb/fe80:0:0:0:280e:a326:367e:6dfb%5:9080
! ... 30 common frames omitted
! Suppressed: java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
! at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
! at java.net.Socket.connect(Socket.java:613)
! at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:117)
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:283)
! ... 29 common frames omitted
! Causing: java.net.ConnectException: Failed to connect to welcomeb/fe80:0:0:0:1451:2eab:3b33:ff4d%11:9080
! ... 30 common frames omitted
! Causing: java.lang.RuntimeException: Health check request failed to connect with exception
! at com.appdynamics.platformadmin.util.HealthCheckWrapper.healthCheckRequestOkHttp3(HealthCheckWrapper.java:102)
! at com.appdynamics.platformadmin.util.HealthCheckWrapper.healthCheckRequest(HealthCheckWrapper.java:71)
! at com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage.getClusterHealth(ESClusterHealthCheckStage.java:164)
! at com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage.getDetailedClusterHealth(ESClusterHealthCheckStage.java:102)
! at com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage.execute(ESClusterHealthCheckStage.java:76)
! at com.appdynamics.platformadmin.core.job.JobProcessor.run(JobProcessor.java:260)
! at com.appdynamics.platformadmin.core.job.JobProcessor.call(JobProcessor.java:114)
! at com.appdynamics.platformadmin.core.service.JobExecutionServiceImpl.lambda$registerRecurringJobHelper$0(JobExecutionServiceImpl.java:204)
! at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
! at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1596)
! at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
! at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
! at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
! at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
INFO [2024-01-06 15:37:45,028] com.appdynamics.orcha.core.OrchaRunnerImpl: Running playbook K:\Platform\platform-admin\conf\..\archives\platform-configuration\..\controller\23.9.0-10017\playbooks\check-single-controller-appserver-processes.orcha with variables file K:\Platform\platform-admin\conf\..\archives\platform-configuration\..\controller\23.9.0-10017\playbooks\controller.groovy
INFO [2024-01-06 15:37:45,032] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Uploading the diagnosis script if necessary
INFO [2024-01-06 15:37:45,063] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Uploading the diagnosis script if necessary] completed executing in [14] ms.
INFO [2024-01-06 15:37:45,074] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Setting execute permissions for the script
INFO [2024-01-06 15:37:45,091] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Setting execute permissions for the script] completed executing in [11] ms.
INFO [2024-01-06 15:37:45,101] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Running diagnosis scripts
INFO [2024-01-06 15:37:45,705] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Running diagnosis scripts] completed executing in [596] ms.
INFO [2024-01-06 15:37:45,719] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Stringify appserver diagnosis result
INFO [2024-01-06 15:37:45,736] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Stringify appserver diagnosis result] completed executing in [11] ms.
INFO [2024-01-06 15:37:47,208] com.appdynamics.orcha.core.OrchaRunnerImpl: Running playbook K:\Platform\platform-admin\conf\..\archives\platform-configuration\..\controller\23.9.0-10017\playbooks\check-single-controller-mysql-process.orcha with variables file K:\Platform\platform-admin\conf\..\archives\platform-configuration\..\controller\23.9.0-10017\playbooks\controller.groovy
INFO [2024-01-06 15:37:47,211] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Uploading the diagnosis script if necessary
INFO [2024-01-06 15:37:47,242] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Uploading the diagnosis script if necessary] completed executing in [13] ms.
INFO [2024-01-06 15:37:47,254] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Setting execute permissions for the script
INFO [2024-01-06 15:37:47,271] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Setting execute permissions for the script] completed executing in [11] ms.
INFO [2024-01-06 15:37:47,282] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Running diagnosis scripts
INFO [2024-01-06 15:37:48,170] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Running diagnosis scripts] completed executing in [879] ms.
INFO [2024-01-06 15:37:48,183] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Stringify MySQL diagnosis result
INFO [2024-01-06 15:37:48,202] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Stringify MySQL diagnosis result] completed executing in [12] ms.
INFO [2024-01-06 15:37:48,231] com.appdynamics.platformadmin.controller.job.stages.ControllerHealthCheckStage: Discovered Primary controller health status on host 'welcomeB': UNHEALTHY.
INFO [2024-01-06 15:37:48,231] com.appdynamics.platformadmin.core.job.JobProcessor: Platform/Job [1/e85fa4e1-fd2e-4ebc-aa0d-e3d77ff13258]: Ignored stage [controller_health_check_stage]'s error: [Health check result: Controller status check failed for the url http://welcomeB:8090/controller/rest/serverstatus with message: Health check request failed with HTTP 404: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>GlassFish v4 - Error report</title><style type="text/css"><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P{font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 404 - </h1><hr/><p><b>type</b> Status report</p><p><b>message</b></p><p><b>description</b>The requested resource is not available.</p><hr/><h3>GlassFish Server Open Source Edition 4.1.1 </h3></body></html>]
INFO [2024-01-06 15:37:48,245] com.appdynamics.platformadmin.controller.listener.ControllerRecurringJobListener: Controller listener expected health status [Critical] and received real time status [Critical] in platform 1
ERROR [2024-01-06 15:37:52,934] com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage: Failed to reach URL [http://welcomeB:9080/v1/store/report]
! java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
! at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
! at java.net.Socket.connect(Socket.java:613)
! at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:117)
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:283)
! ... 29 common frames omitted
! Causing: java.net.ConnectException: Failed to connect to welcomeb/2001:0:2851:782c:1451:2eab:3b33:ff4d:9080
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:285)
! at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:195)
! at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:249)
! at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:108)
! at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:76)
! at okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:245)
! at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:32)
! at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
! at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:82)
! at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
! at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
! at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
! at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
! at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:100)
! at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:197)
! at okhttp3.internal.connection.RealCall.execute(RealCall.kt:148)
! at com.appdynamics.platformadmin.util.HealthCheckWrapper.healthCheckRequestOkHttp3(HealthCheckWrapper.java:99)
! ... 13 common frames omitted
! Suppressed: java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
! at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
! at java.net.Socket.connect(Socket.java:613)
! at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:117)
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:283)
! ... 29 common frames omitted
! Causing: java.net.ConnectException: Failed to connect to welcomeb/192.168.27.207:9080
! ... 30 common frames omitted
! Suppressed: java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
! at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
! at java.net.Socket.connect(Socket.java:613)
! at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:117)
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:283)
! ... 29 common frames omitted
! Causing: java.net.ConnectException: Failed to connect to welcomeb/fe80:0:0:0:280e:a326:367e:6dfb%5:9080
! ... 30 common frames omitted
! Suppressed: java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
! at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
! at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
! at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
! at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
! at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
! at java.net.Socket.connect(Socket.java:613)
! at okhttp3.internal.platform.Platform.connectSocket(Platform.kt:117)
! at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:283)
! ... 29 common frames omitted
! Causing: java.net.ConnectException: Failed to connect to welcomeb/fe80:0:0:0:1451:2eab:3b33:ff4d%11:9080
! ... 30 common frames omitted
! Causing: java.lang.RuntimeException: Health check request failed to connect with exception
! at com.appdynamics.platformadmin.util.HealthCheckWrapper.healthCheckRequestOkHttp3(HealthCheckWrapper.java:102)
! at com.appdynamics.platformadmin.util.HealthCheckWrapper.healthCheckRequest(HealthCheckWrapper.java:71)
! at com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage.getClusterHealth(ESClusterHealthCheckStage.java:164)
! at com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage.getDetailedClusterHealth(ESClusterHealthCheckStage.java:102)
! at com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage.execute(ESClusterHealthCheckStage.java:76)
! at com.appdynamics.platformadmin.core.job.JobProcessor.run(JobProcessor.java:260)
! at com.appdynamics.platformadmin.core.job.JobProcessor.call(JobProcessor.java:114)
! at com.appdynamics.platformadmin.core.service.JobExecutionServiceImpl.lambda$registerRecurringJobHelper$0(JobExecutionServiceImpl.java:204)
! at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
! at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1596)
! at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
! at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
! at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
! at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
INFO [2024-01-06 15:37:56,850] com.appdynamics.orcha.extensions.es.health.StoreNodeHealth: Connection failure while checking node health status on the host: welcomeB
ERROR [2024-01-06 15:37:56,850] com.appdynamics.orcha.extensions.es.health.StoreNodeHealth: Failed to connect to welcomeb/2001:0:2851:782c:1451:2eab:3b33:ff4d:9080
ERROR [2024-01-06 15:38:00,860] com.appdynamics.orcha.extensions.es.health.StoreNodeHealth: Connection failure while checking for number of data nodes on the host: welcomeB
ERROR [2024-01-06 15:38:02,936] com.appdynamics.platformadmin.core.job.JobProcessor: Platform/Job [1/a6631ee1-e56a-4689-8d9a-4da5127b1e01]: Stage [es_cluster_health_stage] failed due to [Unable to check health of Events Service hosts [welcomeB] through port 9080.]
INFO [2024-01-06 15:38:15,225] com.appdynamics.platformadmin.resources.VersionResource: Found Enterprise Console version 23.9.0-10017, build
INFO [2024-01-06 15:38:45,045] com.appdynamics.orcha.core.OrchaRunnerImpl: Running playbook K:\Platform\platform-admin\conf\..\archives\platform-configuration\..\controller\23.9.0-10017\playbooks\check-single-controller-appserver-processes.orcha with variables file K:\Platform\platform-admin\conf\..\archives\platform-configuration\..\controller\23.9.0-10017\playbooks\controller.groovy
INFO [2024-01-06 15:38:45,049] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Uploading the diagnosis script if necessary
INFO [2024-01-06 15:38:45,080] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Uploading the diagnosis script if necessary] completed executing in [13] ms.
INFO [2024-01-06 15:38:45,093] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Setting execute permissions for the script
INFO [2024-01-06 15:38:45,109] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Setting execute permissions for the script] completed executing in [10] ms.
INFO [2024-01-06 15:38:45,120] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Running diagnosis scripts
INFO [2024-01-06 15:38:45,719] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Running diagnosis scripts] completed executing in [592] ms.
INFO [2024-01-06 15:38:45,733] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Stringify appserver diagnosis result
INFO [2024-01-06 15:38:45,751] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Stringify appserver diagnosis result] completed executing in [11] ms.
INFO [2024-01-06 15:38:47,335] com.appdynamics.orcha.core.OrchaRunnerImpl: Running playbook K:\Platform\platform-admin\conf\..\archives\platform-configuration\..\controller\23.9.0-10017\playbooks\check-single-controller-mysql-process.orcha with variables file K:\Platform\platform-admin\conf\..\archives\platform-configuration\..\controller\23.9.0-10017\playbooks\controller.groovy
INFO [2024-01-06 15:38:47,338] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Uploading the diagnosis script if necessary
INFO [2024-01-06 15:38:47,377] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Uploading the diagnosis script if necessary] completed executing in [20] ms.
INFO [2024-01-06 15:38:47,389] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Setting execute permissions for the script
INFO [2024-01-06 15:38:47,407] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Setting execute permissions for the script] completed executing in [11] ms.
INFO [2024-01-06 15:38:47,418] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Running diagnosis scripts
INFO [2024-01-06 15:38:48,303] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Running diagnosis scripts] completed executing in [878] ms.
INFO [2024-01-06 15:38:48,317] com.appdynamics.orcha.core.executor.DefaultPlaybookExecutor: Executing : Stringify MySQL diagnosis result
INFO [2024-01-06 15:38:48,335] com.appdynamics.orcha.core.executor.TaskExecutor: Task [Stringify MySQL diagnosis result] completed executing in [11] ms.
INFO [2024-01-06 15:38:48,364] com.appdynamics.platformadmin.controller.job.stages.ControllerHealthCheckStage: Discovered Primary controller health status on host 'welcomeB': UNHEALTHY.
INFO [2024-01-06 15:38:48,364] com.appdynamics.platformadmin.core.job.JobProcessor: Platform/Job [1/e85fa4e1-fd2e-4ebc-aa0d-e3d77ff13258]: Ignored stage [controller_health_check_stage]'s error: [Health check result: Controller status check failed for the url http://welcomeB:8090/controller/rest/serverstatus with message: Health check request failed with HTTP 404: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>GlassFish v4 - Error report</title><style type="text/css"><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P{font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 404 - </h1><hr/><p><b>type</b> Status report</p><p><b>message</b></p><p><b>description</b>The requested resource is not available.</p><hr/><h3>GlassFish Server Open Source Edition 4.1.1 </h3></body></html>]
INFO [2024-01-06 15:38:48,379] com.appdynamics.platformadmin.controller.listener.ControllerRecurringJobListener: Controller listener expected health status [Critical] and received real time status [Critical] in platform 1
INFO [2024-01-06 15:38:56,856] com.appdynamics.orcha.extensions.es.health.StoreNodeHealth: Connection failure while checking node health status on the host: welcomeB
ERROR [2024-01-06 15:38:56,856] com.appdynamics.orcha.extensions.es.health.StoreNodeHealth: Failed to connect to welcomeb/2001:0:2851:782c:1451:2eab:3b33:ff4d:9080
ERROR [2024-01-06 15:38:56,878] com.appdynamics.platformadmin.es.job.stages.ESClusterHealthCheckStage: Failed to reach URL [http://welcomeB:9080/v1/store/report]
! java.net.ConnectException: Connection refused: connect
! at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
! at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)

Beshoy.Shaher
Builder

 

@Ryan.Paredez  can you help me 

Ryan.Paredez
Community Manager

Hi @Beshoy.Shaher,

The Community is peer-to-peer. I do my best to help share relevant or helpful information. Since the Community has not jumped in to help out. I would recommend contacting AppD Support, or even your AppD Rep. 
If you do find a solution by any means, please come back and share what you learned as a reply to this post. Knowledge sharing is what makes the community valuable for all members. 


Thanks,

Ryan, Cisco AppDynamics Community Manager




Found something helpful? Click the Accept as Solution button to help others find answers faster.

Liked something? Click the Thumbs Up button.



Check out Observabiity in Action

new deep dive videos weekly in the Knowledge Base.