cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Not a customer? Start a free trial

Click the Start a free trial link to start a 15-day SaaS trial of our product and join our community as a trial user. If you are an existing customer do not start a free trial.

AppDynamics customers and established members should click the sign in button to authenticate.

Business iQ (Analytics)

Starting Events Service cluster

SOLVED

Starting Events Service cluster

I'm trying to start the Events service nodes but it doesn't work

Learn More

[root@console platform-admin]# bin/platform-admin.sh submit-job --platform-name AppDPlatform --service events-service --
job start

( 1/ 7) Load Events Service cluster configuration: SUCCESS
( 2/ 7) Load all ES cluster hosts into execution context.: SUCCESS

( 3/ 7) Start Events Service Cluster: Starting the Events Service api store node
( 3/ 7) Start Events Service Cluster: FAILED
Job failed to execute.
Failure occurred: Start Events Service Cluster
Error message:
Task failed: Starting the Events Service api store node
on host: 10.0.30.45
as user: root
with message: Connection to [http://10.0.30.45:9080/_ping] failed due to [Failed to connect to /10.0.30.45:9080].

 

I also tried to start each node alone I get this error

Learn More

pwd
/opt/appdynamics/platform/product/events-service/processor
[root@events-service-03 processor]# echo $JAVA_HOME
/usr/lib/jvm/jre-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/
[root@events-service-03 processor]# ./bin/events-service.sh start -p ./conf/events-service-api-store.properties &
[1] 43742
[root@events-service-03 processor]# ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging.
[2021-06-01T10:52:52,627+02:00] [APPLICATION_HOME] has been overridden to [/opt/appdynamics/platform/product/events-service/processor/bin/..]
[2021-06-01T10:52:52,631+02:00] [APPLICATION_HOME] has been overridden to [/opt/appdynamics/platform/product/events-service/processor/bin/..]
[2021-06-01T10:52:52,636+02:00] Program arguments:

|-Xmx2990m|-Xms2990m|-Djava.net.preferIPv4Stack=true|-Dfile.encoding=UTF-8|-Djute.maxbuffer=30000000|-Dzookeeper.snapshot.trust.empty=true|-XX:+UseParNewGC|-XX:+UseConcMarkSweepGC|-XX:CMSInitiatingOccupancyFraction=75|-XX:+UseCMSInitiatingOccupancyOnly|-XX:+DisableExplicitGC|-XX:+PrintGCDetails|-XX:+PrintGCDateStamps|-XX:+PrintGCTimeStamps|-XX:+PrintClassHistogram|-XX:+PrintTenuringDistribution|-XX:+PrintGCApplicationStoppedTime|-XX:+PrintPromotionFailure|-verbose:gc|-XX:GCLogFileSize=64m|-XX:NumberOfGCLogFiles=4|-XX:+UseGCLogFileRotation|-XX:CompileCommand=exclude,org/apache/lucene/lucene54/Lucene54DocValuesConsumer.addSortedNumericField|-XX:CompileCommand=exclude,org/apache/lucene/lucene54/Lucene54DocValuesConsumer.addBinaryField|-XX:CompileCommand=exclude,org/elasticsearch/search/aggregations/metrics/percentiles/tdigest/AbstractTDigestPercentilesAggregator$1.collect|-XX:CompileCommand=exclude,org/apache/lucene/index/SortedNumericDocValuesWriter.flush|-XX:CompileCommand=exclude,org/apache/lucene/codecs/PushPostingsWriterBase.writeTerm|-Xloggc:"/opt/appdynamics/platform/product/events-service/processor/bin/../logs/%p-gc.log"|-DAPPLICATION_HOME="/opt/appdynamics/platform/product/events-service/processor/bin/.."|-classpath|"/opt/appdynamics/platform/product/events-service/processor/bin/../lib/*"|com.appdynamics.analytics.processor.AnalyticsService|-p|"/opt/appdynamics/platform/product/events-service/processor/./conf/events-service-api-store.properties"|-y|"/opt/appdynamics/platform/product/events-service/processor/bin/../conf/events-service-api-store.yml"

CompilerOracle: exclude org/apache/lucene/lucene54/Lucene54DocValuesConsumer.addSortedNumericField
CompilerOracle: exclude org/apache/lucene/lucene54/Lucene54DocValuesConsumer.addBinaryField
CompilerOracle: exclude org/elasticsearch/search/aggregations/metrics/percentiles/tdigest/AbstractTDigestPercentilesAggregatorstart.collect
CompilerOracle: exclude org/apache/lucene/index/SortedNumericDocValuesWriter.flush
CompilerOracle: exclude org/apache/lucene/codecs/PushPostingsWriterBase.writeTerm
10:52:52.934 [main] INFO com.appdynamics.analytics.processor.AnalyticsService - Starting analytics processor with arguments [-p, /opt/appdynamics/platform/product/events-service/processor/./conf/events-service-api-store.properties, -y, /opt/appdynamics/platform/product/events-service/processor/bin/../conf/events-service-api-store.yml]
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging.

 

 

Solution Architect
Tags (1)
By replying you agree to the Terms and Conditions of the AppDynamics Community.
Starting Events Service cluster
4 REPLIES 4
Ryan.Paredez
Community Manager

Hi @Abdulrahman.Kazamel,

 

I couldn't find any relevant info on those errors, but I did find this AppD Knowledge base article that describes how to turn on debugging mode. Perhaps this will give you more insight into the issue.

 

https://community.appdynamics.com/t5/Knowledge-Base/How-do-I-enable-the-Events-Service-debugging-por...

 

 


Thanks,
Ryan, AppDynamics Community Manager



Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Hi @Ryan.Paredez 

I made some edit like 

ssh by hostname, not only with the IP address

maximize each node to 25 G ram 

 

reinstalled the events service cluster by 

bin/platform-admin.sh install-events-service --profile prod --hosts 10.0.30.45 10.0.30.46 10.0.30.47 --data-dir /opt/appdynamics/eventsservice --platform-name AppDPlatform

 

also in each machine 

 

iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 9080 -j ACCEPT
iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 9081 -j ACCEPT
iptables -A INPUT -m state --state NEW -m multiport -p tcp --dports 9300:9400 -j ACCEPT

 

however, the firewall was off

and tried again to restarted the cluster by 

Learn More
[root@console platform-admin]# bin/platform-admin.sh submit-job --platform-name AppDPlatform --service events-service --job restart-cluster
( 1/ 6) Load Events Service cluster configuration: SUCCESS
( 2/ 6) Load all ES cluster hosts into execution context.: SUCCESS
( 3/ 6) Disable transient allocations on masters: Disable transient allocation in the api store node
( 3/ 6) Disable transient allocations on masters: SUCCESS
( 4/ 6) Restart all nodes: Stop the Events Service api store node
( 4/ 6) Restart all nodes: Starting the Events Service api store node
( 4/ 6) Restart all nodes: Stop the Events Service api store node
( 4/ 6) Restart all nodes: Starting the Events Service api store node
( 4/ 6) Restart all nodes: Stop the Events Service api store node
( 4/ 6) Restart all nodes: Starting the Events Service api store node
( 4/ 6) Restart all nodes: SUCCESS
( 5/ 6) Enable transient allocations on masters: Enable transient allocation in the api store node
( 5/ 6) Enable transient allocations on masters: FAILED
Job failed to execute.
Failure occurred: Enable transient allocations on masters
Error message:
Task failed: Enable transient allocation in the api store node
on host: 10.0.30.45
as user: root
with message: Connection to [http://10.0.30.45:9080/v1/elasticsearch/_cluster/settings] failed due to [Failed to connect to /10.0.30.45:9080].

 

 

 

Solution Architect
Ryan.Paredez
Community Manager

Hi @Abdulrahman.Kazamel,

 

Thanks for the additional info. Let's see if the Community can jump in and help you out. In the meantime, if you happen to find a solution, please do come back and share the results here. Knowledge sharing is key to the peer-to-peer community. 

 

 


Thanks,
Ryan, AppDynamics Community Manager



Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Hello @Ryan.Paredez 

 

I made a couple of configuration and at the end 

each time I do the installation the ports in each machine changes to 0.0.0.0 

 

For now, the issue was in DNS resolve as I had to make it manual in each machine at
/etc/hosts file
to give an example

eventsservice-01 10.0.30.xx
eventsservice-02 10.0.30.xx
eventsservice-03 10.0.30.xx


the installation done successfully after this configuration and yet still checking the health as it's not good.

 

*another update *


after the last configuration, the events service were not able to keep running so,

I changed every hostname to IP in the conf/events-service-api-store.properties in each machine.
I stopped/disabled the firewall and iptables,
and now it's up and running in green health since two hours!

 

kindly I would like to thank you for your support.

Thanks
Abdulrahman

Solution Architect