cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Cluster Agent Pod Stuck in CrashLoopBackOff

sajo.sam
Producer

I'm trying to deploy a cluster agent in my Kubernetes cluster to monitor the infrastructure using the kubectl CLI. I've followed the steps and executed these commands:

 

 

kubectl create -f cluster-agent-operator.yaml

kubectl -n appdynamics create secret generic cluster-agent-secret --from-literal=controller-key=<access-key>

kubectl create -f cluster-agent.yaml

 

 

However, the cluster agent pod is stuck in the "CrashLoopBackOff" state. The logs indicate an issue with the account access key:

 

 

[ERROR]: 2024-04-03 18:29:45 - main.go:183 - Account accessKey is not specified
[ERROR]: 2024-04-03 18:29:45 - main.go:184 - Please provide account accessKey before starting cluster-agent. Exiting...

 

 

  • I've verified that the cluster-agent-secret contains the controller-key with the correct access key value.

What could be causing this issue despite providing the access key in the secret? Are there any additional configuration steps I might be missing?

Reference :

https://docs.appdynamics.com/appd/22.x/latest/en/infrastructure-visibility/monitor-kubernetes-with-t...

7 REPLIES 7

Ryan.Paredez
Community Manager

Hi @sajo.sam,

I found this TKB article. Please check it out and see if it helps.
https://community.appdynamics.com/t5/Knowledge-Base/How-do-I-debug-common-Linux-Private-Synthetic-Ag...

 


Thanks,

Ryan, Cisco AppDynamics Community Manager




Found something helpful? Click the Accept as Solution button to help others find answers faster.

Liked something? Click the Thumbs Up button.



Check out Observabiity in Action

new deep dive videos weekly in the Knowledge Base.

Ryan.Paredez
Community Manager

Hi @sajo.sam,

Did you get a chance to check out the TKB article or have you found a solution you can share?


Thanks,

Ryan, Cisco AppDynamics Community Manager




Found something helpful? Click the Accept as Solution button to help others find answers faster.

Liked something? Click the Thumbs Up button.



Check out Observabiity in Action

new deep dive videos weekly in the Knowledge Base.

Hi @Ryan.Paredez 


I tried but I'm stuck with another issue. The logs given below show it faces some errors with "Failed to send agent registration request: Post "accountname.saas.appdynamics.com:8080/sim/v2/agent/clusterRegistration ": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"

 

[ERROR]: 2024-04-09 11:20:38 - secretconfig.go:68 - Problem With Getting /opt/appdynamics/cluster-agent/secret-volume/api-user Secret: open /opt/appdynamics/cluster-agent/secret-volume/api-user: no such file or directory

[INFO]: 2024-04-09 11:20:38 - main.go:78 - Kubernetes version: v1.29.0
[INFO]: 2024-04-09 11:20:38 - main.go:236 - Registering cluster agent with
 controller host : accountname.saas.appdynamics.com
 controller port : 8080
 account name : accountname
[WARNING]: 2024-04-09 11:20:38 - agentregistrationmodule.go:352 - "default" is not a valid namespace in your kubernetes cluster

[INFO]: 2024-04-09 11:20:38 - agentregistrationmodule.go:356 - Established connection to Kubernetes API
[INFO]: 2024-04-09 11:20:38 - agentregistrationmodule.go:68 - Cluster name: fromKube
[INFO]: 2024-04-09 11:20:38 - agentregistrationmodule.go:119 - Initial Agent registration
[ERROR]: 2024-04-09 11:21:08 - agentregistrationmodule.go:131 - Failed to send agent registration request: Post "accountname.saas.appdynamics.com:8080/sim/v2/agent/clusterRegistration": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
[ERROR]: 2024-04-09 11:21:08 - agentregistrationmodule.go:132 - clusterId: -1
[ERROR]: 2024-04-09 11:21:08 - agentregistrationmodule.go:134 - Registration properties: {}
[INFO]: 2024-04-09 11:21:38 - agentregistrationmodule.go:119 - Initial Agent registration

 

^ Post edited by @Ryan.Paredez to remove mentions and links to Account name. For security and privacy reasons, please redact the name of your Account in Community posts.

I see this in your log, have you figured out why this is happening?
agentregistrationmodule.go:352 - "default" is not a valid namespace in your kubernetes cluster

 

Hy @Marcie.Sirbaugh 

I just looked into it but didn't get any information from this because the namespace default is already there but I don't know where i'm missing

Ryan.Paredez
Community Manager

Hi @Marcie.Sirbaugh,

I see you have an open ticket with the same error you asked Sajo about 

agentregistrationmodule.go:352

Perhaps you can continue to share any outcomes from that interaction with your ticket here with Sajo.

 


Thanks,

Ryan, Cisco AppDynamics Community Manager




Found something helpful? Click the Accept as Solution button to help others find answers faster.

Liked something? Click the Thumbs Up button.



Check out Observabiity in Action

new deep dive videos weekly in the Knowledge Base.

Rajesh.Ganapavarapu
AppDynamics Team

Hello @sajo.sam,

The error message you've encountered "agentregistrationmodule.go:132 - clusterId: -1" indicates that the cluster agent received incorrect controller information or that the controller is rejecting the registration.

To verify if you've configured the correct controller access key, you can execute the following command:

kubectl get secret cluster-agent-secret -n appdynamics -o jsonpath='{.data.controller-key}' | base64 --decode

Please note that DONOT share any controller sensitive information in this public platform.

 

 

Regarding the error mentioned in your comments:

 

[ERROR]: 2024-04-09 11:21:08 - agentregistrationmodule.go:131 - Failed to send agent registration request: Post "accountname.saas.appdynamics.com:8080/sim/v2/agent/clusterRegistration": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

 

 

This is likely due to a network connection issue. You can check the network connection from one of the running pods using the following command:

curl -v -k -u singularity-agent@<accountname>  https://<controllerhost>.saas.appdynamics.com:<port>/sim/v2/agent/clusterRegistration

 

Please check the above if that helpful.

Best Regards,
Rajesh Ganapavarapu

Join Us On December 10
Learn how Splunk and AppDynamics are redefining observability


Register Now!

Observe and Explore
Dive into our Community Blog for the Latest Insights and Updates!


Read the blog here