Help
Not a customer? Start a free trial

Not a customer? Click the 'Start a free trial' link to begin a 30-day SaaS trial of our product and to join our community.

Existing Cisco AppDynamics customers should click the 'Sign In' button to authenticate to access the community

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Auto instrument not working

sattha.puangput
Creator

‎06-28-2021 09:38 AM

Hi All,

 

Currently, I try to implement cluster-operator and cluster-agent on K8s. the cluster metric shows under the "Server/Cluster" menu correctly. However, I would like to config auto-instrument for Java application but no hope follow the validate guide here is still not much helpful...

 

https://docs.appdynamics.com/21.5/en/infrastructure-visibility/monitor-kubernetes-with-the-cluster-a...

 

apiVersion: appdynamics.com/v1alpha1
kind: Clusteragent
metadata:
  name: k8s-cluster-agent
  namespace: appdynamics
spec:
  appName: "TEST-CLUSTER"
  controllerUrl: "https://xxx.appdynamics.com"
  account: "xxxdev"
  logLevel: DEBUG
  logFileSizeMb: 7
  logFileBackups: 6
  # docker image info
  image: "my.own.repository.org:5000/appdynamics/cluster-agent:latest"
  serviceAccountName: appdynamics-cluster-agent
  nsToMonitor:
    - "appdynamics"
    - "cma-dev"
    - "cma-qa"
  #
  # auto-instrumentation config
  #
  instrumentationMethod: Env
  nsToInstrumentRegex: "xxx-dev|xxx-qa"
  appNameStrategy: label
  defaultAppName: not-specific
  defaultCustomConfig: "-Dappdynamics.agent.maxMetrics=15000"
  defaultEnv: JAVA_TOOL_OPTIONS
  resourcesToInstrument:
    - Deployment
  imageInfo:
    java:
      image: "my.own.repository.org:5000/appdynamics/java-agent:latest"
      agentMountPath: /opt/appdynamics
  instrumentationRules:
    - namespaceRegex: "xxx-dev"
      language: java
      env: JAVA_TOOL_OPTIONS
      appNameLabel: app
      instrumentContainer: first
      imageInfo:
        image: "my.own.repository.org:5000/appdynamics/java-agent:latest"
        agentMountPath: /opt/appdynamics
    - namespaceRegex: "xxx-qa"
      language: java
      env: JAVA_TOOL_OPTIONS
      appNameLabel: app
      instrumentContainer: first
      imageInfo:
        image: "my.own.repository.org:5000/appdynamics/java-agent:latest"
        agentMountPath: /opt/appdynamics

 

But I got this....

 

[DEBUG]: 2021-06-28 16:12:36 - instrumentationconfig.go:628 - rule xxx-qa matches Deployment spring-api-productprofile in namespace xxx-qa with labels map[app:spring-api-productprofile]
[DEBUG]: 2021-06-28 16:12:36 - instrumentationconfig.go:639 - Found a matching rule {xxx-qa  map[]   java first  -Dappdynamics.agent.maxMetrics=15000 JAVA_TOOL_OPTIONS map[agent-mount-path:/opt/appdynamics image:my-own-repository.org:5000/appdynamics/java-agent:latest] map[bci-enabled:true port:3892] 0 0 app  0 false []} for Deployment spring-api-productprofile in namespace xxx-qa with labels map[app:spring-api-productprofile]
[DEBUG]: 2021-06-28 16:12:36 - instrumentationconfig.go:249 - Instrumentation state for Deployment spring-api-productprofile in namespace xxx-qa with labels map[app:spring-api-productprofile] is false
[DEBUG]: 2021-06-28 16:12:36 - instrumentationconfig.go:628 - rule xxx-dev matches Deployment spring-api-deliveries in namespace xxx-dev with labels map[app:spring-api-deliveries]
[DEBUG]: 2021-06-28 16:12:36 - instrumentationconfig.go:639 - Found a matching rule {xxx-dev  map[]   java first  -Dappdynamics.agent.maxMetrics=15000 JAVA_TOOL_OPTIONS map[agent-mount-path:/opt/appdynamics image:my-own-repository.org:5000/appdynamics/java-agent:latest] map[bci-enabled:true port:3892] 0 0 app  0 false []} for Deployment spring-api-deliveries in namespace xxx-dev with labels map[app:spring-api-deliveries]
[DEBUG]: 2021-06-28 16:12:36 - instrumentationconfig.go:249 - Instrumentation state for Deployment spring-api-deliveries in namespace xxx-dev with labels map[app:spring-api-deliveries] is false

 

Please advice

8 REPLIES 8

Mario.Morelli
Architect

‎06-29-2021 06:05 AM

Hi Sattha

 

You have set the appNameStrategy : label

And in the instrumentation rules, you state that it should use the below label of the Java pods to name the Application.

 

Do you have a label in your java app's deployment spec that is named "app"? As it will look for that label to set the Application Name. If it's empty it wont register. Your logs show that the instrumentation rule is matching the pod so the rest seem to be correct.

appNameLabel: app

 

I would suggest to set the following, to just validate testing, before switching back to using labels to set the AppD Application Name

 

 

appNameStrategy: manual

Change 

appNameLabel: app to appName: XXX_DEV

                                             appName: XXX_QA

 

Then redeploy and let us know

 

Ciao

 


Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Learn more about me in the Community Member Spotlight Q & A

sattha.puangput
Creator
In response to Mario.Morelli

‎06-29-2021 06:21 PM

Thank you for your advice, I update configure to this one now... but still got the same log output

 

apiVersion: appdynamics.com/v1alpha1
kind: Clusteragent
metadata:
  name: k8s-cluster-agent
  namespace: appdynamics
spec:
  appName: "SEACMACLUSTER-TH-DEVQA"
  controllerUrl: "https://xxx.saas.appdynamics.com"
  account: "xxx"
  logLevel: DEBUG
  logFileSizeMb: 7
  logFileBackups: 6
  # docker image info
  image: "my-own-repository.org:5000/appdynamics/cluster-agent:latest"
  serviceAccountName: appdynamics-cluster-agent
  nsToMonitor:
    - "appdynamics"
    - "cma-dev"
    - "cma-qa"
  proxyUrl: "http://xx.xx.xx.xx:80"
  proxyUser: "xxx\\svc_localapp_dev"
  #
  # auto-instrumentation config
  #
  instrumentationMethod: Env
  nsToInstrumentRegex: cma-dev|cma-qa
  appNameStrategy: manual
  defaultAppName: SEACMA-TH-NONPROD
  #defaultCustomConfig: "-Dappdynamics.agent.maxMetrics=15000"
  defaultEnv: JAVA_TOOL_OPTIONS
  resourcesToInstrument:
    - Deployment
  instrumentContainer: first
  imageInfo:
    java:
      image: "my-own-repository.org:5000/appdynamics/java-agent:latest"
      agentMountPath: /opt/appdynamics
    dotnetcore:
      image: "docker.io/appdynamics/dotnet-core-agent:latest"
      agentMountPath: /opt/appdynamics
    nodejs:
      image: "docker.io/appdynamics/nodejs-agent:20.8.0-stretch-slimv10"
      agentMountPath: /opt/appdynamics
  instrumentationRules:
    - namespaceRegex: "cma-dev"
      appName: SEACMA-TH-DEV
      env: JAVA_TOOL_OPTIONS
      instrumentContainer: first
      imageInfo:
        image: "my-own-repository.org:5000/appdynamics/java-agent:latest"
        agentMountPath: /opt/appdynamics
    - namespaceRegex: "cma-qa"
      appName: SEACMA-TH-QA
      env: JAVA_TOOL_OPTIONS
      instrumentContainer: first
      imageInfo:
        image: "my-own-repository.org:5000/appdynamics/java-agent:latest"
        agentMountPath: /opt/appdynamics

 

Log

[DEBUG]: 2021-06-30 01:08:59 - instrumentationconfig.go:628 - rule cma-qa matches Deployment spring-api-productprofile in namespace cma-qa with labels map[app:spring-api-productprofile]
[DEBUG]: 2021-06-30 01:08:59 - instrumentationconfig.go:639 - Found a matching rule {cma-qa  map[] SEACMA-TH-QA  java first   JAVA_TOOL_OPTIONS map[agent-mount-path:/opt/appdynamics image:my-own-repository:5000/appdynamics/java-agent:latest] map[bci-enabled:true port:3892] 0 0   0 false []} for Deployment spring-api-productprofile in namespace cma-qa with labels map[app:spring-api-productprofile]
[DEBUG]: 2021-06-30 01:08:59 - instrumentationconfig.go:249 - Instrumentation state for Deployment spring-api-productprofile in namespace cma-qa with labels map[app:spring-api-productprofile] is false


[DEBUG]: 2021-06-30 01:08:59 - instrumentationconfig.go:628 - rule cma-dev matches Deployment spring-api-productprofile in namespace cma-dev with labels map[app:spring-api-productprofile]
[DEBUG]: 2021-06-30 01:08:59 - instrumentationconfig.go:639 - Found a matching rule {cma-dev  map[] SEACMA-TH-DEV  java first   JAVA_TOOL_OPTIONS map[agent-mount-path:/opt/appdynamics image:my-own-repository.org:5000/appdynamics/java-agent:latest] map[bci-enabled:true port:3892] 0 0   0 false []} for Deployment spring-api-productprofile in namespace cma-dev with labels map[app:spring-api-productprofile]
[DEBUG]: 2021-06-30 01:08:59 - instrumentationconfig.go:249 - Instrumentation state for Deployment spring-api-productprofile in namespace cma-dev with labels map[app:spring-api-productprofile] is false

 

What I try...

  1. check K8s deployment config to make sure that config is applied to the current "k8s-cluster-agent" pod
  2. delete and recreate application deployment but no init-container config apply (it should not I think since Instrument still false

 

Current dashboard screenshot...

clusterclusterapplicationapplication

0 Kudos

Mario.Morelli
Architect
In response to sattha.puangput

‎06-29-2021 11:58 PM

Thanks Sattha

 

Could you remove all instances of the following from the cluster agent yaml

instrumentContainer: first

and add the following to each of the 2 instrumentation rules

containerMatchString: .*

 

The redeploy, and check if it instruments the pods now


Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Learn more about me in the Community Member Spotlight Q & A
0 Kudos

sattha.puangput
Creator
In response to Mario.Morelli

‎06-30-2021 09:22 AM

I try to add the following config but it does not work...

 

instrumentContainer: select
containerMatchString: .*

 

apiVersion: appdynamics.com/v1alpha1
kind: Clusteragent
metadata:
  name: k8s-cluster-agent
  namespace: appdynamics
spec:
  appName: "SEACMACLUSTER-TH-DEVQA"
  controllerUrl: "https://xxxxxdev.saas.appdynamics.com"
  account: "xxxxxdev"
  logLevel: DEBUG
  logFileSizeMb: 7
  logFileBackups: 6
  # docker image info
  image: "my-own-repository.org:5000/appdynamics/cluster-agent:latest"
  serviceAccountName: appdynamics-cluster-agent
  ### Uncomment the following two lines if you need pull secrets
  #imagePullSecrets:
  #  name: "<your-docker-pull-secret-name>"
  nsToMonitor:
    - "appdynamics"
    - "cma-dev"
    - "cma-qa"
  proxyUrl: "http://172.30.60.46:80"
  proxyUser: "Th-xxxxx\\svc_localapp_dev"
  #
  # auto-instrumentation config
  #
  instrumentationMethod: Env
  nsToInstrumentRegex: cma-dev|cma-qa
  appNameStrategy: manual
  defaultAppName: SEACMA-TH-NONPROD
  instrumentContainer: select
  #defaultCustomConfig: "-Dappdynamics.agent.maxMetrics=15000"
  defaultEnv: JAVA_TOOL_OPTIONS
  resourcesToInstrument:
    - Deployment
  imageInfo:
    java:
      image: "my-own-repository.org:5000/appdynamics/java-agent:latest"
      agentMountPath: /opt/appdynamics
    dotnetcore:
      image: "docker.io/appdynamics/dotnet-core-agent:latest"
      agentMountPath: /opt/appdynamics
    nodejs:
      image: "docker.io/appdynamics/nodejs-agent:20.8.0-stretch-slimv10"
      agentMountPath: /opt/appdynamics
  instrumentationRules:
    - namespaceRegex: "cma-dev"
      appName: SEACMA-TH-DEV
      env: JAVA_TOOL_OPTIONS
      instrumentContainer: select
      containerMatchString: .*
      imageInfo:
        image: "my-own-repository.org:5000/appdynamics/java-agent:latest"
        agentMountPath: /opt/appdynamics
    - namespaceRegex: "cma-qa"
      appName: SEACMA-TH-QA
      env: JAVA_TOOL_OPTIONS
      instrumentContainer: select
      containerMatchString: .*
      imageInfo:
        image: "my-own-repository.org:5000/appdynamics/java-agent:latest"
        agentMountPath: /opt/appdynamics

  Still same error

[DEBUG]: 2021-06-30 16:12:11 - instrumentationconfig.go:628 - rule cma-qa matches Deployment spring-api-productprofile in namespace cma-qa with labels map[app:spring-api-productprofile]
[DEBUG]: 2021-06-30 16:12:11 - instrumentationconfig.go:639 - Found a matching rule {cma-qa  map[] SEACMA-TH-QA  java select .*  JAVA_TOOL_OPTIONS map[agent-mount-path:/opt/appdynamics image:pvnexusho001th.dmz.th-tesco.org:5000/appdynamics/java-agent:latest] map[bci-enabled:true port:3892] 0 0   0 false []} for Deployment spring-api-productprofile in namespace cma-qa with labels map[app:spring-api-productprofile]
[DEBUG]: 2021-06-30 16:12:11 - instrumentationconfig.go:249 - Instrumentation state for Deployment spring-api-productprofile in namespace cma-qa with labels map[app:spring-api-productprofile] is false

 

What I try

  • Update config according to your suggestion
  • Delete cluster-operator  K8S deployment
  • Delete cluster-agent K8S deployment
  • recreate both cluster-operator and cluster-agent
  • check deployment configuration change to what we need
  • delete application deployment and recreate it --- not thing happen no init-container
0 Kudos

Mario.Morelli
Architect
In response to sattha.puangput

‎07-01-2021 12:40 AM - edited ‎07-01-2021 03:06 AM

Hi Sattha

 

Can you try the below YAML Config

 

 

apiVersion: appdynamics.com/v1alpha1
kind: Clusteragent
metadata:
  name: k8s-cluster-agent
  namespace: appdynamics
spec:
  appName: "SEACMACLUSTER-TH-DEVQA"
  controllerUrl: "https://xxxxxdev.saas.appdynamics.com"
  account: "xxxxxdev"
  image: "my-own-repository.org:5000/appdynamics/cluster-agent:latest"
  logLevel: Debug
  logFileSizeMb: 10
  logFileBackups: 5
  stdoutLogging: "true"
  serviceAccountName: appdynamics-cluster-agent
  nsToMonitorRegex: appdynamics|cma-dev|cma-qa
  proxyUrl: "http://172.30.60.46:80"
  proxyUser: "Th-xxxxx\\svc_localapp_dev"
  #
  # auto-instrumentation config
  #
  instrumentationMethod: Env
  nsToInstrumentRegex: cma-dev|cma-qa
  appNameStrategy: manual
  defaultAppName: SEACMA-TH-NONPROD
  defaultEnv: JAVA_TOOL_OPTIONS
  instrumentationRules:
    - namespaceRegex: cma-dev
      language: java
      appName: SEACMA-TH-DEV
      containerMatchString: .*
      imageInfo:
        image: "my-own-repository.org:5000/appdynamics/java-agent:latest"
        agentMountPath: /opt/appdynamics

 


Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Learn more about me in the Community Member Spotlight Q & A

sattha.puangput
Creator
In response to Mario.Morelli

‎07-01-2021 07:40 AM

I try your .yaml file but when creating `cluster-agent.yaml` there is the following error

error: error validating "appdynamics/cluster-agent-test.yaml": error validating data: ValidationError(Clusteragent.spec): unknown field "nsToMonitorRegex" in com.appdynamics.v1alpha1.Clusteragent.spec; if you choose to ignore these errors, turn validation off with --validate=false

 

0 Kudos

Mario.Morelli
Architect
In response to sattha.puangput

‎07-01-2021 07:53 AM - edited ‎07-01-2021 07:55 AM

Thanks Sattha

 

Can you just confirm a couple of things

 

1. Which version of K8S are you running?

2. What version of the Cluster Agent Operator are you running? If running version 1.14 or less you should have deployed the Operator with the below YAML File

cluster-agent-operator-1.14-or-less.yaml

 

You can try the below YAML and advise

apiVersion: appdynamics.com/v1alpha1
kind: Clusteragent
metadata:
  name: k8s-cluster-agent
  namespace: appdynamics
spec:
  appName: "SEACMACLUSTER-TH-DEVQA"
  controllerUrl: "https://xxxxxdev.saas.appdynamics.com"
  account: "xxxxxdev"
  image: "my-own-repository.org:5000/appdynamics/cluster-agent:latest"
  logLevel: Debug
  logFileSizeMb: 10
  logFileBackups: 5
  stdoutLogging: "true"
  serviceAccountName: appdynamics-cluster-agent
  nsToMonitor:
    - "appdynamics"
    - "cma-dev"
    - "cma-qa"
  proxyUrl: "http://172.30.60.46:80"
  proxyUser: "Th-xxxxx\\svc_localapp_dev"
  #
  # auto-instrumentation config
  #
  instrumentationMethod: Env
  nsToInstrument:
    - "appdynamics"
    - "cma-dev"
    - "cma-qa"
  appNameStrategy: manual
  defaultAppName: SEACMA-TH-NONPROD
  defaultEnv: JAVA_TOOL_OPTIONS
  instrumentationRules:
    - namespaceRegex: cma-dev
      language: java
      appName: SEACMA-TH-DEV
      containerMatchString: .*
      imageInfo:
        image: "my-own-repository.org:5000/appdynamics/java-agent:latest"
        agentMountPath: /opt/appdynamics

 


Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Learn more about me in the Community Member Spotlight Q & A
0 Kudos

Pooja.Patel
Explorer

‎03-06-2023 11:57 AM

did it work @sattha.puangput ? I am also facing the same issue with my containerized app.

 

0 Kudos
Join Us On December 10
Learn how Splunk and AppDynamics are redefining observability

Register Now!
Observe and Explore
Dive into our Community Blog for the Latest Insights and Updates!

Read the blog here

Thank you! Your submission has been received!

Oops! Something went wrong while submitting the form

By clicking subscribe, I have read and understood the Privacy Policy and Terms of Use.
Platform
Solutions
Learn

Company
Support