Discussion Feed

Hands-Free Agent Deployments using Kubernetes Air-Date January 17, 2019 Run time: 60 minutes   In this technical deep-dive session, @Sasha.Jeltuhin and @Mark.Prichard demonstrate how to deploy agents hands-free in OpenShift and other Kubernetes environments.   You’ll walk away with: -An overview of the Kubernetes operator framework
 -A guided tour through an automated agent instrumentation using the Kubernetes operator framework
 -Best practices and insightful advice based on AppDynamics customers’ experience
 Also, tune in to hear more about our new cluster monitoring capabilities based on this technology.   Resources -   coming soon There will be a GitHub repo available in the next few weeks, please be sure to check back.  Q&A Transcript Is the Cluster Agent available to download? What does "Hands-Free Agent Deployments using Kubernetes" refer to? If we are using the Cluster Agent, do we still need the Kubernetes extension? Does the Cluster Agent replace the Machine and Java Agent? Can we run the agent inside a namespace? Can we monitor some namespaces but not others? If there are multiple AppDynamics instances, does AppDynamices handle which AppDynamices instance the application data will be sent to? Is AppDynamics looking at deploying the Java Agent as Sidecar instead of copying agent jars within a pod? Is AppDynamics going add supoort for custom metrics for Horizontal Pod Autoscaler like Prometheus does today? With the Cluster Agent, will inactive containers show up under tiers? Are you looking at Linux processes to determine how to dynamically inject the agent and will this work for the Java Agent? Can one cluster report to multiple AppDynamics Controllers? Is there a beta program? Do you have any final thoughts?   Q: Is the Cluster Agent available to download? A: We'll be making it available in a few weeks via our Official GitHub site with full instructions and source code. One of the reasons for this is that it's extensible and there are likely additional features and tweaks that people want to make.   Q: When referring to "Hands-Free Agent Deployments using Kubernetes", is this referring to the hands-free installation of the Machine and App Agent on Linux machines using the Kubernetes framework or just on Kubernetes machines and deployed applications? A: It’s on Linux machines. When we roll out dynamic instrumentation for .NET workloads, it will still only be on Linux for now. This particular session was about App Agents but we can also deploy Machine Agents through the Cluster Agent. However, there is no dynamic instrumentation of the Machine Agent because it is our image.    Q: If we are using the Cluster Agent, do we still need the Kubernetes extension? A: No - If you switch to the cluster agent, you do not need the extension.    Q: Does the Cluster Agent replace the Machine and Java agent? A: Yes, the agent will supercede the MA extension.   Q: Can we run the agent inside a namespace? A: The agent always runs in a single namespace, but it can be configured to monitor the entire cluster or 1-n namespaces.   Q: Can we monitor some namespaces but not others? A: Yes, the agent can monitor all or selected namespaces.   Q: If there are multiple AppDynamics instances, does AppDynamices handle which AppDynamices instance the application data will be sent to? A: Assuming you are referring to a Controller, right now, an instance of the Cluster Agent is tied to a Controller but this is where the operator comes in. If you send a custom resource spec to the operator with another Controller, it will create a new instance of the Cluster Agent that will talk to the other Controller. That's one of the things that the operator makes transparent.   Q: Is AppDynamics looking at deploying the Java Agent as Sidecar instead of copying agent jars within a pod? A: No, we're not looking into Sidecar because our agents are instrumented in the actual JVM of the application. That being said, there are customers who use Init containers as a way of staging agents appropriately so they can be pulled into containers dynamically. We've talked about that approach in other webinars.   Q: Is AppDynamics going add supoort for custom metrics for Horizontal Pod Autoscaler like Prometheus does today? A: We are planning to do some ingestion of Prometheus metrics, but Horizontal Pod Autoscaler is something that would be part of the scope of the Cluster Agent. It's on the list of specialized workers that track metrics. We do have plans and we’ll talk more about that shortly.   Q: With the Cluster Agent, will inactive containers show up under tiers?  A: The current implementation of the Cluster Agent does not affect the presentation of tiers on flowmaps, etc. The metrics are collected for active containers and it is possible to go back in time to see non-running containers. In the next iteration of the agent, when it is integrated into the product, the UI will show only active nodes for the selected timeframe.   Q: Are you looking at Linux processes to determine how to dynamically inject the agent and will this work for the Java Agent?  A: Yes. This instance of the Cluster Agent is configured to look for Linux Java processes, find the ID, and attach. This session is about Java but we’re looking at a similar approach for .NET and Node.js   Q: Can one cluster report to multiple AppDynamics Controllers? A: Currently, one cluster reports to one Controller.   Q: Is there a beta program? A: Yes. Reach out to let us know if you’re interested.   Q: Do you have any final thoughts or takeaways? A: We want to develop this together WITH you so we can open up opportunities for how people instrument and monitor applications. ... View more

 3 Steps to Better Install Agents in Docker Environments Air-Date December 5, 2018 Run time: 60 minutes (Please pardon the 15 seconds of choppy audio during the first minute).     Resource links referenced in the session :  (new blog post)- Hands Off My Docker Containers: Dynamic Java Instrumentation in Three Easy Steps Today's Project: Dynamic-Agent-MA Docs: Attaching the Java Agent to a Running JVM Process Release Upgrade Checklist for Java Agents Community Forum: Java Agent Installation   You have never seen a machine agent like this! This session focuses on best practices for installing AppDynamics Agents in Docker environments. @James.Barfield,Technical Sales Enablement Manager, will lead us through a live demonstration of the 3-step solution he has developed. This solution provides an easy and safe way to add APM monitoring to an existing application, without changing your Dockerfiles or container builds. @Mark.Prichard, Product Management, shares with us where the product team is taking development further on this topic.       Q&A Transcript  Is there a specific version of Docker for which this is supported and does it support Swarm managed containers? Does it needs to be Docker image or it can be done on normal Java apps? Is this currently supported for both Docker and Kubernetes? Which version of the agent will have these Agentless Analytics and Dynamic Attach? What version of AppDynamics Controller are you running for this demo? Is Dynamic Attach only supported with Java apps at this time? What about .NET apps and services? How is the agent configured to do dynamic reporting? Can you explain how Dynamic Attach works in a more JVM way? How many dynamic agents are there per docker host? What do custom metrics work in relation to this solution? Does the container-instance agent report machine statistics of the host machine other than the containers metrics? Do you have to specify a directory or will the .env install in the correct directory automatically? You had a config file for the Controller. Where are the application and tier names pulled from? Does anything have to be instrumented in the container images (e.g., web-front-end, billing-services)? How does this approach apply to PCF? Would this work for Pivotal Kubernetes Services (PKS) for CaaS? Do you have to rerun all the steps if you upgrade one of your own containers? Is there a similar demo for Node.js applications? Can the Machine Agent run in the same container as the Java Agent? Is the Machine Agent container an actual running agent, essentially just a repo for the JVM agents, or both? In the demo, you showed that all containers were running on single box/machine. Does it work the same way if the containers are on different/multiple machines? We have multiple products and instances of each product that may need to point to different Controllers. Is it easy to parameterize the Controller file? When you shut down the container, it appeared that the node was removed immediately from the tier. Is that build into the MA container or something else in place to manage the agent life cycle? How do health rules work when containers disappear from flowmap? If the dynamic agent container is shutdown, do we simply start the container up again in order to start monitoring? Unless the .tar file you upload and deploy inside of a container is deployed to a persistent volume or bind mount location, will it be lost when the container goes down? How does Dynamic Attach work in auto-scale environments? Are there sidecars in this implementation? How does this work with Gemfire? Are there any docs on auto attach using JBoss fuse? What are your final thoughts?   Q: Is there a specific version of Docker for which this is supported and does it support Swarm managed containers? This supports the most recent version of Docker. We've tested against many Java version. The features of docker that we're using are very basic and have been in the product forever. In terms of Dynamic Attach, this has been in the JVM for a long time and is supported with JVM versions going back a long time. We haven't tested against Swarm, but it shouldn't be any different.   Q:Does it needs to be Docker image or it can be done on normal Java apps? It can be. This is one package solution. We could package a similar solution that will go through a server and instrument all the Java apps on that server.   Q: Is this currently supported for both Docker and Kubernetes? It supports Docker and will work on a Kubernetes cluster, but it’s not a Kubernetes solution yet. One of our next webinars will show Kubernetes.    Q: Which version of the agent will have these Agentless Analytics and Dynamic Attach? Dynamic Attach has been available for several years. Agentless Analytics is still in beta. That's straight forward because that's on the Events Service side and enabling collectors to listen to those things. This solution makes all of that transparent. The dynamic agent solution does all that for you anyway. It creates the Analytics Agent, does the reporting, etc.   Q: What version of AppDynamics Controller are you running for this demo? We used the most recent v4.5.4 Controller and the newest agents. If you are running a v4.4.3+ Controller, it should work with that. Anything after v4.5.2 is backwards compatible starting with v4.4.3. You can run the newest agent or older agents against a new Controller.   Q: Is Dynamic Attach only supported with Java apps at this time? What about .NET apps and services? Dynamic Attach is a technology that's specific to the JVM. However, there are other approaches that can get the same effect using features of the langauge runtimes for the other agents. That's on the roadmap.   Q: How is the agent configured to do dynamic reporting? "Dynamic attach” just means that you’re taking our agent and attaching it to an already running JVM. The agent behaves in a normal way. For dynamic reporting, all we're doing is attaching a normal agent dynamically to a running JVM. If you want to learn more about that, see:   Q: Can you explain how Dynamic Attach works in a more JVM way? "It does class retransformations on the JVM, similar to what the Java agent will do if it's there from the beginning. It injects the various markers - it does that at runtime instead of at the beginning. There's a description of this in our JVM documentation. There are hooks in the JVM that are flagged to do this. It uses Xbootclasspath.   Q: How many dynamic agents are there per docker host? One.   Q: What do custom metrics work in relation to this solution? The agent is working as it normally does. You can define custom metrics as you usually would and it will work.   Q: Does the container-instance agent report machine statistics of the host machine other than the containers metrics? Yes, this is part of our standard Server Agent.   Q: Do you have to specify a directory or will the .env install in the correct directory automatically? When you pull down the github repo, it will pull down 4 files: the run.sh, the stop.sh, the controller.env, and the readme.txt. The controller.env needs to be in the same directory as the run.sh file. But it will all be there together.   Q: You had a config file for the Controller. Where are the application and tier names pulled from? The Controller.env file is configuration file for the Controller. That's where the application name is set, as well as the connection information for the Controller. By default, the solution will use the host name of the container as the tier name but there are other options for arriving at the tier name for each container. If you have a specific schema for naming your nodes and tiers, you can incorporate that into the solution. If you look at readme.txt file and look at the TIER_NAME_FROM and TIER_NAME_PARAM, those are around naming tiers.   Q: Does anything have to be instrumented in the container images (e.g., web-front-end, billing-services)? No, not ahead of time. This will happen dynamically as it’s running.   Q: How does this approach apply to PCF? This doesn’t, since the approach/solution we've shown here is Docker only. In general, instrumentation of agents in a PCF world is a lot easier due to the nature of the buildpack. Therefore, there's less of a need for something like this. However, if this is something people really want to do in a PCF environment, please reach out to your Account Manager, who can help you connect with Mark Prichard.   Q: Would this work for Pivotal Kubernetes Services (PKS) for CaaS? Yes, but this is something we will discuss more in a future webinar.   Q: Do you have to rerun all the steps if you upgrade one of your own containers? We are assuming that you are referring to upgrading your own application. In that case, you don’t need to rerun any of these steps. The dynamic agent will still be running and checking for new containers. If you're upgrading a container, we assume you're going to be creating a new container. The new container will get picked up and instrumented.   Q: Is there a similar demo for Node.js applications? It’s not quite ready yet. but keep an eye out for it. It will work a little differently since there's no Dynamic Attach, but it can add the pieces it needs to add. It will then stop and start the container. When the container starts, it will pick up the instrumentation that was added. Please reach out if this is an urgent need for your organization.   Q: Can the Machine Agent run in the same container as the Java Agent? It can, but it's not normally good practice though because there's no added value. You typically want just one process per container. The normal pattern would be to have your application containers running with the APM agents attached dynamically and you have one additional container which is configured to report on the host and containers that are running.   Q: Is the Machine Agent container an actual running agent, essentially just a repo for the JVM agents, or both? The Machine Agent container is a running machine agent and it has a dynamic agent running as an extension. Server monitoring capabilities use the same mechanism. This is a very standard part of how our Machine Agent works.   Q: In the demo, you showed that all containers were running on single box/machine. Does it work the same way if the containers are on different/multiple machines? This would need to be installed on each host machine.   Q: We have multiple products and instances of each product that may need to point to different Controllers. Is it easy to parameterize the Controller file? Not currently, but it's on our list of features to add to the product. It's a good example of where you have containers that would report to different applications.   Q: When you shut down the container, it appeared that the node was removed immediately from the tier. Is that build into the MA container or something else in place to manage the agent life cycle? That’s part of the Controller functionality and that can be confingured on the Controller.   Q: How do health rules work when containers disappear from flow map? They work the same as they would if they had been instrumented with Java from the start.   Q: If the dynamic agent container is shutdown, do we simply start the container up again in order to start monitoring? Yes. The application will carry on being monitored once the agents are there. That container can drop down and it wouldn't affect anything. If you should down the container, it would have to be brought back just to pick up any new containers that were running or restarted.   Q: Unless the .tar file you upload and deploy inside of a container is deployed to a persistent volume or bind mount location, will it be lost when the container goes down? Yes, if the container goes down and a new one comes up, it will reinstrument the new container. The whole point is that the other container image is unmodified.   Q: How does Dynamic Attach work in auto-scale environments? Autoscale would work the same we way we saw when we took down the container and spun a new one during the demo. Every minute it’s looking to see if there are new containers and if it sees a new one (like in the autoscale scenario), it will instrument it then. It will handle autoscaling environments just fine.   Q: Are there sidecars in this implementation? There are no sidecars in this implementation. You're just injecting the agent into the container. It’s a different approach from what you would do with sidecars. In some ways, it's much simpler.   Q: How does this work with Gemfire?  Gemfire works with our normal agents. We haven't tried it, but I see no reason why it wouldn't work. The same is true for a lot of JVM-based products. You just need to have a way to dynamically attach that to the JVM.   Q: Are there any docs on auto attach using JBoss Fuse? Dynamic attach works fine with jboss. The docs mentioned something about when you do AppDynamics instrumentation with JBoss, there is a particular class you need to include - the JBoss modules system packages - but that's something you'd have to do with the agent anyway. It's in the blog under "Requirements" section: https://blog.appdynamics.com/engineering/hands-off-my-docker-containers-dynamic-java-instrumentation/   Q: What are your final thoughts? This is our first step and we have a lot we can add to it. Most features have been things we’ve thought of but once this goes out to the field in real world situations, there will be more scenarios that we can cover. We’ll be coming back in the New Year with Kubernetes environments and we'll try to address Swarm or Node.js. If you have a specific question or environment you want to try this in, reach out to your AppDynamics Account Manager and they can help you. All materials are out there - get going and let’s hear the feedback. ... View more