Discussion Feed

Enabling SSL termination keeps the incoming communication channel secure all the way to the Events Service. Enabling encryption across nodes keeps all communication between the Elastic Search nodes and the data center secure. You have to have both in order to keep communication encrypted coming into and between the Events Service.   Refer to https://docs.oracle.com/cd/E19830-01/819-4712/ablqw/index.html for more information on how to work with (importing) certificates and SSL.   Table of Contents Enable SSL Termination Events Service Configuration - Server Certificate keystore Trust store Events Service Configuration - Server Certificate trust store EUM Server Certificate trust store Controller Configuration - Server Certificate trust store Analytics Agent Configuration - Server Certificate trust store Using a Load Balancer Enable Encryption Across Nodes Releases 4.4.3 and Later Releases 4.4.2 and Earlier   Enable SSL Termination This topic provides information on how to  secure an  on-premises  installation of the Events Service using SSL.  Enabling SSL for the Events Service requires  configuration changes for the Events Service and may involve changes for other components, such as the Controller, the Analytics Agent, or the load balancers, if any. Terminology: events-svc.jks: keystore containing the key/certificate pair for the Events Service https server. events-svc:  keystore alias referencing the key/pair certificate in the Events Service key store. events-svc.csr: certificate signing request for the Events Service https server. events-svc.crt: signed certificate for the Events Service https server. root_ca.crt: root certificate of a private singing authority. events-svc.chain: concatenated ordered set of intermediate certificate(s) for the server certificate. events-svc.p7b: signed certificate with chain for the Events Service https server. ${JAVA_HOME}: directory containing the JRE that is used to launch the Events Service. cacerts: default JRE trusted root certificate store (${JAVA_HOME}/lib/security/ cacerts). Events Service Configuration - Server Certificate keystore To establish trust between the Events Service and other components of the AppDynamics Platform,  you must create a keystore for the Events Service that contains its own signed certificate.  Run the keytool command to create a new key pair for the Events Service  and store it in es-keystore.jks.  Follow the onscreen instructions to configure the certificate. This generates a self-signed certificate in the keystore. For example:  This command creates a key pair with a validity of 730 days (2 years). The standard key size is 2048 bits with a 2 year validity. Replace 730 with the validity period appropriate for your environment, if desired.  ${JAVA_HOME}/bin/keytool -genkeypair -keyalg RSA -keysize 2048 -validity 730 -alias events-svc -keystore events-svc.jks Follow the onscreen instructions to configure the certificate. Note that: The domain name used in the appdynamics.on.premise.event.service.url  property and the http.event.endpoint property must match the Common Name (CN) of the certificate used by the Events Service. Enter a password for the key that follows your corporate password policy. This generates a self-signed certificate in the keystore. We'll generate a signing request for the certificate next. Generate a certificate signing request (CSR) for the certificate you created, using key events-svc stored in events-svc.jks, which was generated in step 1. ${JAVA_HOME}/bin/keytool -certreq -alias event-svc -keystore events-svc.jks -file events-svc.csr Request the certificate from your PKI service provider in the preferred format PKCS#7. The PKI service provider will provide you with the certificate and intermediate certificate(s). When the PKI service provider returns the individual certificates, create two files and proceed to step 3A. Store the server certificate in the events-svc.crt file and the intermediate certificate(s) in the events-svc.chain file.  The intermediate certificates in the server.chain file must be ordered, with each certificate immediately following the certificate it signed.    3A. Convert the certificate in PKCS#7 format. It is important that the certificate is imported as one entry with it's intermediate certificates Create the PKCS#7 file. From the OpenSSL certificate, when using intermediate certificate(s): openssl crl2pkcs7 -nocrl -certfile events-svc.crt -certfile events-svc.chain -out events-svc.p7b From OpenSSL certificate: openssl crl2pkcs7 -nocrl -certfile events-svc.crt -out events-svc.p7b Import the signed certificate in the key store. For example:  ${JAVA_HOME}/bin/keytool -importcert -alias events-svc -keystore events-svc.jks -file events-svc.crt Stop the Events Service. In the Events Service API properties file, edit the properties a fter the line port: ${ad.dw.http.port}  to appear as follows: ad.dw.https.enabled=true ad.dw.https.keyStorePath=<full path to es-keystore.jks> ad.dw.https.keyStorePassword=<es-keystore.jks password> ad.dw.https.trustStorePath: ${JAVA_HOME}/jre/lib/security/cacerts ad.dw.https.trustStorePassword=<password> Note: ad.dw.https.trustStorePassword can be any value, depending on what the password was set to when you created the trust store. It is "changeme" by default. ad.dw.https.certAlias=<alias> Start the Events Service. Once the node comes up successfully, repeat this procedure for all the nodes in the Events Service deployment.  Trust store The acquired server certificate must be trusted by the client. For self signed certificates and certificates with a private root CA, additional actions are required to create the trust. When using a self signed certificate export the certificate from the keystore ${JAVA_HOME}/bin/keytool -exportcert -alias events-svc -keystore events-svc.jks -file events-svc Events Service Configuration - Server Certificate trust store The web server embedded in the Events Service validates the server certificate and requires the complete certificate chain to be available. The chain is used to verify the trust. Run the following command: ${JAVA_HOME}/bin/keytool -importcert -alias events-crt -keystore cacerts -file events-svc.crt or ${JAVA_HOME}/bin/keytool -importcert -alias events-crt -keystore cacerts -file root_ca.crt Restart the Events Service node. Reconfigure the Events Service node. EUM Server Certificate trust store C omplete the following steps to establish trust between the EUM Server and the Events Service. Run the following commands: cd <eum_install_dir> jre/bin/keytool -importcert -alias events-crt -keystore jre/lib/security/cacerts -file events-svc.crt or cd <eum_install_dir> jre/bin/keytool -importcert -alias events-crt -keystore jre/lib/security/cacerts -file root_ca.crt Restart the EUM processor. Reconfigure the EUM processor. Controller Configuration - Server Certificate trust store Complete the following steps to establish trust between the Controller and the Events Service. Run the following commands: cd <controller_install_dir> jre8/bin/keytool -importcert -alias events-crt -keystore /appserver/glassfish/domains/domain1/config/cacerts -file events-svc.crt or cd <controller_install_dir> jre8/bin/keytool -importcert -alias events-crt -keystore /appserver/glassfish/domains/domain1/config/cacerts -file root_ca.crt  Restart the Controller. Reconfigure the Controller. Analytics Agent Configuration - Server Certificate trust store There are two ways to establish trust between the Analytics Agent and the Events Service. Import the certificate by either: -Using the JVM certs store, of the machine agent or standalone analytics agent, or, -Using the cacerts.jks file of the machine agent. This might require additional validation as the Analytics Agents runs as an independent processor. Reconfigure the analytics agent. Restart the analytics agent. Using a Load Balancer These steps are specific to NGINX. Traffic from the Controller and Analytics Agent to NGINX (from controller/agent) is over http, but traffic from NGINX to Events Service is over https Add following settings to nginx.conf file under http section: upstream eventsvc { server <api-store-1>[:<port>]; server <api-store-2>[:<port>]; ... } server { location / { proxy_pass https://eventsvc; } Restart NGINX. Enable Encryption Across Nodes Elasticsearch does not have built in support for inter-node communication over SSL, but this can be achieved using the SearchGuard plugin. SearchGuard uses relative paths for keystore and truststore, so copies of them need to be placed in the specified directory paths for SearchGuard to find the keystore and truststore. Releases 4.4.3 and Later Navigate to the events-service base directory (heretoafter <install-dir>) Edit the existing properties file: <install-dir>/conf/events-service-api-store.properties - all SearchGuard properties are exposed in this property file. Note, in the properties below, the keyStoreAlias and trustStoreAlias properties need to have some value. If there is no alias in use, simply pass an empty string: "".  # If set to true then TLS will be enabled ad.es.searchguard.ssl.enable=true # Relative path of keyStore to ${APPLICATION_HOME}/elasticsearch/config ad.es.searchguard.ssl.transport.keyStoreFilePath=eventsservice_keystore.jks ad.es.searchguard.ssl.transport.keyStorePassword=changeit ad.es.searchguard.ssl.transport.keyStoreAlias=eventsservice # Relative path of trustStore to ${APPLICATION_HOME}/elasticsearch/config ad.es.searchguard.ssl.transport.trustStoreFilePath=eventsservice_truststore.jks ad.es.searchguard.ssl.transport.trustStorePassword=changeit ad.es.searchguard.ssl.transport.trustStoreAlias="" ad.es.searchguard.ssl.transport.enforceHostnameVerification=false ad.es.searchguard.ssl.transport.resolveHostname=false Copy the entire SearchGuard plugin directory from the archive attached to this FEZ, found in the search-guard-ssl directory, to BOTH <install-dir>/plugins and <install-dir>/elasticsearch/plugins Example: cp -r /tmp/events-service-search-guard-plugin/search-guard-ssl <install-dir>/events-service/processor/plugins/ cp -r /tmp/events-service-search-guard-plugin/search-guard-ssl <install-dir>/events-service/processor/elasticsearch/plugins/ Create the directory <install-dir>/events-service/data/config if it does not already exist. Place the certificate keystore and truststore (eventsservice_keystore.jks and eventsservice_truststore.jks in the above configuration) to BOTH <install-dir>/events-service/data/config and <install-dir>/elasticsearch/config . If your certificate do not have SAN (Subject Alternative Name) for all of the nodes that make up the cluster, than ensure that each of the nodes have its own certificate in its keystore, and truststore has the certificate chain for the certificate of other nodes. The above steps must be followed on all nodes in the events-service cluster, using a different node certificate for each node. Restart all nodes to load the plugin:  ./bin/events-service.sh stop; ./bin/events-service.sh start -p conf/events-service-api-store.properties & Make any necessary modifications on the AppDynamics Controller(s), EUM Server(s), Analytics Agent(s) to update connectivity to be over HTTPS and also the   truststore   updates. Releases 4.4.2 and Earlier Copy the entire SearchGuard plugin directory from the archive attached to this FEZ, found in the search-guard-ssl directory, to  <install-dir>/plugins Example: cp -r /tmp/events-service-search-guard-plugin/search-guard-ssl <install-dir>/events-service/processor/plugins/ Backup the  <install-dir>/conf/events-service-api-store.yml  file and copy and adjust below section in the  <install-dir>/conf/events-service-api-store.yml   file: Find the section:   - className:com.appdynamics.analytics.processor.elasticsearch.node.single.ElasticSearchSingleNodeModule  and then find the  nodeSettings:   properties section nested in this  ElasticSearchSingleNodeModule   section. Add the following at the end of this section, right before the  systemSettings:   properties section begins. Change the highlighted values for your environment:                ########## BEGIN SearchGuard Cluster SSL Configuration ##########               #SearchGuard Configuration               # Enable or disable node-to-node ssl encryption (default: true)               searchguard.ssl.transport.enabled: true               # JKS or PKCS12 (default: JKS)               searchguard.ssl.transport.keystore_type: JKS               # Relative path to the keystore file (mandatory, this stores the server certificates), must be placed under the config/ dir               searchguard.ssl.transport.keystore_filepath: eventsservice_keystore.jks               # Keystore password (default: changeit) in the order of above               searchguard.ssl.transport.keystore_password: changeit               # Alias name (default: first alias which could be found)               searchguard.ssl.transport.keystore_alias: eventsservice               # JKS or PKCS12 (default: JKS)               searchguard.ssl.transport.truststore_type: JKS               # Relative path to the truststore file (mandatory, this stores the client/root certificates), must be placed under the config/ dir searchguard.ssl.transport.truststore_filepath: eventsservice_truststore.jks               # Alias name (default: trust all aliases)               #searchguard.ssl.transport.truststore_alias: my_alias               # Truststore password (default: changeit)               searchguard.ssl.transport.truststore_password: changeit              ########## END SearchGuard Cluster SSL Configuration ########## Create directory   <install-dir>/events-service/data/config   if it does not already exist. Place the certificate   keystore   and   truststore  (eventsservice_keystore.jks and eventsservice_truststore.jks in the above configuration)  in the above directory. If your certificate do not have SAN (Subject Alternative Name) for all of the nodes that make up the cluster, than ensure that each of the nodes have its own certificate in its keystore, and truststore has the certificate chain for the certificate of other nodes. Make same changes on all of the cluster nodes and restart. Validate afterwards that SSL is enabled without issues by searching for SearchGuard in the logs files. Also validate the Platform Admin can still manage the nodes and see the cluster healthy. Make any necessary modifications on the AppDynamics Controller(s), EUM Server(s), Analytics Agent(s) to update connectivity to be over HTTPS and also the   truststore   updates. If one or more nodes do not start properly, it's almost certainly an issue with the TLS certificates. We can help troubleshoot these issues. ... View more

Due to an Adobe Flash bug, your Safari browser will hang when you try to export a PDF of an On Demand Report.    You can perform a workaround by disabling Safe Mode for your Adobe Flash Player plugin. Navigate to your Preferences on your Safari browser. Open the Security tab. Click Plugin Settings. While holding the option key on your keyboard, click the applicable AppDynamics website name. In the dialog box that opens, uncheck Run in Safe Mode. Restart your browser and try exporting again.    Reference:   https://forums.adobe.com/thread/2007620 ... View more

Question How do you change the Controller username for a Controller install for Linux on-premise? Answer To change the AppDynamics username complete the following steps: Ensure that all AppDynamics processes are stopped. Obtain the username and group name for the AppDynamics install. Edit in the AppDynamics username into the db.cnf: 'user' option. Run the following commands: cd <controller install>; chown -R <AppD username>:<AppD groupname> Restart AppDynamics processes. If the HA Toolkit is installed, then install the Linux services as root: cd HA; ./install-init.sh -s OR ./install-init.sh -c # to match earlier choice Note that changing the install directory is basically a re-install. ... View more