TIBCO BusinessWorks 6.x integrates with different TIBCO and non-TIBCO systems like databases, TIBCO EMS Server, SOAP Service over HTTP/JMS, and TIBCO messaging frameworks like FTL.
AppDynamics supports JMS, SOAP, JDBC and HTTP backend detection from TIBCO BusinessWorks out-of-the-box. However, non-standard and proprietary backend calls to TIBCO FTL are not detected by the AppDynamics Java Agent out-of-the-box. Backend instrumentation and custom-activity-correlation are needed to capture the end-to-end flow between upstream and downstream tiers in a single Business Transaction.
Below is a solution that we successfully tested and implemented with a customer to instrument cross-tier FTL communication and correlate exit calls in TIBCO BusinessWorks 6.x.
For testing purposes, we used:
We had three TIBCO BusinessWorks Applications: PT, BT and TT. Each JVM used a unique TierName and all the three JVMs reported to the same application in the AppDynamics Controller.
PT is a simple REST Service implemented with TIBCO BusinessWorks 6.x that accepts API requests at http://localhost:8080. It sends an FTL request to BT and receives a FTL reply from BT using FTLRequestReply activity. It then provides the reply back to the end user.
BT uses a FTLSubscriber activity to receive the FTL request from PT, uses FTLRequestReply activity to send another request to TT and receive reply from TT, then replies back to BT using FTLReply activity.
TT uses a FTLSubscriber activity to receive the FTL request from BT, makes an outgoing HTTP call using SendHTTPRequest activity, gets back a response and then replies back to BT using FTLReply activity.
singularityheaderin both the FTL request and reply messages in order to preserve the cross-tier correlation. However, since FTL is a non-standard call, they would need to identify the proper place/method to inject/extract the
singularityheaderin the FTL message.
These 8 steps will guide you through how to instrument cross-tier FTL communication and correlation in TIBCO BusinessWorks 6.x.
custom-activity-correlation.xmlfor all three JVMs (open the attached file named “Step 1 - custom-activity-correlation.xml.txt”). There are two producer and two consumer entries here. For this specific customer, we used three FTL activities:
custom-activity-correlation.xmlfile, the first producer and the second consumer are from this activity.
custom-activity-correlation.xmlfile, the second producer is from this activity.
custom-activity-correlation.xmlfile, the first consumer is from this activity.
singularityheaderin both producers and reading the
singularityheaderin both consumers in the respective FTL calls invoked by these activities. If there is no FTLRequestReply and FTLReply activity involved in the BW code (no request-reply scenario) and you just have FTLPublisher and FTLSubscriber activities (only send-receive scenario), you can use the
custom-activity-correlation.xmlin the attached file named “Step 1 - custom-activity-correlation.xml (only send-receive scenario).”
singularityheaderadded to it. You have to do this code change to make sure the application can understand the presence of this new header element in the FTL message and successfully parse this.
NOTE: Make sure that this element is added at the end of your other existing elements in the FTL message schema or the application will fail to parse the FTL message with the
fork-configsection for all your FTL nodes. (Note: The field scrolls horizontally)
<include filter-type="EQUALS" filter-value="org.glassfish.jersey.server.ServerRuntime$AsyncResponder$2"/>
error-safety-rule-error-threshold=-1for all nodes that are sending/receiving FTL messages.
NOTE: The custom correlation is applied on the nodes about 15 minutes after the application starts. If you start putting load on the application as soon as the nodes start, the
singularityheaderwill not be recognized and it will trigger an error in the Business Transaction. Eventually, the instrumentation will be neutralized and the
singularityheaderwill never be set. You need to add the above node property to make sure that the instrumentation is not neutralized during the first 15 minutes of the application startup when there is no
singularityheaderset in the FTL messages.
track-async-inside-custom-exit=truefor all nodes sending/receiving FTL messages. This node property is used to track async calls inside the custom exit points.
ignore-exitcall-on-reentry=falsefor all nodes sending/receiving FTL messages. This will ensure that your custom exit calls are not lost due to SpringBeans instrumentation. Otherwise, you can simply disable SpringBeans instrumentation for BTs and SEPs on all such nodes.
Below are examples of Controller dashboards that display the complete end-to-end logic after following the steps above.