cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

OpenTelemetry business transactions only show HTTP Post

Björn.Wenzel
Creator

For 3 Spring Boot applications (2 classic servlet and one spring cloud gateway in front) I have enabled OTEL and see the first metrics arriving successfully in my Controller.

Due to whatever reasons it seems that the Business Transactions only show that the app is NodeJS & the name of the Transaction is HTTP Post. I don't see the different routes that are available via the gateway & routed to the 2 classic spring boot applications.

For a short test I also reconfigured the collector to log every request instead of forwarding it to the controller, please see attached details about the request:

Resource attributes:
     -> appdynamics.controller.account: Str(acme)
     -> appdynamics.controller.host: Str(acme.saas.appdynamics.com)
     -> appdynamics.controller.port: Int(443)
     -> service.name: Str(gateway)
     -> service.namespace: Str(acme-namespace)
     -> telemetry.sdk.language: Str(java)
     -> telemetry.sdk.name: Str(opentelemetry)
     -> telemetry.sdk.version: Str(1.16.0)
ScopeSpans #0
ScopeSpans SchemaURL:
InstrumentationScope org.springframework.cloud.sleuth
Span #0
    Trace ID       : 5fea943a828ccd2fff3072992d4e5ab6
    Parent ID      :
    ID             : 5a8b847d55a33279
    Name           : HTTP GET
    Kind           : SPAN_KIND_SERVER
    Start time     : 2022-10-29 17:19:45.595667 +0000 UTC
    End time       : 2022-10-29 17:19:45.640038791 +0000 UTC
    Status code    : STATUS_CODE_ERROR
    Status message :
Attributes:
     -> http.host: Str(localhost:8881)
     -> http.method: Str(GET)
     -> http.path: Str(/api/acme/endpoint)
     -> http.route: Str(/api/acme/endpoint)
     -> http.scheme: Str(http)
     -> http.status_code: Int(500)
     -> http.target: Str(/api/acme/endpoint)
     -> http.user_agent: Str(curl/7.79.1)
     -> net.host.name: Str(localhost)
     -> net.transport: Str(http)
Events:
SpanEvent #0
     -> Name: exception
     -> Timestamp: 2022-10-29 17:19:45.637675125 +0000 UTC
     -> DroppedAttributesCount: 0
     -> Attributes::
          -> exception.message: Str(Connection refused: localhost/[0:0:0:0:0:0:0:1]:8884)
          -> exception.stacktrace: Str(io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/[0:0:0:0:0:0:0:1]:8884
	Suppressed: The stacktrace has been enhanced by Reactor, refer to additional information below:
Error has been observed at the following site(s):
	*__checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
	*__checkpoint ⇢ org.springframework.security.web.server.authorization.AuthorizationWebFilter [DefaultWebFilterChain]
	*__checkpoint ⇢ org.springframework.security.web.server.authorization.ExceptionTranslationWebFilter [DefaultWebFilterChain]
	*__checkpoint ⇢ org.springframework.security.web.server.authentication.logout.LogoutWebFilter [DefaultWebFilterChain]
	*__checkpoint ⇢ org.springframework.security.web.server.savedrequest.ServerRequestCacheWebFilter [DefaultWebFilterChain]
	*__checkpoint ⇢ org.springframework.security.web.server.context.SecurityContextServerWebExchangeWebFilter [DefaultWebFilterChain]
	*__checkpoint ⇢ org.springframework.security.web.server.authentication.AuthenticationWebFilter [DefaultWebFilterChain]
	*__checkpoint ⇢ org.springframework.security.web.server.context.ReactorContextWebFilter [DefaultWebFilterChain]
	*__checkpoint ⇢ org.springframework.security.web.server.header.HttpHeaderWriterWebFilter [DefaultWebFilterChain]
	*__checkpoint ⇢ org.springframework.security.config.web.server.ServerHttpSecurity$ServerWebExchangeReactorContextWebFilter [DefaultWebFilterChain]
	*__checkpoint ⇢ org.springframework.security.web.server.WebFilterChainProxy [DefaultWebFilterChain]
Original Stack Trace:
Caused by: java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.pollConnect(Native Method)
	at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
	at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
	at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337)
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
)
          -> exception.type: Str(io.netty.channel.AbstractChannel.AnnotatedConnectException)

Any idea?

1 REPLY 1

Satbir.Singh
AppDynamics Team

Hi,

It seems that the Business Transactions in the AppDynamics Controller are not able to differentiate between the different routes that are available via the gateway and routed to the 2 classic Spring Boot applications. This is likely because the routes are not being properly instrumented with OTEL.

To fix this issue, you should make sure that all the necessary libraries are being instrumented with OTEL and that the necessary spans are being created for each route. You may also need to check the configuration of the OTEL collector to ensure that it is properly forwarding all the data to the AppDynamics Controller.

Looking at the log you provided, it seems that there was an error connecting to a service running on port 8884. This could be related to the issue you are experiencing with the Business Transactions not being properly identified. You may need to investigate this error further and resolve it before the Business Transactions can be properly instrumented.


Thanks,
Satbir Singh



Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.