cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Not a customer? Start a free trial

Click the Start a free trial link to start a 15-day SaaS trial of our product and join our community as a trial user. If you are an existing customer do not start a free trial.

AppDynamics customers and established members should click the sign in button to authenticate.

Controller (SaaS, On Premises)

Problem using OAUTH API connecting to Elastic Logstash http_poller plugin to get oauth token

SOLVED

Problem using OAUTH API connecting to Elastic Logstash http_poller plugin to get oauth token

Good Morning,

 

I am using the http_poller logstash input filter to connect to the AppDynamics OAUTH API that I will then use to pass to the http filter plugin to retrieve AppDynamics REST API metric data to insert into Elasticsearch.

My logstash http_poller configuration is:

 

```
input {
http_poller {
urls => {
AppDynamics => {
method => post
url => "https://*appdynamics-test-account*/controller/api/oauth/access_token"
headers => {
"Accept" => "application/json"
"Content-Type" => "application/vnd.appd.cntrl+protobuf;v=1"
"Authorizations" => "*AppDynamics bearer token*"
"grant_type" => "client_credentials"
"client_id" => "Stage-CurlTest"
"client_secret" => "*AppDynamics Client Secret*"
}
}
}
request_timeout => 60
schedule => { every => "1h"}
codec => "json"
metadata_target => "appd-token"
type => "AppDynamics"
}
}
}
```

I am getting the following error response when the poller tries to connect to AppDynamics

```
{
"type": "AppDynamics",
"@timestamp": "2021-10-18T14:02:11.191Z",
"appd-token": {
"request": {
"method": "post",
"url": "https://*appdynamics-test-account*/controller/api/oauth/access_token",
"headers": {
"Authorizations": "AppDynamics Bearer Token",
"grant_type": "client_credentials",
"client_secret": "`AppDynamics Client Secret`",
"client_id": "Stage-CurlTest",
"Content-Type": "application/vnd.appd.cntrl+protobuf;v=1",
"Accept": "application/json"
}
},
"response_headers": {
"x-content-type-options": "nosniff",
"x-frame-options": "SAMEORIGIN",
"x-xss-protection": "1; mode=block",
"connection": "keep-alive",
"server": "AppDynamics",
"date": "Mon, 18 Oct 2021 14:02:10 GMT"
},
"runtime_seconds": 0.36,
"host": "SAPPLOG03",
"name": "AppDynamics",
"response_message": "Not Acceptable",
"code": 406,
"times_retried": 0
},
"@version": "1",
"tags": [
"_httprequestfailure"
]
}
```

New to both the http_poller filter and the AppDynamics Metric REST API so not sure what I have configured wrong. I have also posted to the Elastic Community forum.

 

I have removed any account-specific information for security reasons so any additional information that would be helpful let me know and I'll get it for this post.

 

Thanks,
Bill

By replying you agree to the Terms and Conditions of the AppDynamics Community.
Problem using OAUTH API connecting to Elastic Logstash http_poller plugin to get oauth token
6 REPLIES 6
Hiroki.Ito
AppDynamics Team


Thank you for posting to the community.

First, could you please try curl command below to make sure your apiClientName and client_secret are correct?
curl -X POST -H “Content-Type: application/vnd.appd.cntrl+protobuf;v=1” “https://<controller address>/controller/api/oauth/access_token” -d ‘grant_type=client_credentials&client_id=<apiClientName>@<accountName>&client_secret=*AppDynamics Client Secret*"’

The curl command should return access_token.

Then, in your http_poller input, please include the header and body just like the curl command.
Currently, it includes grant_type, client_id, and client_secret in the headers, so they should be in the body.
In addition, Accept and Authorizations headers are not necessary for the REST API that generates the token.
Authorization header is necessary for other REST API that uses the generated token.
e.g.
 urls => {
AppDynamics => {
method => post
url => "https://*appdynamics-test-account*/controller/api/oauth/access_token"
body => "grant_type=client_credentials&client_id=<apiClientName>@<accountName>&client_secret=*AppDynamics Client Secret*"
headers => {
"Content-Type" => "application/vnd.appd.cntrl+protobuf;v=1"
}
}
}

Best Regards,
Hiroki

Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Thanks Hiroki,

 

That worked and I am getting an access token back but I am still getting an http_request failure even though the response status is 'OK'

 

{
"tags": [
"_httprequestfailure"
],
"@timestamp": "2021-10-20T13:39:44.541Z",
"type": "AppDynamics",
"@version": "1",
"appd-token": {
"code": 200,
"request": {
"body": "grant_type=client_credentials&client_id=xxxx@vertex-test&client_secret=xxxx",
"headers": {
"Content-Type": "application/vnd.appd.cntrl+protobuf;v=1"
},
"method": "post",
"url": "https://xxxx/controller/api/oauth/access_token"
},
"name": "AppDynamics",
"times_retried": 0,
"response_headers": {
"x-frame-options": "SAMEORIGIN",
"content-type": "application/vnd.appd.cntrl+json;v=1",
"date": "Wed, 20 Oct 2021 13:39:44 GMT",
"server": "AppDynamics",
"x-content-type-options": "nosniff",
"connection": "keep-alive",
"x-xss-protection": "1; mode=block"
},
"runtime_seconds": 0.935,
"host": "SAPPLOG03",
"response_message": "OK"
},
"expires_in": 3600,
"access_token": "xxxxx"
}

 

The filter body of my logstash.conf file looks like this

 

filter {
if ([type] == "AppDynamics") {
http {
body_format => "json"
follow_redirects => false
body => {
"attribute" => "%{[attribute_value]}"
}
url => "https://xxxx.appdynamics.com/controller/rest/applications/Vertex%20Cloud%20%28Stage%29/metric-data"
verb => "GET"
headers => [ "Authorization", "Bearer xxxx" ]
target_body => "[@metadata][api_response]"
target_headers => "[@metadata][api_headers]"
}
}
}

 

The output section looks like this (which is where the error is being trapped)

 

 if ([type] == "AppDynamics") {
if [http_request_failure] or [http_poller_metadata][code] != 200 {
file {
path => "D:\ELK\Logstash\logstash-7.14.0\logs\AppDynamics_Error.txt"
}
}
else {
file {
path => "D:\ELK\Logstash\logstash-7.14.0\logs\AppDynamics.txt"
}
}
}

 

My end goal is to get the data from the metrics api into an Elasticsearch index for persistence purposes.

 

Thanks

Hiroki.Ito
AppDynamics Team


Thank you for your reply.
I think you are getting error with metric-data API.

As written in the document below, you need to specify metric-path, time-range-type, and duration-in-mins/start-time/end-time in the url of metric data API.

Please try with curl command first, and I think you can configure logstash.conf just like the curl command.
body_format and body may not be necessary, and you need to include the necessary parameters in the url.

e.g. of curl command
curl -H "Authorization:Bearer <access_token>" "http://demo.appdynamics.com/controller/rest/applications/ECommerce_E2E/metric-data?metric-path=Overall%20Application%20Performance%7CAverage%20Response%20Time%20%28ms%29&time-range-type=BEFORE_NOW&duration-in-mins=15"

Best Regards,
Hiroki

Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Hey @Hiroki.Ito 

 

So I was able to get the http_poller working and returning an access token and I changed my http filter to--

 

http {
body_format => "json"
follow_redirects => false
url => REST API url taken from one of my metrics
verb => "GET"
headers => [ "Authorization", "Bearer %{appd-token}" ]
target_body => "[@metadata][api_response]"
target_headers => "[@metadata][api_headers]"
}

 

'appd-token' is from the metadata_target => "appd-token" property from the http_poller plugin.

 

That is giving me "response=>"Failed to authenticate: invalid access token." message but I can take the same metrics rest url and run it with curl with no problems getting a json response for the metric that I am querying.

 

The documentation that I am using here is coming from a post in the Elastic community

 

https://discuss.elastic.co/t/http-poller-rest-api-authentication-token/246870/5

 

Thanks,

Bill

Hiroki.Ito
AppDynamics Team


Thank you for confirming the curl command for metric data API and changing url in the http filter.
Could you please confirm if %{appd-token} is actually taking the access_token? What happens if it is replaced by %{access_token}?
It might be possible to debug by configuring the url as %{appd-token} or %{access_token} and check the error message in the log to see if the url is replaced by actual access token.

Regarding logstash configuration, I can't help much.
However, I could get Average Response time of a BT from metric data API in my local environment, so I am attaching this for reference.
I hope it helps.

input {
http_poller {
urls => {
AppD => {
method => post
url => "https://<controller_url>/controller/api/oauth/access_token"
body => "grant_type=client_credentials&client_id=<apiClientName>@<accountName>&client_secret=<appdynamics_client_secret>"
headers => {
"Content-Type" => "application/vnd.appd.cntrl+protobuf;v=1"
}
}
}
request_timeout => 60
schedule => { every => "1m"}
codec => "json"
}
}

filter{
http {
url => "https://<controller_url>/controller/rest/applications/<application_id>/metric-data?metric-path=Business%20Transaction%20Performance%7CBusiness%20Transactions%7CTier%7C/BT%7CAverage%20Response%20Time%20%28ms%29&time-range-type=BEFORE_NOW&output=JSON&duration-in-mins=60"
verb => "GET"
headers => [ "Authorization", "Bearer %{access_token}" ]
}
}

Best Regards,
Hiroki

Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Hiroki,

 

I was able to get this solution working. I had my headers field incorrectly configured in the http filter it should have been 

 

headers => {
"Authorization" => "Bearer %{access_token}"
}

 

Then I removed the target_response & target_header fields and just let the response data go to the default 'body' field and now everything is working great.

 

Thanks for all of your help,

Bill