cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Problem using OAUTH API connecting to Elastic Logstash http_poller plugin to get oauth token

Good Morning,

 

I am using the http_poller logstash input filter to connect to the AppDynamics OAUTH API that I will then use to pass to the http filter plugin to retrieve AppDynamics REST API metric data to insert into Elasticsearch.

My logstash http_poller configuration is:

 

```
input {
http_poller {
urls => {
AppDynamics => {
method => post
url => "https://*appdynamics-test-account*/controller/api/oauth/access_token"
headers => {
"Accept" => "application/json"
"Content-Type" => "application/vnd.appd.cntrl+protobuf;v=1"
"Authorizations" => "*AppDynamics bearer token*"
"grant_type" => "client_credentials"
"client_id" => "Stage-CurlTest"
"client_secret" => "*AppDynamics Client Secret*"
}
}
}
request_timeout => 60
schedule => { every => "1h"}
codec => "json"
metadata_target => "appd-token"
type => "AppDynamics"
}
}
}
```

I am getting the following error response when the poller tries to connect to AppDynamics

```
{
"type": "AppDynamics",
"@timestamp": "2021-10-18T14:02:11.191Z",
"appd-token": {
"request": {
"method": "post",
"url": "https://*appdynamics-test-account*/controller/api/oauth/access_token",
"headers": {
"Authorizations": "AppDynamics Bearer Token",
"grant_type": "client_credentials",
"client_secret": "`AppDynamics Client Secret`",
"client_id": "Stage-CurlTest",
"Content-Type": "application/vnd.appd.cntrl+protobuf;v=1",
"Accept": "application/json"
}
},
"response_headers": {
"x-content-type-options": "nosniff",
"x-frame-options": "SAMEORIGIN",
"x-xss-protection": "1; mode=block",
"connection": "keep-alive",
"server": "AppDynamics",
"date": "Mon, 18 Oct 2021 14:02:10 GMT"
},
"runtime_seconds": 0.36,
"host": "SAPPLOG03",
"name": "AppDynamics",
"response_message": "Not Acceptable",
"code": 406,
"times_retried": 0
},
"@version": "1",
"tags": [
"_httprequestfailure"
]
}
```

New to both the http_poller filter and the AppDynamics Metric REST API so not sure what I have configured wrong. I have also posted to the Elastic Community forum.

 

I have removed any account-specific information for security reasons so any additional information that would be helpful let me know and I'll get it for this post.

 

Thanks,
Bill

12 REPLIES 12

Hiroki.Ito
AppDynamics Team

Thank you for posting to the community.

First, could you please try curl command below to make sure your apiClientName and client_secret are correct?
curl -X POST -H “Content-Type: application/vnd.appd.cntrl+protobuf;v=1” “https://<controller address>/controller/api/oauth/access_token” -d ‘grant_type=client_credentials&client_id=<apiClientName>@<accountName>&client_secret=*AppDynamics Client Secret*"’

The curl command should return access_token.

Then, in your http_poller input, please include the header and body just like the curl command.
Currently, it includes grant_type, client_id, and client_secret in the headers, so they should be in the body.
In addition, Accept and Authorizations headers are not necessary for the REST API that generates the token.
Authorization header is necessary for other REST API that uses the generated token.
e.g.
 urls => {
AppDynamics => {
method => post
url => "https://*appdynamics-test-account*/controller/api/oauth/access_token"
body => "grant_type=client_credentials&client_id=<apiClientName>@<accountName>&client_secret=*AppDynamics Client Secret*"
headers => {
"Content-Type" => "application/vnd.appd.cntrl+protobuf;v=1"
}
}
}

Best Regards,
Hiroki

Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Thanks Hiroki,

 

That worked and I am getting an access token back but I am still getting an http_request failure even though the response status is 'OK'

 

{
"tags": [
"_httprequestfailure"
],
"@timestamp": "2021-10-20T13:39:44.541Z",
"type": "AppDynamics",
"@version": "1",
"appd-token": {
"code": 200,
"request": {
"body": "grant_type=client_credentials&client_id=xxxx@vertex-test&client_secret=xxxx",
"headers": {
"Content-Type": "application/vnd.appd.cntrl+protobuf;v=1"
},
"method": "post",
"url": "https://xxxx/controller/api/oauth/access_token"
},
"name": "AppDynamics",
"times_retried": 0,
"response_headers": {
"x-frame-options": "SAMEORIGIN",
"content-type": "application/vnd.appd.cntrl+json;v=1",
"date": "Wed, 20 Oct 2021 13:39:44 GMT",
"server": "AppDynamics",
"x-content-type-options": "nosniff",
"connection": "keep-alive",
"x-xss-protection": "1; mode=block"
},
"runtime_seconds": 0.935,
"host": "SAPPLOG03",
"response_message": "OK"
},
"expires_in": 3600,
"access_token": "xxxxx"
}

 

The filter body of my logstash.conf file looks like this

 

filter {
if ([type] == "AppDynamics") {
http {
body_format => "json"
follow_redirects => false
body => {
"attribute" => "%{[attribute_value]}"
}
url => "https://xxxx.appdynamics.com/controller/rest/applications/Vertex%20Cloud%20%28Stage%29/metric-data"
verb => "GET"
headers => [ "Authorization", "Bearer xxxx" ]
target_body => "[@metadata][api_response]"
target_headers => "[@metadata][api_headers]"
}
}
}

 

The output section looks like this (which is where the error is being trapped)

 

 if ([type] == "AppDynamics") {
if [http_request_failure] or [http_poller_metadata][code] != 200 {
file {
path => "D:\ELK\Logstash\logstash-7.14.0\logs\AppDynamics_Error.txt"
}
}
else {
file {
path => "D:\ELK\Logstash\logstash-7.14.0\logs\AppDynamics.txt"
}
}
}

 

My end goal is to get the data from the metrics api into an Elasticsearch index for persistence purposes.

 

Thanks


Thank you for your reply.
I think you are getting error with metric-data API.

As written in the document below, you need to specify metric-path, time-range-type, and duration-in-mins/start-time/end-time in the url of metric data API.

Please try with curl command first, and I think you can configure logstash.conf just like the curl command.
body_format and body may not be necessary, and you need to include the necessary parameters in the url.

e.g. of curl command
curl -H "Authorization:Bearer <access_token>" "http://demo.appdynamics.com/controller/rest/applications/ECommerce_E2E/metric-data?metric-path=Overall%20Application%20Performance%7CAverage%20Response%20Time%20%28ms%29&time-range-type=BEFORE_NOW&duration-in-mins=15"

Best Regards,
Hiroki

Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Hey @Hiroki.Ito 

 

So I was able to get the http_poller working and returning an access token and I changed my http filter to--

 

http {
body_format => "json"
follow_redirects => false
url => REST API url taken from one of my metrics
verb => "GET"
headers => [ "Authorization", "Bearer %{appd-token}" ]
target_body => "[@metadata][api_response]"
target_headers => "[@metadata][api_headers]"
}

 

'appd-token' is from the metadata_target => "appd-token" property from the http_poller plugin.

 

That is giving me "response=>"Failed to authenticate: invalid access token." message but I can take the same metrics rest url and run it with curl with no problems getting a json response for the metric that I am querying.

 

The documentation that I am using here is coming from a post in the Elastic community

 

https://discuss.elastic.co/t/http-poller-rest-api-authentication-token/246870/5

 

Thanks,

Bill


Thank you for confirming the curl command for metric data API and changing url in the http filter.
Could you please confirm if %{appd-token} is actually taking the access_token? What happens if it is replaced by %{access_token}?
It might be possible to debug by configuring the url as %{appd-token} or %{access_token} and check the error message in the log to see if the url is replaced by actual access token.

Regarding logstash configuration, I can't help much.
However, I could get Average Response time of a BT from metric data API in my local environment, so I am attaching this for reference.
I hope it helps.

input {
http_poller {
urls => {
AppD => {
method => post
url => "https://<controller_url>/controller/api/oauth/access_token"
body => "grant_type=client_credentials&client_id=<apiClientName>@<accountName>&client_secret=<appdynamics_client_secret>"
headers => {
"Content-Type" => "application/vnd.appd.cntrl+protobuf;v=1"
}
}
}
request_timeout => 60
schedule => { every => "1m"}
codec => "json"
}
}

filter{
http {
url => "https://<controller_url>/controller/rest/applications/<application_id>/metric-data?metric-path=Business%20Transaction%20Performance%7CBusiness%20Transactions%7CTier%7C/BT%7CAverage%20Response%20Time%20%28ms%29&time-range-type=BEFORE_NOW&output=JSON&duration-in-mins=60"
verb => "GET"
headers => [ "Authorization", "Bearer %{access_token}" ]
}
}

Best Regards,
Hiroki

Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Hiroki,

 

I was able to get this solution working. I had my headers field incorrectly configured in the http filter it should have been 

 

headers => {
"Authorization" => "Bearer %{access_token}"
}

 

Then I removed the target_response & target_header fields and just let the response data go to the default 'body' field and now everything is working great.

 

Thanks for all of your help,

Bill

Hello @Hiroki.Ito 

I'm trying to get the API to work with postman or curl. I am not a software developer. I do not know much. But when I try to try the curl you sent, I get an error. Please help.

 

If I have other questions about api, can I write to you?

 

 

1 - Request cmd  :

curl -X POST -H “Content-Type: application/vnd.appd.cntrl+protobuf;v=1” “https://appd/controller/api/oauth/access_token” -d‘grant_type=client_credentials&client_id=Hyetest@customer1&client_secret=36fac97b-xxx-416f-ac4a-d72766d4c073"’

Grant type value not specified.'client_id' is not recognized as an internal or external command,
operable program or batch file.
'client_secret' is not recognized as an internal or external command,
operable program or batch file.

 

 

2 - Postman Request :

curl -X POST \
https://appd/controller/api/oauth/access_token \
-H 'authorization: Basic aGFsZXJvbEBjdXN0b21lcjE6Q3liZXJlcmVuNzI5Nw==' \
-H 'cache-control: no-cache' \
-H 'content-type: application/x-www-form-urlencoded' \
-d grant_type=36fac97b-1930-xxx-ac4a-d72799d4c073

Response : 

No enum constant com.singularity.ee.controller.servlet.api.OAuthAccessTokenResource$GrantType.36FAC97B-xxx-416F-AC4A-D72766D4C073curl: (6) Could not resolve host: \
curl: (6) Could not resolve host: Basic
curl: (6) Could not resolve host: aGFsZXJvbEBjdXN0b21lcjE6Q3liZXJlcmVuNzI5Nw=='
curl: (6) Could not resolve host: \
curl: (6) Could not resolve host: no-cache'
curl: (6) Could not resolve host: \
curl: (6) Could not resolve host: application
curl: (6) Could not resolve host: \

 

 

https://docs.appdynamics.com/4.5.x/en/extend-appdynamics/appdynamics-apis/api-clients

 

Hi @apm.turkey ,

 

Thank you for reaching out to us.

For curl command to get the token, could you please try command like below?
curl -X POST -H "Content-Type: application/vnd.appd.cntrl+protobuf;v=1" "https://<controller address>/controller/api/oauth/access_token" -d "grant_type=client_credentials" -d "client_id=<apiClientName>@<accountName>" -d "client_secret=<AppDynamics Client Secret>"

 

I am also attaching screenshots to get access token using Postman.

Postman_Header.png

Postman_Body.png

This will return access_token that can be used for other AppDynamics Controller REST APIs.

 

e.g. Requesting applications API
curl -H "Authorization:Bearer <access_token>" https://<controller address>/controller/rest/applications

 

If you have other questions about API, could you please raise another post?
This post is for using API with Elastic Logstash http_poller plugin.
 
Best Regards,
Hiroki

Found something helpful? Click the Accept as Solution button to help others find answers faster.
Liked something? Click the Thumbs Up button.

Thank you so much !

Can I send a private question about my other api questions?