Cloudera Hadoop

[CommunityUser]

Hi,

Do we have plugins available for Cloudera installation of Hadoop?

There is a good amount of documentation for Ambari installation but can't seem to find any for Cloudera.

Our Hadoop ecosystem consists of Sqoop, HDFS, Cloudera UI, Hive, HBase, Drool, Spark & Tableau.

Many Thanks in advance,

Deepak

[Rajesh_Putta · Accepted solution]

Hi,

As such there is no documentation available.

But you could hook java agent to each and every java process in Hadoop eco system and that should instrument those JVM process and cpature performance metrics out of it

Please find docs link here

https://docs.appdynamics.com/display/PRO43/Install+the+Java+Agent

https://docs.appdynamics.com/display/PRO43/Agent+Installation+by+Java+Framework

https://docs.appdynamics.com/display/PRO43/Instrument+JVMs+in+a+Dynamic+Environment

https://docs.appdynamics.com/display/PRO43/Use+System+Properties+for+Java+Agent+Settings

https://docs.appdynamics.com/display/PRO43/Use+Environment+Variables+for+Java+Agent+Settings

https://docs.appdynamics.com/display/PRO43/Instrument+JVMs+Started+by+Batch+or+Cron+Jobs

https://docs.appdynamics.com/display/PRO43/Instrument+JVMs+in+Restricted+Environments

https://docs.appdynamics.com/display/PRO43/Automate+Java+Agent+Deployment

https://docs.appdynamics.com/display/PRO43/Instrument+Java+System+Classes

We do also have AppDynamics extensions which using JMX or REST API capture performance metrics out of hadoop ecosystem and capture performance metrics with out instrumenting code deployed on those daemon processes

https://www.appdynamics.com/community/exchange/extension/hadoop-monitoring-extension/

Zookeper extension

https://www.appdynamics.com/community/exchange/extension/zookeeper-monitoring-extension/

Please also look for other extensions here

https://www.appdynamics.com/community/exchange/

Thanks

Rajesh

[CommunityUser]

Hello, Here is the Solution for your question go through it! 

install the bare cluster using Vagrant, and install the Hadoop stack using the Cloudera Manager. So I created a simple Vagrant setup myself. You can find it here.

Setting up the Virtual Machines
As per the instructions from the Gitub repo:

Depending on the hardware of your computer, installation will probably take between 15 and 25 minutes.

First install VirtualBox and Vagrant.

Hope this helps you! 

Thanks

Sarahjohn

[Rajesh_Putta]

Hi Deepak,

Are you looking for AppDynamics java agent installation for Cloudera distribution of Hadoop? or a general plugins for Cloudera installation of Hadoop. Please confirm. Post that we could assist you accordingly.

Thanks

Rajesh

[CommunityUser]

Classification: Public

Hi Rajesh,



We are looking to monitor and diagnose the various components, highlighted before, as part of our performance testing.

If we do the java agent installation for Cloudera distribution of Hadoop, would we be able to monitor the health for all the components in the ecosystem?

What suits this ecosystem best? Do we have individual plugin for all these components?



Regards

Deepak

[Rajesh_Putta · Accepted solution]

Hi,

As such there is no documentation available.

But you could hook java agent to each and every java process in Hadoop eco system and that should instrument those JVM process and cpature performance metrics out of it

Please find docs link here

https://docs.appdynamics.com/display/PRO43/Install+the+Java+Agent

https://docs.appdynamics.com/display/PRO43/Agent+Installation+by+Java+Framework

https://docs.appdynamics.com/display/PRO43/Instrument+JVMs+in+a+Dynamic+Environment

https://docs.appdynamics.com/display/PRO43/Use+System+Properties+for+Java+Agent+Settings

https://docs.appdynamics.com/display/PRO43/Use+Environment+Variables+for+Java+Agent+Settings

https://docs.appdynamics.com/display/PRO43/Instrument+JVMs+Started+by+Batch+or+Cron+Jobs

https://docs.appdynamics.com/display/PRO43/Instrument+JVMs+in+Restricted+Environments

https://docs.appdynamics.com/display/PRO43/Automate+Java+Agent+Deployment

https://docs.appdynamics.com/display/PRO43/Instrument+Java+System+Classes

We do also have AppDynamics extensions which using JMX or REST API capture performance metrics out of hadoop ecosystem and capture performance metrics with out instrumenting code deployed on those daemon processes

https://www.appdynamics.com/community/exchange/extension/hadoop-monitoring-extension/

Zookeper extension

https://www.appdynamics.com/community/exchange/extension/zookeeper-monitoring-extension/

Please also look for other extensions here

https://www.appdynamics.com/community/exchange/

Thanks

Rajesh

[CommunityUser]

Classification: Public

Many Thanks Rajesh!



Do we need to buy separate licences for the Hadoop extensions or are these part of the enterprise license, what we already hold?



Thanks

Deepak

[Jean-Baptiste_M]

Extension found the community site are free. You need to use a machine
agent deployed on a server with an APM agent already deployed (Java, .Net,
PHP etc).
If you instrument with the Java agents then you need to have free licenses
for that, as regular Java applications.

Cheers
JB

[Rajesh_Putta]

Hi Deepak,

Thanks for the update. If you got machine agent licenses as a part of your trial licences, then that should be sufficient for you to monitor your Hadoop environment using extensions, as extension works with machine agent.

Please let us know if you got any more questions. If not, can we close this case.

Thanks

Rajesh

[CommunityUser · Accepted solution]

Yes Rajesh!!



You may close the case!



If we encounter any problem during set up, we'll reach out to you.



Many Thanks

Deepak

[Rajesh_Putta]

Thanks Deepak. Closing this case as per your update. Feel free to log new cases for any further help.

Thanks

Rajesh