-
Hortonworks Ambari VM for HSP Deployment Guide
MK-95HSP017-01 1
Setup HSP Ambari Cluster
Prerequisites 1. An initialized HSP cluster running at least HSP
1.1.1. 2. Downloaded copy of the ISO containing the vm-template for
Ambari from HortonWorks. This
ISO is available from TISC, and may be called something like
HDP-2.3.4.7_Ambari-2.2.1.1_HSP-1.1.1.iso
3. Linux client. First, add the vm-template to the HSP
cluster.
1. Extract the qcow2 image from the ISO. On the linux client,
mount the ISO image as displayed below: linux# mount –t iso9660 –o
loop /tmp/HDP-2.3.4.7_Ambari-2.2.1.1_HSP-1.1.1.iso /mnt/iso
2. Mount a share from the HSP cluster on your Linux system. Copy
the qcow2 image from the ISO to the share: linux# mount IP:/myshare
/mnt/hsp linux# cp
/mnt/iso/HDP-2.3.4.7_Ambari-2.2.1.1_HSP-1.1.1.qcow2 /mnt/hsp Where
IP is the VIP of the HSP cluster, and /myshare is the name of the
available share.
3. On the HSP cluster, add the vm-template: admin@Node003:~$
hspadm vm-template add --name ambari_template1 --image-path
myshare:/HDP-2.3.4.7_Ambari-2.2.1.1_HSP-1.1.1.qcow2
--use-address-pool Y
If this was successful, you can verify: admin@Node003:~$ hspadm
vm-template list
--------------------------------------------------------------------------------------------------------------------------------------------------
| Name | Run State | Golden | VM Size | Memory Size | CPUs | Pool |
IP Address | MAC Address |
---------------------------------------------------------------------------------------------------------------------------------------------------
| ambari_template1 | DOWN | No | medium | 64.000 GiB | 6 | Yes | |
|
---------------------------------------------------------------------------------------------------------------------------------------------------
4. Now deploy the vm-template. In this example we are creating a
5 node Hadoop cluster.
admin@Node003:~$ hspadm vm-template deploy –name
ambari_template1 –num-instances 5 –instance-name ambaridev
If this was successful, you can verify: admin@Node001:~$ hspadm
vm-instance list --template ambari_template1
-------------------------------------------------------------------------------------------------------------------------------
| Name | Run State | Template | IP Address | MAC Address | Node
Name |
--------------------------------------------------------------------------------------------------------------------------------
| ambaridev-1 | UP | ambari_template1 | 10.129.71.21/24 |
52:54:00:5e:f3:76 | Node001 | | ambaridev-2 | UP | ambari_template1
| 10.129.71.22/24 | 52:54:00:0a:30:94 | Node004 | | ambaridev-3 |
UP | ambari_template1 | 10.129.71.23/24 | 52:54:00:75:06:47 |
Node002 | | ambaridev-4 | UP | ambari_template1 | 10.129.71.24/24 |
52:54:00:17:c1:7f | Node005 | | ambaridev-5 | UP | ambari_template1
| 10.129.71.25/24 | 52:54:00:ad:d2:eb | Node001 |
-------------------------------------------------------------------------------------------------------------------------------
-
Hortonworks Ambari VM for HSP Deployment Guide
MK-95HSP017-01 2
Setting up Ambari 1. Log into the first vm-instance. This
instance will become the Ambari server.
ssh 10.129.71.21 Username = root Password = smrace1
2. Run the install, setup and start of Ambari server. To do this
run /opt/escale/bin/ambariPrep.py. There are several options that
can be given, these can be seen by using the –h option. The
vm-instances are configured to inherit the domain name that is set
on the HSP cluster. Note that the script creates the file system
supplied via the –fsname option. The script will fail if the
filesystem was created prior. Example: [root@ambaridev-1 ~]#
/opt/escale/bin/ambariPrep.py --fsname ambaridevfs –domain
escale.bluearc.com --user admin –pw Upon completion of the
ambariPrep.py, ambari will have been installed, configured and
started. Please note that it can take Ambari server up to 3 minutes
to start responding on port 8080 for the GUI. You can follow the
progress on ambaridev-1 (i.e. the instance on which you ran
ambariPrep.py) by: a. cd /var/log/ambari-server b. tail –f
ambari-server.log c. Wait for words to the effect of: 04 May 2016
16:16:16,003 INFO [main] AmbariServer:548 - ********* Started
Services **********
Configuring a New Cluster 1. Open up a browser on a desktop that
has access to the vm-instance running ambari. This is the
vm-instance on which you ran ambariPrep.py. Connect to
http://10.129.71.21:8080 The login will be Username = admin
Password = admin
http://10.129.71.21:8080/
-
Hortonworks Ambari VM for HSP Deployment Guide
MK-95HSP017-01 3
2. The wizard screen will come up next. Choose to “Launch
install Wizard”.
3. Assign a name to the new Hadoop cluster.
-
Hortonworks Ambari VM for HSP Deployment Guide
MK-95HSP017-01 4
4. Select HDP stack version to load on new cluster. Currently
only HDP 2.3 is available with this template.
5. Install options. On this page we will need to gather
information from the Ambari server. Ssh 10.129.71.21
-
Hortonworks Ambari VM for HSP Deployment Guide
MK-95HSP017-01 5
Username = root Password = smrace1 First we need to gather the
FQDNs of each of the vm-instances that will run Ambari agents and
be a part of the Hadoop cluster. The ambariPrep.py script has
already setup the hosts file which is where the information can be
pulled from.
Example: [root@ambaridev-1 ~]# grep escale /etc/hosts | awk
'{print $2}' ambaridev-1.escale.bluearc.com
ambaridev-2.escale.bluearc.com ambaridev-3.escale.bluearc.com
ambaridev-4.escale.bluearc.com ambaridev-5.escale.bluearc.com Paste
the output above into the Target hosts section. Second we will need
to get the private ssh key from the Ambari server. This must be
pasted into the text box below the radio button indicating “Provide
your SSH Private Key..”. Please copy and paste from id_rsa.
For example: [root@ambaridev-1 ~]# cat /root/.ssh/id_rsa
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAu7WJMc2HZWO0S/iPTPXpjMkbB+m6DAAweMUwJu7aw325dfR6
5VIPWR0GRZR+zJPJho4EvvgKYmHg2t7ZA3yfwU8G3ghAwvTHqB4f4jPoaAkw9wH1
6U299ZVhdNdjwuE09o0GGidpd8Wd8Yvfuot78JjpttL7H5vtIfq6WfrMqUbjVdou
IlLkuIwvMP1U8Uskzz1pvSkaokpXZ/3yuWSXQOx/cArEDlRALhAt+B7/tz54gF1a
Nog/ZKUcHhM50NiazB04AJiQbP1cWPXF+XD2F/jm5OkeyYVKA0TqQfveFbD27u6r
fF48/vXqvNCppBh5/zqTpQuArdmziC1B2y6vnQIDAQABAoIBAARXkrDgPD2QLN+F
BGuXVKu5CzMHqRIV/wkMJqz/4XecNOoRmbYsS8aX8UkywiwhTwwZxOpV+oeEFBt8
BDEcdOWX2mTQdA8Ocn/lEn+9C6c8bzGHjhSzg61mozZ8tQYRB5ScdoOsXbgzjg/E
EkDcsafHUYO0dlBpXiyd9dPlDgyoZoiM8awo6mbXNchmFZdJ9oJvnlH5TG3APaJh
0Qm4iIXytz4buwKI8THv6QKBgEfjjvQPaIhzbsLQaxeyv4zvrM0R2HOHNC5vBV1b
dKQ01VE8NeJH+pmX1KPbKUddLH7GC7zDNV3anV1BOvjV0QhmUCMT6FT0LF8kfcbz
ngPQt3aLkMOdvRcqMD8RLPP/G/KAi1w79JeCEZ81ulIVrBa1tErEvA5YCLTSIwlP
depRAoGBALKkwr5/HshhFPGoFiAnpTCH/L7dJuPql7J1oPiN5MpjzaPu1T1BkTVK
AhVeqO8IYq7bX4p0bnK/hDgEF0YJ9TdmHlyb7E+feOGN/3YmUR3SMHW1E9nvui41
fusi2EEQY63uHMzxo7p7GRrAnHRbyNndivwugn56c3a3C5o8LTWQ -----END RSA
PRIVATE KEY----- Important: Please do not copy and paste the key
from this document. You must use the id_rsa private
key that is on the VM on which you ran ambariPrep.py.
-
Hortonworks Ambari VM for HSP Deployment Guide
MK-95HSP017-01 6
6. Click “Register and Confirm”. All hosts should show
success.
7. Choose Services. Install the following: Yarn + Mapreduce2
Tez
-
Hortonworks Ambari VM for HSP Deployment Guide
MK-95HSP017-01 7
Hive HBase Pig Sqoop Zookeeper Ambari Metrics EscaleFS Spark
Currently not in the local repo: Oozie Falcon Storm Flume Accumulo
Atlas Kafka Knox Mahout Slider
8. Assign Masters. In this step ensure that the Ambari server is
only going to be used for nodemanager client. It is also a good
practice to run Hive components (HiveServer, Hive MetaStore) on the
same VM. Depending on the load in your environment, you may want to
consider a separate VM for HBase.
-
Hortonworks Ambari VM for HSP Deployment Guide
MK-95HSP017-01 8
9. Assign Slaves and Clients. Assign Nodemanager for all.
RegionServer for all except the ambari server. Phoenix server none.
Spark Thrift server none. Client all.
10. Customize Services. There should be no changes required
here. Any memory settings can be changed here on a per service
basis if needed.
-
Hortonworks Ambari VM for HSP Deployment Guide
MK-95HSP017-01 9
A memory warning may come up. This is ok. There is some memory
left over and Ambari wants to try and assign 100%.
11. Review. The following information can be saved off of
printed. This is a high level layout of what is going to be
installed.
-
Hortonworks Ambari VM for HSP Deployment Guide
MK-95HSP017-01 10
12. Start Install.
Everything should complete and present as green. Takes ~10
minutes.
-
Hortonworks Ambari VM for HSP Deployment Guide
MK-95HSP017-01 11
13. Summary of the Install.
14. Dashboard view of the up and running Hadoop cluster.
-
Hortonworks Ambari VM for HSP Deployment Guide
MK-95HSP017-01 12
Setup HSP Ambari ClusterSetting up AmbariConfiguring a New
Cluster