原文地址：http://blog.chinaunix.net/xmlrpc.php?r=blog/article&uid=26230811&id=4023821
环境配置
|
Hostname |
IP |
OS |
|
Ambari |
172.16.235.164 |
Centos X64 base install |
|
Maste.hadoop |
172.16.235.165 |
Centos X64 base install |
|
Datanode1.hadoop |
172.16.235.166 |
Centos X64 base install |
操作系统建议以software server 方式安装，base 方式安装后，需要安装Development tools，本文以base举例。
一：制作ambari，hadoop本地源
首先需要有一台能够联网的linux机器
1：制作ambari（1.4.1.25）本地源
[root@controller ~]# wget http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.4.1.25/ambari.repo
[root@controller ~]# cp ambari.repo /etc/yum.repos.d/
将网络源中的包同步到本地：
reposync -r HDP-UTILS-1.1.0.16
reposync -r Updates-ambari-1.4.1.25
reposync -r ambari-1.x
会得到相应的三个目录，每个目录中有相应的rpm包
准备一台http服务器，本文以172.16.235.164作为http服务器发布安装源
mkdir /var/www/html/ambari
将 HDP-UTILS-1.1.0.16 Updates-ambari-1.4.1.25 ambari-1.x复制到 /var/www/html/ambari下
cd /var/www/html/ambari
createrepo ./ #没有createrepo命令自己去安装
这样ambari安装源就做好了，下载好jdk-6u31-linux-x64.bin备用
2：准备hortonworks hadoop （HDP）2.0.6本地源
准备HDP网络源
vim /etc/yum.repos.d/HDP.repo，添加以下内容
[HDP-2.0.6]
name=HDP
baseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.0.6.0
enabled=1
gpgcheck=0
同步网络源到本地：
reposync -r HDP-2.0.6 #将近1G
讲 HDP-2.0.6 复制到http server的/var/www/html/hdp下
cd /var/www/html/hdp
createrepo ./
这样hadoop的本地源就做好了
3：制作系统本地源（安装ambari和hadoop时，可能会依赖些系统包，这时需要有系统的本地源来安装）
http server 挂载系统光盘
mount /dev/cdrom /var/www/html/centos/
二：安装ambari
系统准备：关闭iptables、selinux,启动ntpd服务
[root@ambari ]vim /etc/hosts
172.16.235.164 ambari
172.16.235.165 master.hadoop
172.16.235.166 datanode1.hadoop
配置与所有主机的无密码登入
[root@ambari ~]# ssh-keygen #一路回车（所有主机都要执行）
[root@ambari ~]# cd .ssh/
[root@ambari .ssh]# ls
id_rsa id_rsa.pub
[root@ambari .ssh]# scp id_rsa.pub master.hadoop:/root/.ssh/1
The authenticity of host 'master.hadoop (172.16.235.165)' can't be established.
RSA key fingerprint is fc:33:53:5f:f2:20:86:92:ac:96:09:e2:dc:e7:33:ea.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'master.hadoop,172.16.235.165' (RSA) to the list of known hosts.
root@master.hadoop's password:
id_rsa.pub 100% 393 0.4KB/s 00:00
[root@ambari .ssh]# scp id_rsa.pub datanode1.hadoop:/root/.ssh/1
The authenticity of host 'datanode1.hadoop (172.16.235.166)' can't be established.
RSA key fingerprint is fc:33:53:5f:f2:20:86:92:ac:96:09:e2:dc:e7:33:ea.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'datanode1.hadoop,172.16.235.166' (RSA) to the list of known hosts.
root@datanode1.hadoop's password:
id_rsa.pub 100% 393 0.4KB/s 00:00
登入master.hadoop
[root@master ~]# cd .ssh/
[root@master .ssh]# cat 1> authorized_keys
登入datanode1.hadoop
[root@datanode1 ~]# cd .ssh/
[root@datanode1 .ssh]# cat 1> authorized_keys
回到ambari 测试无密码登入
[root@ambari .ssh]# ssh master.hadoop
Last login: Wed Dec 4 11:11:15 2013 from 172.16.38.48
[root@master ~]# exit
logout
Connection to master.hadoop closed.
[root@ambari .ssh]# ssh datanode1.hadoop
Last login: Wed Dec 4 10:38:06 2013 from 172.16.38.48
[root@datanode1 ~]# exit
logout
Connection to datanode1.hadoop closed.
[root@ambari .ssh]#
安装与配置ambari服务：
[root@ambari ]vim /etc/yum.repos.d/ambari.repo 添加以下内容
[ambari-1.x]
name=Ambari 1.x
baseurl=http://172.16.235.164/ambari/
gpgcheck=0
enabled=1
[HDP-UTILS-1.1.0.16]
name=Hortonworks Data Platform Utils Version - HDP-UTILS-1.1.0.16
baseurl=http://172.16.235.164/ambari/
gpgcheck=0
enabled=1
[Updates-ambari-1.4.1.25]
name=ambari-1.4.1.25 - Updates
baseurl=http://172.16.235.164/ambari/
gpgcheck=
enabled=1
[root@ambari ]vim /etc/yum.repos.d/centos.repo 添加以下内容
[centos]
baseurl=http://172.16.235.164/centos/
enabled=1
gpgcheck=0
执行安装ambari：
[root@ambari yum.repos.d]# yum -y install ambari-server
Loaded plugins: fastestmirror, security
Repository 'Updates-ambari-1.4.1.25': Error parsing config: Error parsing "gpgcheck = ''": invalid boolean value
Repository 'centos' is missing name in configuration, using id
Loading mirror speeds from cached hostfile
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package ambari-server.noarch 0:1.4.1.25-1 will be installed
--> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server-1.4.1.25-1.noarch
--> Running transaction check
---> Package postgresql-server.x86_64 0:8.4.13-1.el6_3 will be installed
--> Processing Dependency: postgresql-libs(x86-64) = 8.4.13-1.el6_3 for package: postgresql-server-8.4.13-1.el6_3.x86_64
--> Processing Dependency: postgresql(x86-64) = 8.4.13-1.el6_3 for package: postgresql-server-8.4.13-1.el6_3.x86_64
--> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-server-8.4.13-1.el6_3.x86_64
--> Running transaction check
---> Package postgresql.x86_64 0:8.4.13-1.el6_3 will be installed
---> Package postgresql-libs.x86_64 0:8.4.13-1.el6_3 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
====================================================================================================================================
Package Arch Version Repository Size
====================================================================================================================================
Installing:
ambari-server noarch 1.4.1.25-1 HDP-UTILS-1.1.0.16 37 M
Installing for dependencies:
postgresql x86_64 8.4.13-1.el6_3 centos 2.8 M
postgresql-libs x86_64 8.4.13-1.el6_3 centos 200 k
postgresql-server x86_64 8.4.13-1.el6_3 centos 3.4 M
Transaction Summary
====================================================================================================================================
Install 4 Package(s)
Total download size: 43 M
Installed size: 88 M
Downloading Packages:
(1/4): ambari-server-1.4.1.25-1.noarch.rpm | 37 MB 00:00
(2/4): postgresql-server-8.4.13-1.el6_3.x86_64.rpm | 3.4 MB 00:00
(3/4): postgresql-libs-8.4.13-1.el6_3.x86_64.rpm | 200 kB 00:00
(4/4): postgresql-8.4.13-1.el6_3.x86_64.rpm | 2.8 MB 00:00
------------------------------------------------------------------------------------------------------------------------------------
Total 43 MB/s | 43 MB 00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : postgresql-libs-8.4.13-1.el6_3.x86_64 1/4
Installing : postgresql-8.4.13-1.el6_3.x86_64 2/4
Installing : postgresql-server-8.4.13-1.el6_3.x86_64 3/4
Installing : ambari-server-1.4.1.25-1.noarch 4/4
Verifying : postgresql-server-8.4.13-1.el6_3.x86_64 1/4
Verifying : postgresql-libs-8.4.13-1.el6_3.x86_64 2/4
Verifying : ambari-server-1.4.1.25-1.noarch 3/4
Verifying : postgresql-8.4.13-1.el6_3.x86_64 4/4
Installed:
ambari-server.noarch 0:1.4.1.25-1
Dependency Installed:
postgresql.x86_64 0:8.4.13-1.el6_3 postgresql-libs.x86_64 0:8.4.13-1.el6_3 postgresql-server.x86_64 0:8.4.13-1.el6_3
Complete!
开始配置ambari 服务：
首先上传jdk文件
[root@ambari ]cd /var/lib/ambari-server/resources 将jdk-6u31-linux-x64.bin上传到该目录
[root@ambari resources]# chmod +x jdk-6u31-linux-x64.bin
[root@ambari yum.repos.d]# ambari-server setup
Using python /usr/bin/python2.6
Initializing...
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'permissive'
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)?
Customize user account for ambari-server daemon [y/n] (n)?
Adjusting ambari-server permissions and ownership...
Checking iptables...
WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.
OK to continue [y/n] (y)?
Checking JDK...
JDK already exists, using /var/lib/ambari-server/resources/jdk-6u31-linux-x64.bin
Installing JDK to /usr/jdk64
Successfully installed JDK to /usr/jdk64/jdk1.6.0_31
Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-6.zip to /var/lib/ambari-server/resources/jce_policy-6.zip
To download the JCE Policy files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.htmlNot accepting will result in errors when configuring Kerberos security.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)?
Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/jce_policy-6.zip
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)?
Default properties detected. Using built-in database.
Checking PostgreSQL...
Running initdb: This may take upto a minute.
About to start PostgreSQL
Configuring local database...
Connecting to the database. Attempt 1...
Configuring PostgreSQL...
Restarting PostgreSQL
Ambari Server 'setup' completed successfully.
[root@ambari ~]# ambari-server start
Using python /usr/bin/python2.6
Starting ambari-server
Ambari Server running with 'root' privileges.
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Ambari Server 'start' completed successfully.
到此ambari安装好
三：本地安装hadoop
给ambari配置本地hadoo源地址：
[root@ambari ~]# cd /var/lib/ambari-server/resources/stacks/HDPLocal/2.0.6/repos/
[root@ambari repos]# vim repoinfo.xml 修改红色部分为本地源地址
<reposinfo>
< os type="centos6">
< repo>
< baseurl>http://172.16.235.164/hdp/</baseurl>
< repoid>HDP-2.0.6</repoid>
< reponame>HDP</reponame>
< /repo>
< /os>
这样在ambari执行hadoop安装时，会将本地源地址配置到所有主机上，在为所有主机安装ambari-agent时，会将ambari server下的/etc/yum.repo/ambari.repo文件复制到所有主机
主机准备（master.hadoop、datanode1.hadoop）配置相同，只举例master.hadoop
关闭iptables、selinux、启动ntpd服务（略）
[root@master ~]# vim /etc/hosts 添加以下
172.16.235.164 ambari
172.16.235.165 master.hadoop
172.16.235.166 datanode1.hadoop
[root@master ~]# vim /etc/yum.repos.d/centos.repo 添加以下内容
[root
[centos]
baseurl=http://172.16.235.164/centos/
enabled=1
gpgcheck=0
@master ~]# yum -y groupinstall "Development tools"
登入ambari管理页面
http://172.16.235.164:8080/ 账户：admin 密码：admin
配置集群名称
选择HDP版本
选择目标主机，一行一个主机。要让ambar server自动在所以主机安装ambari-agent，需要提供ambari server root账户的私钥，即/root/.ssh/ id_rsa文件，选中使用本地源安装hadoop，然后下一步
确认并且选择主机
确认OK
下一步选择要安装的组件，选好后下一步
选择组件分配在哪些主机，都是master 的服务
选择安装client，主机有限，master同时安装client
配置服务，其中nagios中要填写密码及报警接受邮箱必填，其他选择性配置
显示配置信息
开始安装，过程稍微有点长
安装过程
Google浏览器反应比较慢，用另一某浏览器已经显示succes了
Ganglia：
nagios：