RedHat Ent 6.5 64bit编译安装hadoop2.4.1

RedHat Ent 6.5 64bit编译安装hadoop2.4.1

感谢原帖：http://blog.csdn.net/w13770269691/article/details/16883663/

  step 1.修改yum：（针对redhat ent未注册用户，注册用户直接跳过这一步）

    参考：http://blog.csdn.net/zhngjan/article/details/20843465

  step 2.下载源码包：http://mirrors.hust.edu.cn/apache/hadoop/common/stable2/

wget http://mirrors.hust.edu.cn/apache/hadoop/common/stable2/hadoop-2.4.1-src.tar.gz

  step 3.准备maven作为编译hadoop的工具

    a.下载编译好的maven包：

wget http://mirror.bit.edu.cn/apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz
tar -zxvf apache-maven-3.1.1-bin.tar.gz -C /opt/

    b.配置maven的环境变量，在/etc/profile文件结尾中添加如下代码

export MAVEN_HOME=/opt/apache-maven-3.1.1
export PATH=$PATH:${MAVEN_HOME}/bin

    c.执行如下命令使配置文件生效

source /etc/profile

    d.测试maven

mvn -version

   e.由于maven国外服务器可能连不上，先给maven配置一下国内镜像

    在maven目录下，conf/settings.xml,在<mirrors></mirros>里添加如下内容（注意不要添加到注释里面了）

<mirror> 
  <id>nexus-osc</id>  
  <mirrorOf>*</mirrorOf>  
  <name>Nexusosc</name>  
  <url>http://maven.oschina.net/content/groups/public/</url>  
</mirror>

 在maven目录下，conf/settings.xml,在<profiles></profiles>添加如下内容（注意不要添加到注释里面了）

<profile>  
  <id>jdk-1.7</id>  
  <activation>  
  <jdk>1.7</jdk>  
  </activation>  
  <repositories>  
  <repository>  
   <id>nexus</id>  
   <name>local private nexus</name>  
   <url>http://maven.oschina.net/content/groups/public/</url>  
   <releases>  
    <enabled>true</enabled>  
   </releases>  
   <snapshots>  
    <enabled>false</enabled>  
   </snapshots>  
  </repository>  
  </repositories>  
  <pluginRepositories>  
   <pluginRepository>  
    <id>nexus</id>  
    <name>local private nexus</name>  
    <url>http://maven.oschina.net/content/groups/public/</url>  
    <releases>  
      <enabled>true</enabled>  
    </releases>  
    <snapshots>  
      <enabled>false</enabled>  
    </snapshots>  
   </pluginRepository>  
  </pluginRepositories>  
</profile>  

  step 4.hadoop2.4.1编译需要protoc2.5.0的支持，所以还要安装下载protoc2.5.0

    官方网址：https://code.google.com/p/protobuf/downloads/list

    百度网盘网址：http://pan.baidu.com/s/1pJlZubT

    a.对protoc进行编译安装前先要装几个依赖包:gcc,gcc-c++,make 如果已经安装的可以忽略

yum install gcc  
yum intall gcc-c++  
yum install make 

   b.配置protoc

tar -xvf protobuf-2.5.0.tar.bz2  
cd protobuf-2.5.0  
./configure --prefix=/opt/protoc/  
make && make install 
yum install gcc 

    红色字体是自定义的protoc安装目录，完成上面的命令后，再配置protoc环境变量，同样在/etc/profile尾部加入：

export PATH=/opt/protoc/bin:$PATH

  step 5.，还不要着急开始编译安装，不然又是各种错误,需要安装cmake,openssl-devel,ncurses-devel依赖

yum install cmake  
yum install openssl-devel  
yum install ncurses-devel  

  step 6. 解压并编译hadoop源码(一定要进入到源码解压的位置再执行mvn编译)

tar -zxvf hadoop-2.4.1-src.tar.gz -C /opt/
cd /opt/hadoop-2.4.1-src
mvn package -Pdist,native -DskipTests -Dtar

编译耗时比较长，耐心等待，成功后结果类似于：

 pache Hadoop Distributed Copy .................... SUCCESS [33.648s]
[INFO] Apache Hadoop Archives ............................ SUCCESS [7.303s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [21.288s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [14.611s]
[INFO] Apache Hadoop Data Join ........................... SUCCESS [8.334s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [10.737s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [18.321s]
[INFO] Apache Hadoop OpenStack support ................... SUCCESS [17.136s]
[INFO] Apache Hadoop Client .............................. SUCCESS [14.563s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [0.254s]
[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [17.245s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [14.478s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.084s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [41.979s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 34:24.464s
[INFO] Finished at: Thu Aug 07 14:25:51 CST 2014
[INFO] Final Memory: 159M/814M
[INFO] ------------------------------------------------------------------------
You have new mail in /var/spool/mail/root

编译后的路径在:hadoop-2.4.1-src/hadoop-dist/target/hadoop-2.4.1

进入hadoop-2.4.1目录，测试安装是否成功

[root@cupcs-redhat6 bin]# cd /opt/hadoop-2.4.1-src/hadoop-dist/target/hadoop-2.4.1/bin
[root@cupcs-redhat6 bin]# ./hadoop version
Hadoop 2.4.1
Subversion Unknown -r Unknown
Compiled by root on 2014-08-07T05:52Z
Compiled with protoc 2.5.0
From source with checksum bb7ac0a3c73dc131f4844b873c74b630
This command was run using
/opt/hadoop-2.4.1-src/hadoop-dist/target/hadoop-2.4.1/share/hadoop/common/hadoop-common-2.4.1.jar
[root@cupcs-redhat6 bin]# cd ..
[root@cupcs-redhat6 hadoop-2.4.1]# file lib/native/*
lib/native/libhadoop.a: current ar archive
lib/native/libhadooppipes.a: current ar archive
lib/native/libhadoop.so: symbolic link to `libhadoop.so.1.0.0'
lib/native/libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1
(SYSV), dynamically linked, not stripped
lib/native/libhadooputils.a: current ar archive
lib/native/libhdfs.a: current ar archive
lib/native/libhdfs.so: symbolic link to `libhdfs.so.0.0.0'
lib/native/libhdfs.so.0.0.0: ELF 64-bit LSB shared object, x86-64, version 1
(SYSV), dynamically linked, not stripped
[root@cupcs-redhat6 hadoop-2.4.1]#