编译安装hadoop2.x

1、Requirements:

* Unix System
* JDK 1.7+
* Maven 3.0 or later
* Findbugs 1.3.9 (if running findbugs)
* ProtocolBuffer 2.5.0
* CMake 2.6 or newer (if compiling native code), must be 3.0 or newer on Mac
* Zlib devel (if compiling native code)
* openssl devel ( if compiling native hadoop-pipes )
* Internet connection for first build (to fetch all Maven and Hadoop dependencies)

安装以上先决条件：
yum groupinstall “Development tools”
yum -y install cmake zlib openssl gcc zlib-devel openssl-devel make

tar -zxvf protobuf-2.5.0.tar.gz
cd protobuf-2.5.0
./configure
make && make install
protoc --version

tar  -zxvf findbugs-3.0.0.tar.gz -C /usr/local/findbugs
mv /usr/local/findbugs-3.0.0 /usr/local/findbugs

设置环境变量：
export FINDBUGS_HOME = /usr/local/findbugs
export PATH=$PATH:$FINDBUGS_HOME/bin

生效环境变量：source /etc/profile
验证：findbugs -version

安装jdk，略。

2、安装mave：

1）解压缩maven
tar -zxvf apache-maven-3.3.3-bin.tar.gz
mv apache-maven-3.3.3 /opt/service/maven-3.3.3
chown -R hadoop:hadoop /opt/service/maven-3.3.3/

2）配置maven环境变量：
export MAVEN_HOME=/opt/service/maven-3.3.3
export PATH=$MAVEN_HOME/bin:$PATH

3）本机编译环境如下：
#java -version
java version "1.7.0_67"
Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

# scala -version
Scala code runner version 2.10.6 -- Copyright 2002-2013, LAMP/EPFL

# mvn -version
Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; 2015-04-22T19:57:37+08:00)
Maven home: /opt/service/maven-3.3.3
Java version: 1.7.0_67, vendor: Oracle Corporation
Java home: /opt/service/jdk1.7.0_67/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-514.el7.x86_64", arch: "amd64", family: "unix"

4）修改maven仓库默认位置：

vim conf/settings.xml
  
   --添加如下内容：
   <localRepository>/home/hadoop/.m2/repository/</localRepository>
  
5）修改maven镜像位置为阿里云的maven镜像仓库

<mirrors>
      <!-- mirror
       | Specifies a repository mirror site to use instead of a given repository. The repository that
       | this mirror serves has an ID that matches the mirrorOf element of this mirror. IDs are used
       | for inheritance and direct lookup purposes, and must be unique across the set of mirrors.
       |
      <mirror>
        <id>mirrorId</id>
        <mirrorOf>repositoryId</mirrorOf>
        <name>Human Readable Name for this Mirror.</name>
        <url>http://my.repository.com/repo/path</url>
      </mirror>
       -->

    <mirror>
          <id>alimaven</id>
          <mirrorOf>central</mirrorOf>
          <name>aliyun maven</name>
          <url>http://maven.aliyun.com/nexus/content/repositories/central/</url>
      </mirror>
     
      <mirror>
          <id>repo1</id>
          <mirrorOf>central</mirrorOf>
          <name>Human Readable Name for this Mirror</name>
          <url>http://repo1.maven.org/maven2/</url>
      </mirror>
     
      <mirror>
          <id>repo2</id>
          <mirrorOf>central</mirrorOf>
          <name>Human Readable Name for this Mirror</name>
          <url>http://repo2.maven.org/maven2/</url>
      </mirror>

  </mirrors>
   
更改完成之后可以在命令行，执行命令验证一下：mvn help:system
命令执行完成后我们可以在目录/home/hadoop/.m2/repository/下看到很多文件，这时表示maven已经配置成功了。
$ ls /home/hadoop/.m2/repository/
backport-util-concurrent  classworlds  com  commons-cli  commons-lang  jdom  jtidy  junit  org  plexus  xmlpull  xpp3

3、编译hadoop
export MAVEN_OPTS="-Xms256m -Xmx512m"
mvn package -DskipTests -Pdist,native -Drequire.snappy

4、编译成功后的tar包存储在
./hadoop-dist/target/hadoop-2.6.0.tar.gz

5、配置hadoop的native支持
     用步骤6压缩包中lib/native目录下文件替换目标节点lib/native文件，然后通过以下指令检查hadoop库支持：
$bin/ hadoop checknative
Native library checking:
hadoop:  true /opt/cdh5/hadoop-2.6.0-cdh5.10.0/lib/native/libhadoop.so
zlib:    true /lib64/libz.so.1
snappy:  true /lib64/libsnappy.so.1
lz4:     true revision:99
bzip2:   true /lib64/libbz2.so.1
openssl: true /lib64/libcrypto.so

    至此，hadoop完全分布式安装成功。