CentOS7+JDK8编译Hadoop2.6.4

1. 下载相关软件

apache-maven-3.3.1-bin.tar.gz

protobuf-2.5.0.tar.gz

hadoop-2.6.4-src.tar.gz

2.配置好jdk8环境(请看相关文章)

3. 在centos7安装相关的依赖包

yum install -y autoconf automake libtool cmake
yum install -y ncurses-devel openssl-devel

4.安装maven

tar xvf apache-maven-3.3.1-bin.tar.gz
mv apache-maven-3.3.1 /opt/apache/

配置maven

vi /etc/profile.d/maven.sh

输入以下内容

export MAVEN_HOME=/opt/apache/apache-maven-3.3.1
export MVN_HOME=$MAVEN_HOME
export PATH=$PATH:$MVN_HOME/bin

使配置立即生效

source /etc/profile

检查安装是否成功

mvn -version
Apache Maven 3.3.1 (cab6659f9874fa96462afef40fcf6bc033d58c1c; 2015-03-14T04:10:27+08:00)
Maven home: /opt/apache/apache-maven-3.3.1
Java version: 1.8.0_65, vendor: Oracle Corporation
Java home: /usr/java/jdk1.8.0_65/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-327.10.1.el7.x86_64", arch: "amd64", family: "unix"

4.安装protobuf-2.5.0

tar xvf protobuf-2.5.0.tar.gz
cd protobuf-2.5.0
./configure
make && make install

编译完成后检查是否安装成功

# protoc --version
libprotoc 2.5.0

5. 编译hadoop-2.6.4

# tar xvf hadoop-2.6.4-src.tar.gz
# cd hadoop-2.6.4-src/
# mvn clean package -Pdist,native -DskipTests -Dtar  -Dmaven.javadoc.skip=true

接下来就是漫长的等待, maven要下载各类编译依赖包,最后出现大概如下信息:

...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 25:16 min
[INFO] Finished at: 2016-03-30T12:36:53+08:00
[INFO] Final Memory: 158M/440M
[INFO] ------------------------------------------------------------------------

验证一下编译是否已经成功:

# cd hadoop-dist/target/hadoop-2.6.4/lib/native/
# file *
libhadoop.a:        current ar archive
libhadooppipes.a:   current ar archive
libhadoop.so:       symbolic link to `libhadoop.so.1.0.0'
libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=29998b707256fa2a1f967e1f8f2e8486ca0fec15, not stripped
libhadooputils.a:   current ar archive
libhdfs.a:          current ar archive
libhdfs.so:         symbolic link to `libhdfs.so.0.0.0'
libhdfs.so.0.0.0:   ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=1c78b3f3f542cc9c586a0ad6cec05c1ef1d908df, not stripped

说明已经编译成功!