Centos7安装 Hadoop（单节点）

1.Hadoop简介

• Hadoop是一个由Apache基金会所开发的开源分布式系统基础框架，使用Java开发，是处理大规模数据的软件平台。
• Hadoop可以从单一节点扩展到上千节点。用户可以在不了解分布式底层细节的情况下，开发分布式程序。充分利用集群的威力进行高速运算和存储。
• 本次安装的是单点问题

2.安装Java JDK

• 由于Hadoop使用Java编写，要运行它我们需要安装Java运行环境
• jdk下载地址:         http://www.oracle.com/technetwork/java/javase/downloads/index.html
• hadoop下载地址: https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/core/

#1.把事先准备好的包放到/usr/local/src下
[root@hadoop_master src]# ll
total 445732
-rw-r--r-- 1 root root 266688029 Nov 17 14:08 hadoop-2.7.4.tar.gz
-rw-r--r-- 1 root root 189736377 Nov 13 15:37 jdk-8u151-linux-x64.tar.gz
[root@hadoop_master src]# tar xf jdk-8u151-linux-x64.tar.gz 
[root@hadoop_master src]# mv jdk1.8.0_151/  /usr/local/jdk
#2.在环境变量追加
export JAVA_HOME=/usr/local/jdk
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/jre:$JRE_HOME
export PATH=$JAVA_HOME/bin:$PATH
#3.创建用户
[root@hadoop_master src]# useradd hadoop
[root@hadoop_master src]# passwd hadoop
Changing password for user hadoop.
New password: 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
passwd: all authentication tokens updated successfully.

3.解压hadoop包

[root@hadoop_master src]# tar xf hadoop-2.7.4.tar.gz 
[root@hadoop_master src]# mv hadoop-2.7.4 /usr/local/hadoop
[root@hadoop_master src]# chown -R hadoop:hadoop /usr/local/hadoop/

4.配置hadoop

[root@hadoop_master src]# su - hadoop
[hadoop@hadoop_master ~]$ vi .bashrc 
export JAVA_HOME=/usr/local/jdk
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
[hadoop@hadoop_master ~]$ source .bashrc 
[hadoop@hadoop_master ~]$  cd $HADOOP_HOME/etc/hadoop
[hadoop@hadoop_master hadoop]$ vi hadoop-env.sh
export JAVA_HOME=/usr/local/jdk
[hadoop@hadoop_master hadoop]$ grep "JAVA" hadoop-env.sh
# The only required environment variable is JAVA_HOME.  All others are
# set JAVA_HOME in this file, so that it is correctly defined on
export JAVA_HOME=/usr/local/jdk    #修改后的结果
#HADOOP_JAVA_PLATFORM_OPTS="-XX:-UsePerfData $HADOOP_JAVA_PLATFORM_OPTS"

• 编辑core-site.xml文件

<configuration>
<property>
  <name>fs.default.name</name>
    <value>hdfs://localhost:9000</value>
</property>
</configuration>

• 编辑hdfs-site.xml文件：

<configuration>
<property>
 <name>dfs.replication</name>
 <value>1</value>
</property>
 
<property>
  <name>dfs.name.dir</name>
  <value>file:///usr/local/hadoop/hadoopdata/namenode</value>
</property>
 
<property>
  <name>dfs.data.dir</name>
  <value>file:///usr/local/hadoop/hadoopdata/datanode</value>
</property>
 
</configuration>

• 创建目录

[hadoop@hadoop_master hadoop]$ mkdir -p /usr/local/hadoop/hadoopdata/namenode  /usr/local/hadoop/hadoopdata/datanode /usr/local/hadoop/hadoopdata

• 创建mapred-site.xml文件：

[hadoop@hadoop_master hadoop]$ cp mapred-site.xml.template mapred-site.xml
<configuration>
 <property>
  <name>mapreduce.framework.name</name>
  <value>yarn</value>
 </property>
</configuration>

• 编辑yarn-site.xml文件:配置了Hadoop单节点

[hadoop@hadoop_master hadoop]$ cat yarn-site.xml
<configuration>

<!-- Site specific YARN configuration properties -->
<property>
  <name>yarn.nodemanager.aux-services</name>
  <value>mapreduce_shuffle</value>
</property>
</configuration>

• 初始化HDFS文件系统：

[hadoop@hadoop_master hadoop]$ hdfs namenode -format

• Hadoop相关脚本:

[hadoop@hadoop_master hadoop]$ cd ../../sbin/
[hadoop@hadoop_master sbin]$ ls
distribute-exclude.sh  hdfs-config.sh           refresh-namenodes.sh  start-balancer.sh    start-yarn.cmd  stop-balancer.sh    stop-yarn.cmd
hadoop-daemon.sh       httpfs.sh                slaves.sh             start-dfs.cmd        start-yarn.sh   stop-dfs.cmd        stop-yarn.sh
hadoop-daemons.sh      kms.sh                   start-all.cmd         start-dfs.sh         stop-all.cmd    stop-dfs.sh         yarn-daemon.sh
hdfs-config.cmd        mr-jobhistory-daemon.sh  start-all.sh          start-secure-dns.sh  stop-all.sh     stop-secure-dns.sh  yarn-daemons.sh

• 启动Hadoop服务

[hadoop@hadoop_master sbin]$ start-dfs.sh     #启动Hadoop
[hadoop@hadoop_master sbin]$ start-yarn.sh   #启动yarn
[hadoop@hadoop_master sbin]$ jps                  #查看服务状态
3664 NameNode
4368 NodeManager
4771 Jps
3785 DataNode
3945 SecondaryNameNode
4090 ResourceManager

• 登录验证:
• http://10.0.0.60:50070  #namenode端口
• http://10.0.0.60:8088    #所有项目访问