安装hadoop 单机版本.

准备工作1 . 拷贝文件从主机1 到主机2 

1. 将本机的 software 目录拷贝到 hadoop03 的opt目录下.推送到hadoop03上 。

scp -r software root@hadoop03:/opt

2. 将03 opt目录下的东西拷贝到本机

scp -r root@haddop03:/opt/   ./

 

一、安装hdoop

1. tar解压hadoop软件安装

tar -vxf hadoop-2.7.1_64bit.tar.gz

2.添加环境变量 vim  /etc/profile

#hadoop
export HADOOP_HOME=/software/hadoop-2.7.1
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

3. source /etc/profile 生效

4. 配置hadoop-env.sh

[root@hadoop03 hadoop]# pwd
/software/hadoop-2.7.1/etc/hadoop
[root@hadoop03 hadoop]# vim hadoop-env.sh

5. 配置：core-site.xml

新建目录 /software/hadoop-2.7.1/data/tmp

<configuration>

<!-- 指定HDFS中NameNode的地址 -->
<property>
<name>fs.defaultFS</name>
    <value>hdfs://hadoop03:9000</value>
</property>

<!-- 指定Hadoop运行时产生文件的存储目录 -->
<property>
        <name>hadoop.tmp.dir</name>
        <value>/software/hadoop-2.7.1/data/tmp</value>
</property>
</configuration>
-- 插入 --              

 6. 配置：hdfs-site.xml

<configuration>
<!-- 指定HDFS副本的数量 -->
<property>
        <name>dfs.replication</name>
        <value>1</value>
</property>
</configuration>
-- 插入 --     

7. 务必要格式化

[atguigu@hadoop101 hadoop-2.7.2]$ bin/hdfs namenode -formatt

8. 启动namenode、datanode

[root@hadoop03 sbin]# pwd
/software/hadoop-2.7.1/sbin
[root@hadoop03 sbin]# hadoop-daemon.sh start namenode
starting namenode, logging to /software/hadoop-2.7.1/logs/hadoop-root-namenode-hadoop03.out

9.查看jps

[root@hadoop03 bin]# jps
7153 Jps
6955 NameNode
2414 QuorumPeerMain
7071 DataNode
[root@hadoop03 bin]# 

10 web页面查看

http://192.168.6.128:50070/dfshealth.html#tab-overview

 11.创建文件夹

hdfs dfs -mkdir -p /software/testinput

 

 12.查看日志

[root@hadoop03 software]# cd hadoop-2.7.1/logs/
[root@hadoop03 logs]# ls
hadoop-root-datanode-hadoop03.log    hadoop-root-namenode-hadoop03.out
hadoop-root-datanode-hadoop03.out    hadoop-root-namenode-hadoop03.out.1
hadoop-root-datanode-hadoop03.out.1  hadoop-root-namenode-hadoop03.out.2
hadoop-root-datanode-hadoop03.out.2  SecurityAuth-root.audit

二、安装yarn

 1.编辑yarn-en.sh

[root@hadoop03 hadoop]# echo $JAVA_HOME
/software/jdk1.8.0_181/
[root@hadoop03 hadoop]# vim yarn-env.sh 

 

2.配置 yarn-site.xml

<configuration>

<!-- Site specific YARN configuration properties -->


<!-- Reducer获取数据的方式 -->
<property>
                <name>yarn.nodemanager.aux-services</name>
                <value>mapreduce_shuffle</value>
</property>

<!-- 指定YARN的ResourceManager的地址 -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>hadoop03</value>
</property>


</configuration>

 3. 编辑mapred-env.sh

[root@hadoop03 hadoop]# vim mapred-env.sh

 4.

d）配置： (对mapred-site.xml.template重新命名为) mapred-site.xml
[atguigu@hadoop101 hadoop]$ mv mapred-site.xml.template mapred-site.xml
[atguigu@hadoop101 hadoop]$ vi mapred-site.xml

<!-- 指定MR运行在YARN上 -->
<property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
</property>

 5.启动集群，启动resource manager, nodemanager。

[root@hadoop03 sbin]# yarn-daemon.sh start resourcemanager
/software/hadoop-2.7.1/etc/hadoop/hadoop-env.sh:行26: /software/jdk1.8.0_181/: 是一个目录
starting resourcemanager, logging to /software/hadoop-2.7.1/logs/yarn-root-resourcemanager-hadoop03.out
/software/hadoop-2.7.1/etc/hadoop/hadoop-env.sh:行26: /software/jdk1.8.0_181/: 是一个目录
[root@hadoop03 sbin]# 

[root@hadoop03 sbin]# yarn-daemon.sh start nodemanager
/software/hadoop-2.7.1/etc/hadoop/hadoop-env.sh:行26: /software/jdk1.8.0_181/: 是一个目录
starting nodemanager, logging to /software/hadoop-2.7.1/logs/yarn-root-nodemanager-hadoop03.out
/software/hadoop-2.7.1/etc/hadoop/hadoop-env.sh:行26: /software/jdk1.8.0_181/: 是一个目录
[root@hadoop03 sbin]# 

 JPS查看

root@hadoop03 sbin]# jps
8720 NodeManager
8832 Jps
8455 ResourceManager
6955 NameNode
2414 QuorumPeerMain
7071 DataNode
[root@hadoop03 sbin]# 

查看mapreduce的web界面

 

 查看HDFS 为50070端口