JobHistory的使用

开启jobhistory功能步骤:
1.修改etc/hadoop/mapred-site.xml文件
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>master:10020</value>
<description>MapReduce JobHistory Server IPC host:port</description>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>master:19888</value>
<description>MapReduce JobHistory Server Web UI host:port</description>
</property>
<property>
<name>mapreduce.jobhistory.done-dir</name>
<value>/history/done</value>
</property>
<property>
<name>mapreduce.jobhistory.intermediate-done-dir</name>
<value>/history/done_intermediate</value>
</property>
</configuration>

2.修改etc/hadoop/yarn-site.xml文件
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.log-aggregation-enable</name>
<value>true</value>
</property>
</configuration>

3.重启yarn
[root@master sbin]# ./start-yarn.sh

4.开启jobhistory服务
[root@master sbin]# ./mr-jobhistory-daemon.sh start historyserver
starting historyserver, logging to /root/app/hadoop-2.6.0-cdh5.7.0/logs/mapred-root-historyserver-master.out

5.观察进程
[root@master sbin]# jps
3436 NodeManager
3027 DataNode
4599 Jps
3345 ResourceManager
4530 JobHistoryServer
3215 SecondaryNameNode
2942 NameNode

6.重新提交mapreduce任务
[root@master hadoop-2.6.0-cdh5.7.0]# cd share/hadoop/mapreduce
[root@master mapreduce]# hadoop jar hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar pi 2 3

7.查看hdfs上jobhistory设置的历史日志目录
[root@master sbin]# hadoop dfs -ls /

8.查看web端信息(注意端口号)
http://192.168.56.100:8088/