Hadoop伪分布式部署

Hadoop 的配置文件位于 /$HADOOP_HOME/etc/hadoop/

1、 修改core-site.xml

<configuration>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>file:/home/cauchy/software/hadoop/tmp</value>
    <description>Abase for other temporary directories.</description>
  </property>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://hadoop:9000</value>
  </property>
</configuration>

2、 修改hdfs-site.xml

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>

  <property>
    <name>dfs.namenode.name.dir</name>
    <value>file:/home/cauchy/software/hadoop/name</value>
  </property>

  <property>
    <name>dfs.datanode.data.dir</name>
    <value>file:/home/cauchy/software/hadoop/data</value>
  </property>
</configuration>

3、 修改mapred-site.xml

 cp mapred-site.xml.template mapred-site.xml

<configuration>
  <property>
     <name>mapreduce.framework.name</name>
     <value>yarn</value>
  </property>
  <property>
     <name>mapreduce.jobhistory.address</name>
     <value>master:10020</value>
  </property>
  <property>
     <name>mapreduce.jobhistory.webapp.address</name>
     <value>master:19888</value>
  </property>
</configuration>

4、 修改yarn-site.xml

<configuration>

<!-- Site specific YARN configuration properties -->
 <property>
  <name>yarn.nodemanager.aux-services</name>
  <value>mapreduce_shuffle</value>
 </property>
 <property>
  <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
  <value>org.apache.hadoop.mapred.ShuffleHandler</value>
 </property>
 <property>
  <name>yarn.resourcemanager.address</name>
  <value>hadoop:8032</value>
 </property>
 <property>
  <name>yarn.resourcemanager.scheduler.address</name>
  <value>hadoop:8030</value>
 </property>
 <property>
  <name>yarn.resourcemanager.resource-tracker.address</name>
  <value>hadoop:8035</value>
 </property>
 <property>
  <name>yarn.resourcemanager.admin.address</name>
  <value>hadoop:8033</value>
 </property>
 <property>
  <name>yarn.resourcemanager.webapp.address</name>
  <value>hadoop:8088</value>
 </property>
</configuration>

5、 格式化namenode

hdfs namenode –format

6、 修改hadoop-env.sh

# The java implementation to use.
export JAVA_HOME=/home/cauchy/software/jdk

这里很奇怪，设置了JAVA_HOME环境变量，但是访问不到。

7、 启动

start-all.sh

启动过程中有询问请直接输入yes

8、 查看效果

jsp

3426 NameNode
2947 ResourceManager
4276 Jps
3768 SecondaryNameNode
4009 NodeManager
3551 DataNode