zoukankan      html  css  js  c++  java
  • Hadoop in Action 第二章续1(伪分布式)

    Pseudo-distributed mode(伪分布式)

    伪分布式模式就是将所有hadoop程序运行在一台服务器上.这种模式将单机模式分割为各模块,主要还是用来debug你的程序,以便于测试内存占用量,HDFS input/output问题,已经各个deamons之间的交互等等.

    下面就是这种模式的配置例子:

    core-site.xml

    <?xml version=”1.0”?>

    <?xml-stylesheet type=”text/xsl” href=”confi guration.xsl”?>

    <!-- Put site-specifi c property overrides in this fi le. -->

    <confi guration>

    <property>

     <name>fs.default.name</name>

     <value>hdfs://localhost:9000</value>

     <description>The name of the default fi le system. A URI whose

     scheme and authority determine the FileSystem implementation.

     </description>

    </property>

    </confi guration>

    mapred-site.xml

    <?xml version=”1.0”?>

    <?xml-stylesheet type=”text/xsl” href=”confi guration.xsl”?>

    <!-- Put site-specifi c property overrides in this fi le. -->

    <confi guration>

    <property>

     <name>mapred.job.tracker</name>

     <value>localhost:9001</value>

     <description>The host and port that the MapReduce job tracker runs

     at.</description>

    </property>

    </confi guration>

    hdfs-site.xml

    <?xml version=”1.0”?>

    <?xml-stylesheet type=”text/xsl” href=”confi guration.xsl”?>30 CHAPTER 2 Starting Hadoop

    <!-- Put site-specifi c property overrides in this fi le. -->

    <confi guration>

    <property>

     <name>dfs.replication</name>

     <value>1</value>

     <description>The actual number of replications can be specifi ed when the

     fi le is created.</description>

    </property>

    </confi guration>

    core-site.xmlmapred-site.xml中需要指定服务器名和各个服务的端口号.

     In hdfs-site.xml we specify the default  replication factor for HDFS, which should only be one because we’re running on only

    one node.

    hdfs-site.xml中我们需要指定唯一的HDFS代理地址.同时,我们需要指定Secondary NameNode的位置.(mastersslaves脚本在conf目录下)

     [hadoop-user@master]$ cat masters

    localhost

    [hadoop-user@master]$ cat slaves

    Localhost

    虽然所有的deamon程序运行在同一台服务器上,但是它们之间的通讯方式仍然就如同分布式一样是使用ssh协议.

    你如果已经准备好启动hadoop,第一步要做的就是运行格式化HDFS的脚本

     [hadoop-user@master]$ bin/hadoop namenode –format

    jps命令(0.21.0版本中并未提供)将列出所有成功部署的deamons

     

    [hadoop-user@master]$ bin/start-all.sh

    [hadoop-user@master]$ jps

    26893 Jps

    26832 TaskTracker

    26620 SecondaryNameNode

    26333 NameNode

    26484 DataNode

    26703 JobTracker Running Hadoop 31

    当然你也可以关闭所有hadoop进程

    [hadoop-user@master]$ bin/stop-all.sh

  • 相关阅读:
    POJ 1469 COURSES 二分图最大匹配
    POJ 1325 Machine Schedule 二分图最大匹配
    USACO Humble Numbers DP?
    SGU 194 Reactor Cooling 带容量上下限制的网络流
    POJ 3084 Panic Room 求最小割
    ZOJ 2587 Unique Attack 判断最小割是否唯一
    Poj 1815 Friendship 枚举+求最小割
    POJ 3308 Paratroopers 最小点权覆盖 求最小割
    1227. Rally Championship
    Etaoin Shrdlu
  • 原文地址:https://www.cnblogs.com/JohnLiang/p/2243371.html
Copyright © 2011-2022 走看看