zoukankan      html  css  js  c++  java
  • hadoop-1.1.2 在Windows环境下的部署

    1:先安装Cygwin

      参考http://blog.csdn.net/wind520/article/details/9223003

    2:下载

    3:解压在C:cygwinhadoop1

    4:配置

    hadoop1/conf下需要配置的文件:

    1).hadoop-env.sh

    2).core-site.xml

    3).hdfs-site.xml

    4).mapred-site.xml

    如何修改请参考 http://blog.csdn.net/wind520/article/details/9225429

    5:启动Hadoop

    第一步,在hadoop1目录下创建目录logs,用于保存日志
    第二步,Format a new distributed-filesystem: 格式化namenode,创建HDFS
    执行命令: bin/hadoop namenode -format

      开始没有格式化,导致打不开

    • NameNode - http://localhost:50070/
      lenovo@lenovo-PC ~
      $ cd ..
      
      lenovo@lenovo-PC /home
      $ cd hadoop1
      -bash: cd: hadoop1: No such file or directory
      
      lenovo@lenovo-PC /home
      $ cd ../hadoop1
      
      lenovo@lenovo-PC /hadoop1
      $ bin/hadoop namenode -format
      [INFO ][mgmnt  ] Local JMX connector started
      13/07/03 15:44:30 INFO namenode.NameNode: STARTUP_MSG:
      /************************************************************
      STARTUP_MSG: Starting NameNode
      STARTUP_MSG:   host = lenovo-PC/10.6.3.180
      STARTUP_MSG:   args = [-format]
      STARTUP_MSG:   version = 1.1.2
      STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/b ranch-1.1 -r 1440782; compiled by 'hortonfo' on Thu Jan 31 02:03:24 UTC 2013
      ************************************************************/
      Re-format filesystem in 	mphadoop-lenovodfs
      ame ? (Y or N) Y
      13/07/03 15:44:35 INFO util.GSet: VM type       = 32-bit
      13/07/03 15:44:35 INFO util.GSet: 2% max memory = 20.0 MB
      13/07/03 15:44:35 INFO util.GSet: capacity      = 2^22 = 4194304 entries
      13/07/03 15:44:35 INFO util.GSet: recommended=4194304, actual=4194304
      13/07/03 15:44:35 INFO namenode.FSNamesystem: fsOwner=lenovo
      13/07/03 15:44:35 INFO namenode.FSNamesystem: supergroup=supergroup
      13/07/03 15:44:35 INFO namenode.FSNamesystem: isPermissionEnabled=true
      13/07/03 15:44:35 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
      13/07/03 15:44:35 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessK eyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
      13/07/03 15:44:35 INFO namenode.NameNode: Caching file names occuring more than  10 times
      13/07/03 15:44:36 INFO common.Storage: Image file of size 112 saved in 0 seconds .
      13/07/03 15:44:36 INFO namenode.FSEditLog: closing edit log: position=4, editlog =	mphadoop-lenovodfs
      amecurrentedits
      13/07/03 15:44:36 INFO namenode.FSEditLog: close success: truncate to 4, editlog =	mphadoop-lenovodfs
      amecurrentedits
      13/07/03 15:44:36 INFO common.Storage: Storage directory 	mphadoop-lenovodfs name has been successfully formatted.
      13/07/03 15:44:36 INFO namenode.NameNode: SHUTDOWN_MSG:
      /************************************************************
      SHUTDOWN_MSG: Shutting down NameNode at lenovo-PC/10.6.3.180
      ************************************************************/

      第三步,启动Hadoop,执行命令: bin/start-all.sh

      需要输入3次密码

      lenovo@lenovo-PC /hadoop1
      $ bin/start-all.sh
      starting namenode, logging to /hadoop1/libexec/../logs/hadoop-lenovo-namenode-lenovo-PC.out
      lenovo@localhost's password:
      localhost: starting datanode, logging to /hadoop1/libexec/../logs/hadoop-lenovo-datanode-lenovo-PC.out
      lenovo@localhost's password:
      localhost: starting secondarynamenode, logging to /hadoop1/libexec/../logs/hadoop-lenovo-secondarynamenode-lenovo-PC.out
      starting jobtracker, logging to /hadoop1/libexec/../logs/hadoop-lenovo-jobtracker-lenovo-PC.out
      lenovo@localhost's password:
      localhost: starting tasktracker, logging to /hadoop1/libexec/../logs/hadoop-lenovo-tasktracker-lenovo-PC.out
      
      lenovo@lenovo-PC /hadoop1
      $ ps
            PID    PPID    PGID     WINPID   TTY     UID    STIME COMMAND
          59592       1   42744      60224  pty0    1000 15:45:31 /cygdrive/d/java/Tomcat6/jdk/bin/java
          58584   58340   58584      58612  pty0    1000 15:43:22 /usr/bin/bash
          62296       1   42744      63540  pty0    1000 15:45:48 /cygdrive/d/java/Tomcat6/jdk/bin/java
          58844   58584   58844      58936  pty0    1000 15:54:51 /usr/bin/ps
          62812       1   62208      63448  ?       1000 15:45:46 /cygdrive/d/java/Tomcat6/jdk/bin/java
          58340       1   58340      58340  ?       1000 15:43:22 /usr/bin/mintty
      
      lenovo@lenovo-PC /hadoop1
      $


      hadoop只运行3个JVM进程。


       hadoop默认的NameNode 和 JobTracker

  • 相关阅读:
    什么是shell
    Jenkins+python+selenium持续继承自动化测试
    selenium+python自动化
    产品和项目的概念
    继承与派生:赋值兼容规则(转)
    继承与派生:虚基类及其派生类的构造函数(转)
    重载函数与函数模板(转)
    继承与派生:作用域分辨符(转)
    作用域和可见性(转)
    继承与派生:派生类的析构函数(转)
  • 原文地址:https://www.cnblogs.com/dyllove98/p/3170221.html
Copyright © 2011-2022 走看看