zoukankan      html  css  js  c++  java
  • Hadoop HDFS 单节点部署方案

    初学者,再次记录一下。

    确保Java 和 Hadoop已安装完毕(每个人的不一定一样,但肯定都有数据,仅供参考)

    [root@jans hadoop-2.9.0]# pwd
    /usr/local/hadoop-2.9.0
    [root@jans hadoop-2.9.0]# echo $JAVA_HOME
    /usr/
    [root@jans hadoop-2.9.0]# java -version
    openjdk version "1.8.0_161"
    OpenJDK Runtime Environment (build 1.8.0_161-b14)
    OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode)
    [root@jans hadoop-2.9.0]# echo $HADOOP_HOME
    /usr/local/hadoop-2.9.0
    [root@jans hadoop-2.9.0]# ./bin/hadoop version
    Hadoop 2.9.0
    Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r 756ebc8394e473ac25feac05fa493f6d612e6c50
    Compiled by arsuresh on 2017-11-13T23:15Z
    Compiled with protoc 2.5.0
    From source with checksum 0a76a9a32a5257331741f8d5932f183
    This command was run using /usr/local/hadoop-2.9.0/share/hadoop/common/hadoop-common-2.9.0.jar
    Java and Hadoop 安装状态查看

    hadoop根目录:

      /usr/local/hadoop-2.9.0/

    确保当前工作目录处于Hadoop根目录

    1:编辑配置文件
    编辑文件:/usr/local/hadoop-2.9.0/etc/hadoop/core-site.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!--
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at
    
        http://www.apache.org/licenses/LICENSE-2.0
    
      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License. See accompanying LICENSE file.
    -->
    
    <!-- Put site-specific property overrides in this file. -->
    
    <configuration>
        <property>
          <name>hadoop.tmp.dir</name>
          <value>file:/home/hadoop/tmp</value>
          <description>Abase for other temporary directories.</description>
        </property>
        <property>
          <name>fs.defaultFS</name>
          <value>hdfs://localhost:9000</value>
        </property>
    </configuration>

    编辑文件:/usr/local/hadoop-2.9.0/etc/hadoop/hdfs-site.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!--
      Licensed under the Apache License, Version 2.0 (the "License");
      you may not use this file except in compliance with the License.
      You may obtain a copy of the License at
    
        http://www.apache.org/licenses/LICENSE-2.0
    
      Unless required by applicable law or agreed to in writing, software
      distributed under the License is distributed on an "AS IS" BASIS,
      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      See the License for the specific language governing permissions and
      limitations under the License. See accompanying LICENSE file.
    -->
    
    <!-- Put site-specific property overrides in this file. -->
    
    <configuration>
        <property>
          <name>dfs.replication</name>
          <value>1</value>
        </property>
        <property>
          <name>dfs.namenode.name.dir</name>
          <value>file:/home/hadoop/tmp/dfs/name</value>
        </property>
        <property>
          <name>dfs.datanode.data.dir</name>
          <value>file:/home/hadoop/tmp/dfs/data</value>
        </property> 
     
    </configuration>

    2:格式化节点

    ./bin/hadoop namenode -format
    

    3:启动服务

     ./sbin/start-all.sh 

    此操作会启动两个服务:

      This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh

    4:状态查看

    http://{IP address}:50070 查看概况

    http://{IP address}:50075 查看Node Status

    http://{IP address}:8088 查看Hadoop 集群状态

    如果看不到数据则部署失败

    其他端口查看

  • 相关阅读:
    ubuntu的apt
    sudo命令
    MySQL导出数据到csv文件
    MySQL导出数据到文件报错
    git_backup.py gitlab项目备份
    java中图像与数组转换
    mongodb转elasticsearch
    impyla-0.14.2.2安装注意事项
    python3.7.3升级 with-openssl openssl-1.0.2a
    hadoop自带性能测试
  • 原文地址:https://www.cnblogs.com/JansXin/p/8869104.html
Copyright © 2011-2022 走看看