zoukankan      html  css  js  c++  java
  • windows 安装hadoop 3.2.1

    问题:

    1、

    Java installed version: 1.8.0_231 Hadoop installed version: Hadoop 3.2.1

    But after running command hdfs namenode -format I'm getting following error

    Re-format filesystem in Storage Directory root= C:hadoop-3.2.1data
    amenode; location= null ? (Y or N) y
    2019-10-19 12:34:58,809 INFO namenode.FSImage: Allocated new BlockPoolId: BP-1445655329-172.18.148.177-1571468698797
    2019-10-19 12:34:58,809 INFO common.Storage: Will remove files: []
    2019-10-19 12:34:58,812 ERROR namenode.NameNode: Failed to start namenode.java.lang.UnsupportedOperationException
        at java.nio.file.Files.setPosixFilePermissions(Files.java:2044)
        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:452)
        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591)
        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1649)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1759)
    2019-10-19 12:34:58,819 INFO util.ExitUtil: Exiting with status 1: java.lang.UnsupportedOperationException
    2019-10-19 12:34:58,823 INFO namenode.NameNode: SHUTDOWN_MSG:
    /************************************************************
    SHUTDOWN_MSG: Shutting down NameNode at CWT-DST-0051/***.**.***.***

    解决办法:
    https://kontext.tech/column/hadoop/379/fix-for-hadoop-321-namenode-format-issue-on-windows-10
    Root cause
    Refer to the following official issue tracker about the details of the root cause:
    
    HDFS-14890
    
    This issue will only be fixed on Hadoop 3.3.0 or 3.2.0 releases.
    
    If you want to install 3.2.1, follow instructions below.
    
    Resolution
    I've provided detailed steps about installing Hadoop 3.2.1 on Windows 10  in the following article with fix about this issue:
    
    Latest Hadoop 3.2.1 Installation on Windows 10 Step by Step Guide
    
    Fix the issue only
    I've uploaded the updated JAR file into the following location. Please download it from the following link:
    
    https://github.com/FahaoTang/big-data/blob/master/hadoop-hdfs-3.2.1.jar  

    And then rename the file name hadoop-hdfs-3.2.1.jar to hadoop-hdfs-3.2.1.bk in folder %HADOOP_HOME%sharehadoophdfs.

    Copy the downloaded hadoop-hdfs-3.2.1.jar to folder %HADOOP_HOME%sharehadoophdfs.

    warning This is just a temporary fix before the official fix is published. There is no guarantee this temporary fix won't cause any new issue. 

    Refer to this article for more details about how to build a native Windows Hadoop: Compile and Build Hadoop 3.2.1 on Windows 10 Guide.

    If you have any other questions, feel free to add a comment. 

     
     
  • 相关阅读:
    第九次训练赛
    什么是 Catalan 数列以及其应用
    Python pip 安装与使用
    HDU 1179:Ollivanders: Makers of Fine Wands since 382 BC.
    身份证信息
    流量暴增,掌门教育如何基于 Spring Cloud Alibaba 构建微服务体系?
    从零入门 Serverless | 函数计算的可观测性
    如何管理越来越多的 operator?OLM 给你答案
    Fluid: 让大数据和 AI 拥抱云原生的一块重要拼图
    SpringCloud 应用在 Kubernetes 上的最佳实践 — 线上发布(可监控)
  • 原文地址:https://www.cnblogs.com/cbugs/p/13995132.html
Copyright © 2011-2022 走看看