zoukankan      html  css  js  c++  java
  • Hive 执行查询语句报错,由于内存空间不足导致

    org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException): Cannot create directory /tmp/hive/root/5a62374a-9509-4d20-befb-de0e00c0e7ad/hive_2019-06-14_11-12-24_067_6655769013562582754-1/-mr-10016. Name node is in safe mode.
    Resources are low on NN. Please add or free up more resources then turn off safe mode manually. NOTE:  If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1366)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4258)
            at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4233)
            at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:853)
            at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:600)
            at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
            at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
            at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:975)
            at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040)
            at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2036)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:422)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
            at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2034)
    
            at org.apache.hadoop.ipc.Client.call(Client.java:1470)
            at org.apache.hadoop.ipc.Client.call(Client.java:1401)
            at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
            at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
            at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:539)
            at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:498)
            at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
            at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
            at com.sun.proxy.$Proxy12.mkdirs(Unknown Source)
            at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2742)
            at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2713)
            at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:870)
            at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:866)
            at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
            at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:866)
            at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:859)
            at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1819)
            at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:227)
            at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
            at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
            at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
            at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1676)
            at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1435)
            at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1218)
            at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1082)
            at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)
            at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
            at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
            at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
            at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
            at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
            at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:498)
            at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
            at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
    Error launching map-reduce job
    FAILED: Execution Error, return code 5 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
    

      提示说是因为远程问题,直接给出的提示即可。

    NOTE:  If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.
    

      已经告诉我们,使用

    hdfs dfsadmin -safemode leave
    

      将安全模型关闭,但是关闭安全模式之后发现再次执行同样的语句还是得到同样的结果,查阅资料,是因为内存空间不足导致的。

    # 使用 df -hl 查看空间使用情况
    df -hl
    
    # 使用 du -sh /* 查看哪些大文件占用了空间
    du -sh /*
    
    # 将对应的内存空间释放之后,问题就解决了。
    

      

  • 相关阅读:
    Brain network involved in autonomic functions 与自主功能相关的大脑网络
    Brief summary of classical components of ERP 事件相关成分(ERP)经典成分小结
    ICA & Percentage Variance Account For (PVAF)
    数据处理中白化Whitening的作用图解分析
    Loadings vs eigenvectors in PCA 主成分分析(PCA)中的负荷和特征向量
    主成分分析(PCA)和独立成分分析(ICA)相关资料
    Sketch of heart and QRS complex 心脏及QRS波群简图
    Brain Network visulation in EEG 脑电网络可视化
    Phase Locking Value (PLV) 神经信号的锁相值
    ubuntu16.04下的一些基本操作笔记
  • 原文地址:https://www.cnblogs.com/hanwen1014/p/11022435.html
Copyright © 2011-2022 走看看