zoukankan      html  css  js  c++  java
  • SparkSql执行hive插入警告Unable to inherit permissions

    执行sparksql,插入hive 时候提示以下警告:

    2021-04-03 00:22:02 WARN  [org.apache.hadoop.hive.shims.HadoopShimsSecure setFullFileStatus():869]- Unable to inherit permissions for file hdfs://xxxx:8020/user/hive/warehouse/pdw_ls.db/relatives_ztry/part-00000 
    from file hdfs:
    //xxxx:8020/user/hive/warehouse/pdw_ls.db/relatives_ztry Permission denied. user=ccccc is not the owner of inode=part-00000 at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkOwner(DefaultAuthorizationProvider.java:188) at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:174) at org.apache.sentry.hdfs.SentryAuthorizationProvider.checkPermission(SentryAuthorizationProvider.java:194) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:152) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:3877) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:3860) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkOwner(FSDirectory.java:3825) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOwner(FSNamesystem.java:6784) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setAcl(FSNamesystem.java:9296) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setAcl(NameNodeRpcServer.java:1642) at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.setAcl(AuthorizationProviderProxyClientProtocol.java:902) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setAcl(ClientNamenodeProtocolServerSideTranslatorPB.java:1347) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2281) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2277) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2275) 2021-04-03 00:22:02 WARN [org.apache.hadoop.hive.shims.HadoopShimsSecure setFullFileStatus():869]- Unable to inherit permissions for file hdfs://xxxx:8020/user/hive/warehouse/pdw_ls.db/relatives_ztry/part-00004 from file hdfs://xxxx:8020/user/hive/warehouse/pdw_ls.db/relatives_ztry Permission denied. user=ccccc is not the owner of inode=part-00004 at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkOwner(DefaultAuthorizationProvider.java:188) at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:174) at org.apache.sentry.hdfs.SentryAuthorizationProvider.checkPermission(SentryAuthorizationProvider.java:194) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:152) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:3877) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:3860)

    处理方式:

    设置hive参数 hive.warehouse.subdir.inherit.perms:false,关闭hive的文件权限继承来规避该问题

    SparkConf sparkConf = new SparkConf();
    SparkSession session= SparkSession
                    .builder()
                    .config(sparkConf)
                    .config("hive.warehouse.subdir.inherit.perms","false")
                    .enableHiveSupport()
                    .getOrCreate();
  • 相关阅读:
    通过运用CSS框架学到了什么
    浏览器判断一览表
    默认样式表之HTML4
    MooTools 1.4 源码分析 -overloadSetter
    MooTools 1.4 源码分析
    MooTools 1.4 源码分析
    javascript一些底层方法总结及用法
    viewport移动端适配,读文笔记
    vue获取v-model数据类型boolean改变成string
    contextify::ContextifyScript::New(const v8::FunctionCallbackInfo<v8::Value>&):
  • 原文地址:https://www.cnblogs.com/linkmust/p/14626623.html
Copyright © 2011-2022 走看看