zoukankan      html  css  js  c++  java
  • 【Flink系列五】Flink-hive-connector 使用的HiveMetastoreClient不支持Kerberos代理

    Flink-hive-connector

    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-connector-hive_${scala.binary.version}</artifactId>
        <version>${flink.version}</version>
        <scope>compile</scope>
    </dependency>
    

    由于目前项目中连接的是Hive 1.1.0,而HADOOP_PROXY_USER是 2.3.0才支持的 HIVE-COMMIT

    可以看到HiveMetaStoreClient.java中增加了如下代码:

    
        //If HADOOP_PROXY_USER is set in env or property,
        //then need to create metastore client that proxies as that user.
        String HADOOP_PROXY_USER = "HADOOP_PROXY_USER";
        String proxyUser = System.getenv(HADOOP_PROXY_USER);
        if (proxyUser == null) {
          proxyUser = System.getProperty(HADOOP_PROXY_USER);
        }
        //if HADOOP_PROXY_USER is set, create DelegationToken using real user
        if(proxyUser != null) {
          LOG.info(HADOOP_PROXY_USER + " is set. Using delegation "
              + "token for HiveMetaStore connection.");
          try {
            UserGroupInformation.getLoginUser().getRealUser().doAs(
                new PrivilegedExceptionAction<Void>() {
                  @Override
                  public Void run() throws Exception {
                    open();
                    return null;
                  }
                });
            String delegationTokenPropString = "DelegationTokenForHiveMetaStoreServer";
            String delegationTokenStr = getDelegationToken(proxyUser, proxyUser);
            Utils.setTokenStr(UserGroupInformation.getCurrentUser(), delegationTokenStr,
                delegationTokenPropString);
            this.conf.setVar(ConfVars.METASTORE_TOKEN_SIGNATURE, delegationTokenPropString);
            close();
          } catch (Exception e) {
            LOG.error("Error while setting delegation token for " + proxyUser, e);
            if(e instanceof MetaException) {
              throw (MetaException)e;
            } else {
              throw new MetaException(e.getMessage());
            }
          }
        }
    
  • 相关阅读:
    C. Tavas and Karafs 二分查找+贪心
    字符串前缀和后缀匹配
    E
    网络流最经典的入门题 各种网络流算法都能AC。 poj 1273 Drainage Ditches
    B
    Dividing 多重背包 倍增DP
    Doing Homework 状态压缩DP
    Largest Rectangle in a Histogram 常用技巧 stack的运用
    S = A + A2 + A3 + … + Ak. (推矩阵公式+矩阵幂运算)
    安卓开发:图片的显示Mode
  • 原文地址:https://www.cnblogs.com/slankka/p/14250440.html
Copyright © 2011-2022 走看看