zoukankan      html  css  js  c++  java
  • 17、【SparkSQL】org.apache.hadoop.security.HadoopKerberosName.setRuleMechanism异常

    • 背景
      当我在使用IDEA本地连接服务器spark服务时,虽然拷贝了hive-site.xml,提供了相应的依赖,仍然报错:org.apache.hadoop.security.HadoopKerberosName.setRuleMechanism
      依赖如下:

        <dependencies>
            <dependency>
                <groupId>org.apache.spark</groupId>
                <artifactId>spark-core_2.12</artifactId>
                <version>3.1.2</version>
            </dependency>
            <dependency>
                <groupId>org.apache.spark</groupId>
                <artifactId>spark-sql_2.12</artifactId>
                <version>3.1.2</version>
            </dependency>
      
            <dependency>
                <groupId>mysql</groupId>
                <artifactId>mysql-connector-java</artifactId>
                <version>8.0.25</version>
            </dependency>
      
            <dependency>
                <groupId>org.apache.hive</groupId>
                <artifactId>hive-exec</artifactId>
                <version>3.1.2</version>
            </dependency>
      
            <dependency>
                <groupId>org.apache.spark</groupId>
                <artifactId>spark-hive_2.12</artifactId>
                <version>3.1.2</version>
            </dependency>
      
            <dependency>
                <groupId>org.apache.hadoop</groupId>
                <artifactId>hadoop-auth</artifactId>
                <version>3.2.0</version>
            </dependency>
        </dependencies>
      
      
    • 分析
      在远程访问hadoop时,需要提供认证信息

      https://hadoop.apache.org/docs/stable/hadoop-auth/Examples.html
      https://blog.csdn.net/csr_hema/article/details/8147590

    • 措施
      在Maven添加如下依赖:

              <dependency>
                <groupId>org.apache.hadoop</groupId>
                <artifactId>hadoop-auth</artifactId>
                <version>3.2.0</version>
            </dependency>
      
  • 相关阅读:
    网页导出PDF文件
    图片翻转导航
    瀑布流之ajax
    楼梯效果
    数码时钟
    数字字母随机验证码
    kafka shell
    zookeeper shell
    正则
    Linux(一)
  • 原文地址:https://www.cnblogs.com/nuochengze/p/15655531.html
Copyright © 2011-2022 走看看