zoukankan      html  css  js  c++  java
  • Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.security.HadoopKerberosName.setRuleMechanism(Ljava/lang/String;)V

    启动spark-shell时,报错如下:

    Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.security.HadoopKerberosName.setRuleMechanism(Ljava/lang/String;)V
        at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:84)
        at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:318)
        at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:303)
        at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1827)
        at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:709)
        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:659)
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:570)
        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2422)
        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2422)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2422)
        at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:79)
        at org.apache.spark.deploy.SparkSubmit.secMgr$lzycompute$1(SparkSubmit.scala:348)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$secMgr$1(SparkSubmit.scala:348)
        at org.apache.spark.deploy.SparkSubmit$$anonfun$prepareSubmitEnvironment$7.apply(SparkSubmit.scala:356)
        at org.apache.spark.deploy.SparkSubmit$$anonfun$prepareSubmitEnvironment$7.apply(SparkSubmit.scala:356)
        at scala.Option.map(Option.scala:146)
        at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:355)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:784)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:930)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:939)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

    解决方法:

    这是由于错误的配置导致了hadoop的包和spark的包冲突。

    原来是不知道啥时候修改了spark-defaults.conf中的配置。

    所以解决方法也是很简单,删除这个配置就可以。

    作者:尤灯塔
    本文版权归作者和博客园共有,欢迎转载,但未经作者同意必须保留此段声明,且在文章页面明显位置给出原文连接,否则保留追究法律责任的权利.
  • 相关阅读:
    MyBatis学习总结(四)——解决字段名与实体类属性名不相同的冲突
    MyBatis学习总结(三)——优化MyBatis配置文件中的配置
    MyBatis学习总结(二)——使用MyBatis对表执行CRUD操作
    MyBatis学习总结(一)——MyBatis快速入门
    web开发工具IDE
    mysql可视化管理工具
    mysql慢查询
    optimize table
    yii2 安装
    JavaScript中将JSON的字符串解析成JSON数据格式
  • 原文地址:https://www.cnblogs.com/30go/p/15054352.html
Copyright © 2011-2022 走看看