zoukankan      html  css  js  c++  java
  • Error- spark streaming 打包将全部依赖打进去Invalid signature file digest for Manifest main attributes

    spark streaming 打包将全部依赖打进去,运行jar包报错:如下

    Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
    PS D:workspaceideaProjectjavawordcountoutartifactswordcount_jar> spark-submit .wordcount.jar localhost:9092 test
    
    2019-08-23 15:37:18 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
            at sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:284)
            at sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:238)
            at java.util.jar.JarVerifier.processEntry(JarVerifier.java:316)
            at java.util.jar.JarVerifier.update(JarVerifier.java:228)
            at java.util.jar.JarFile.initializeVerifier(JarFile.java:383)
            at java.util.jar.JarFile.getInputStream(JarFile.java:450)
            at sun.misc.JarIndex.getJarIndex(JarIndex.java:137)
            at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:839)
            at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:831)
            at java.security.AccessController.doPrivileged(Native Method)
            at sun.misc.URLClassPath$JarLoader.ensureOpen(URLClassPath.java:830)
            at sun.misc.URLClassPath$JarLoader.<init>(URLClassPath.java:803)
            at sun.misc.URLClassPath$3.run(URLClassPath.java:530)
            at sun.misc.URLClassPath$3.run(URLClassPath.java:520)
            at java.security.AccessController.doPrivileged(Native Method)
            at sun.misc.URLClassPath.getLoader(URLClassPath.java:519)
            at sun.misc.URLClassPath.getLoader(URLClassPath.java:492)
            at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:457)
            at sun.misc.URLClassPath.getResource(URLClassPath.java:211)
            at java.net.URLClassLoader$1.run(URLClassLoader.java:365)
            at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
            at java.security.AccessController.doPrivileged(Native Method)
            at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
            at java.lang.Class.forName0(Native Method)
            at java.lang.Class.forName(Class.java:348)
            at org.apache.spark.util.Utils$.classForName(Utils.scala:239)
            at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:851)
            at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
            at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
            at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
            at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
    

     解决方式:只将有需要的包打进去,因为spark的lib下已经有很多jar包,不需要我们重复打进jar包,只需要把lib没有的jar包打进去运行就好。因为包的重复引用会导致生成多个MF文件

  • 相关阅读:
    springboot 获取enviroment.Properties的几种方式
    Spring Boot加载配置文件的完整步骤
    SpringBoot测试Controller层
    SpringBoot集成测试
    PowerMockito模拟private static final类变量
    玩转Spring JUnit+mockito+powermock单元测试
    Junit测试Controller(MockMVC使用),传输@RequestBody数据解决办法
    spring boot 使用@ConfigurationProperties加载配置文件到类中
    Rsync命令参数详解
    Linux PS 命令详解
  • 原文地址:https://www.cnblogs.com/RzCong/p/11400542.html
Copyright © 2011-2022 走看看