zoukankan      html  css  js  c++  java
  • MapReduce 程序运行报错 java.lang.ClassNotFoundException解决方法

    在创建自定义的Mapper时候,编译正确,但上传到集群执行时出现错误:

    11/16/05 22:53:16 INFO mapred.JobClient: Task Id : attempt_201111301626_0015_m_000000_0, Status : FAILED
    java.lang.RuntimeException: java.lang.ClassNotFoundException: actiondemo.MyJob$MapClass
            at org.apache.Hadoop.conf.Configuration.getClass(Configuration.java:866)
            at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:199)
            at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:719)
            at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
            at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:396)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
            at org.apache.hadoop.mapred.Child.main(Child.java:249)
    Caused by: java.lang.ClassNotFoundException: actiondemo.MyJob$MapClass
            at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
            at java.security.AccessController.doPrivileged(Native Method)
            at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
            at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
            at java.lang.Class.forName0(Native Method)
            at java.lang.Class.forName(Class.java:247)
            at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)
            at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:864)
            ... 8 more

      程序开发的步骤是 在Windows的eclipse下开发,然后倒出JAR包,将JAR包放在集群上运行。

      eclipse的jdk的版本不能高于hadoop里面配置的jdk的版本,否则会报错。(具体是什么类型的错误,记不到了)

      问题原因及解决办法如下:

      因为使用的是0.20以上的Hadoop版本,在调用jar中的自定义mapper时,需要设置setJarByClass方法,设置方法如下:

      job.setJarByClass(MyJob.class);

      其实,在输出日志中也有提示信息:
      11/16/05 22:53:03 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
    job.setJarByClass(MyJob.class);

  • 相关阅读:
    image绑定bitmap( Binding Image.Source from download memory)
    c# 调用控制台程序并传参 获取控制台输出(解释器的前台处理)
    varchar和Nvarchar区别
    Response.Redirect:无法在发送 HTTP 标头之后进行重定向
    href="#"与href="javascript:void(0)"的区别
    CSS样式表的优先级别
    若要允许 GET 请求,请将 JsonRequestBehavior 设置为 AllowGet
    Jquery下Json数据的传递与解析(asp.net mvc与asp.net api下后台json接收方式的不同)
    knockout.js: Uncaught Error: No Modification Allowed Err
    timepicker, datepicker ,datetimepicker
  • 原文地址:https://www.cnblogs.com/wujing-hubei/p/6032626.html
Copyright © 2011-2022 走看看