zoukankan      html  css  js  c++  java
  • NoClassDefFoundError问题解决

      首先推荐一篇博客http://blog.csdn.net/jamesjxin/article/details/46606307

    为我最初解决问题提供了入口。

      问题场景:

      由于我跑mapreduce任务需要按时间取数据,然而数据表是根据其他字段分区的,修改分区的话对现有的业务影响太大,于是改为跑任务之前先把表中的数据迁出,然后map迁出的hive表,来实现既定业务。

      问题如下:

      

    $ ./UpdateUrlStatus.sh
    当前时间零点:|1490976000000
    17/04/01 13:29:32 INFO util.HiveClient: 加载参数完毕
    17/04/01 13:29:33 INFO util.HiveClient: 加载驱动完毕=========org.apache.hive.jdbc.HiveDriver
    Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class hadooputils.util.HiveClient
        at mapreduce.avrocombine.UpdateUrlStatus.main(UpdateUrlStatus.java:189)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

    程序中的问题代码:

            
    main:
        try{ if(HiveClient.createTable(parseTable, "exceptionparseall")){ HiveClient.insertDataToParse(parseTable, startTime, endTime); System.out.println("更新解析表数据完毕"); } if(HiveClient.createTable(extraceTable, "exceptionextractall")){ HiveClient.insertDataToExtract(extraceTable, startTime, endTime); System.out.println("更新抽取表数据完毕"); } } catch (SQLException e){ System.out.println(e.getMessage()); } catch (Exception e){ System.out.println(e.getMessage()); } finally { HiveClient.close(); }
    public class HiveClient {
        private static String sql = "";  
        private static Statement stmt;
        private static Connection conn;
        private static ResultSet rs;
        private static final Logger log = Logger.getLogger(HiveClient.class);  
        
        static{
            if(conn==null||stmt==null){
                init();
            }
        }
        
        private static void init(){
            try {
                LoadProperty lp = new LoadProperty();
                Properties pro = lp.loadConfiguration("/hive.properties");
                String driverName = pro.getProperty("hive.driverName");
                String url = pro.getProperty("hive.url");
                String user = pro.getProperty("hive.user");
                String password = pro.getProperty("hive.password");
                log.info("加载参数完毕");
                Class.forName(driverName);
                log.info("加载驱动完毕=========" + driverName);
                conn = DriverManager.getConnection(url, user, password);
                log.info("建立连接完毕=========" + conn);
    //            conn.setAutoCommit(false);
                stmt = conn.createStatement();
                log.info("Statement创建完毕");
            } catch (ClassNotFoundException e) {
                log.info("找不到驱动类");
            } catch (SQLException e) {
                log.info("初始化失败");
            } catch (Exception e){
                log.info("初始化异常" + e.getMessage());
            }
        }
        
        public static void close(){
            try {
                if (rs != null) {  
                    rs.close();  
                    rs = null;  
                }  
                if (stmt != null) {  
                      stmt.close();  
                      stmt = null;  
                  }  
                if (conn != null) {  
                    conn.close();  
                    conn = null;  
                }  
              
            } catch (SQLException e) {  
                e.printStackTrace();  
            }  
        }
    ...
    }

      由博客中的介绍可以知道,NoClassDefFoundError也可能由于类的静态初始化模块错误导致,当你的类执行一些静态初始化模块操作,如果初始化模块抛出异常,哪些依赖这个类的其他类会抛出NoClassDefFoundError的错误。如果你查看程序日志,会发现一些java.lang.ExceptionInInitializerError的错误日志,ExceptionInInitializerError的错误会导致java.lang.NoClassDefFoundError: Could not initialize class。

      于是在静态块中的init方法里加上日志,发现不是配置文件和驱动的问题,定位在获取连接的语句上。然而main方法中并没有把异常catch住,只是报了一个java.lang.NoClassDefFoundError的错误。

    经过修改代码,把静态块代码注释掉,init方法改为public,并在main方法中调用,然后就发现错误的日志改变了,如下:

    $ ./UpdateUrlStatus.sh
    当前时间零点:|1490976000000
    17/04/01 14:36:45 INFO util.HiveClient: 加载参数完毕
    17/04/01 14:36:45 INFO util.HiveClient: 加载驱动完毕=========org.apache.hive.jdbc.HiveDriver
    Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hive/service/cli/thrift/TCLIService$Iface
        at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
        at java.sql.DriverManager.getConnection(DriverManager.java:571)
        at java.sql.DriverManager.getConnection(DriverManager.java:215)
        at hadooputils.util.HiveClient.init(HiveClient.java:37)
        at mapreduce.avrocombine.UpdateUrlStatus.main(UpdateUrlStatus.java:178)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
    Caused by: java.lang.ClassNotFoundException: org.apache.hive.service.cli.thrift.TCLIService$Iface
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

    这样就找到了问题真正的源头,再把用于找问题的代码修改回去,OK!

  • 相关阅读:
    204. Count Primes (Integer)
    203. Remove Linked List Elements (List)
    202. Happy Number (INT)
    201. Bitwise AND of Numbers Range (Bit)
    200. Number of Islands (Graph)
    199. Binary Tree Right Side View (Tree, Stack)
    198. House Robber(Array; DP)
    191. Number of 1 Bits (Int; Bit)
    190. Reverse Bits (Int; Bit)
    189. Rotate Array(Array)
  • 原文地址:https://www.cnblogs.com/superJF/p/6656209.html
Copyright © 2011-2022 走看看