zoukankan      html  css  js  c++  java
  • azkaban-web-start.sh启动时出现Table 'execution_flows' is marked as crashed and should be repaired Query错误的解决办法(图文详解)

      问题详情

    [hadoop@master bin]$ ./azkaban-web-start.sh 
    Using Hadoop from /home/hadoop/app/hadoop-2.6.0
    Using Hive from /home/hadoop/app/hive
    ./..
    /home/hadoop/app/jdk/lib:.:/home/hadoop/app/jdk/lib:/home/hadoop/app/jdk/jre/lib:./../lib/azkaban-2.5.0.jar:./../lib/commons-collections-3.2.1.jar:./../lib/commons-configuration-1.8.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-email-1.2.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.1.jar:./../lib/commons-pool-1.6.jar:./../lib/guava-13.0.1.jar:./../lib/h2-1.3.170.jar:./../lib/httpclient-4.2.1.jar:./../lib/httpcore-4.2.1.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/junit-4.8.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.6.1.jar:./../lib/slf4j-log4j12-1.6.4.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/mysql-connector-java-5.1.21.jar:./../plugins/*/*.jar:/home/hadoop/app/hadoop-2.6.0/conf:/home/hadoop/app/hadoop-2.6.0/*:/home/hadoop/app/hive/conf:/home/hadoop/app/hive/lib/*
    [hadoop@master bin]$ 2017/07/26 09:06:39.979 +0800 ERROR [AzkabanWebServer] [Azkaban] Starting Jetty Azkaban Executor...
    2017/07/26 09:06:40.098 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban settings file from ./../conf
    2017/07/26 09:06:40.100 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban properties file
    2017/07/26 09:06:40.190 +0800 INFO [AzkabanWebServer] [Azkaban] Setting up Jetty Https Server with port:8443 and numThreads:25
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/home/hadoop/app/azkaban/azkaban-web-2.5.0/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/hadoop/app/hive-1.0.0/lib/hive-jdbc-1.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    2017/07/26 09:06:40.484 +0800 INFO [log] [Azkaban] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
    2017/07/26 09:06:41.013 +0800 INFO [AzkabanWebServer] [Azkaban] Loading user manager class azkaban.user.XmlUserManager
    2017/07/26 09:06:41.477 +0800 INFO [XmlUserManager] [Azkaban] Loading user azkaban
    2017/07/26 09:06:41.484 +0800 INFO [XmlUserManager] [Azkaban] Loading user metrics
    2017/07/26 09:06:41.485 +0800 INFO [XmlUserManager] [Azkaban] Loading user admin
    Exception in thread "main" azkaban.executor.ExecutorManagerException: Error fetching active flows
        at azkaban.executor.JdbcExecutorLoader.fetchActiveFlows(JdbcExecutorLoader.java:209)
        at azkaban.executor.ExecutorManager.loadRunningFlows(ExecutorManager.java:146)
        at azkaban.executor.ExecutorManager.<init>(ExecutorManager.java:84)
        at azkaban.webapp.AzkabanWebServer.loadExecutorManager(AzkabanWebServer.java:248)
        at azkaban.webapp.AzkabanWebServer.<init>(AzkabanWebServer.java:185)
        at azkaban.webapp.AzkabanWebServer.main(AzkabanWebServer.java:726)
    Caused by: java.sql.SQLException: Table 'execution_flows' is marked as crashed and should be repaired Query: SELECT ex.exec_id exec_id, ex.enc_type enc_type, ex.flow_data flow_data, ax.host host, ax.port port, ax.update_time axUpdateTime FROM execution_flows ex INNER JOIN active_executing_flows ax ON ex.exec_id = ax.exec_id Parameters: []
        at org.apache.commons.dbutils.AbstractQueryRunner.rethrow(AbstractQueryRunner.java:363)
        at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:350)
        at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:306)
        at azkaban.executor.JdbcExecutorLoader.fetchActiveFlows(JdbcExecutorLoader.java:202)
        ... 5 more

    问题分析

       错误产生原因,有网友说是频繁查询和更新dede_archives表造成的索引错误,因为我的页面没有静态生成,而是动态页面,因此比较同意这种说法。
      还有说法为是MYSQL数据库因为某种原因而受到了损坏,如:数据库服务器突发性的断电、在提在数据库表提供服务时对表的原文件进行某种操作都有可能导致
      MYSQL数据库表被损坏而无法读取数据。总之就是因为某些不可测的问题造成表的损坏。

       我这里,是在跑azkaban任务的时候,间隔取的太小,同时自己电脑配置不太高,导致,速度很慢和卡顿。我中途关闭了导致。

    解决办法

    [hadoop@master ~]$ mysql -uroot -prootroot
    Welcome to the MySQL monitor.  Commands end with ; or g.
    Your MySQL connection id is 24
    Server version: 5.1.73 Source distribution
    
    Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
    
    Oracle is a registered trademark of Oracle Corporation and/or its
    affiliates. Other names may be trademarks of their respective
    owners.
    
    Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
    
    mysql> show databases;
    +--------------------+
    | Database           |
    +--------------------+
    | information_schema |
    | azkaban            |
    | hive               |
    | mysql              |
    | test               |
    +--------------------+
    5 rows in set (0.09 sec)
    
    mysql> use azkaban
    Reading table information for completion of table and column names
    You can turn off this feature to get a quicker startup with -A
    
    Database changed
    mysql> show tables;
    +------------------------+
    | Tables_in_azkaban      |
    +------------------------+
    | active_executing_flows |
    | active_sla             |
    | execution_flows        |
    | execution_jobs         |
    | execution_logs         |
    | project_events         |
    | project_files          |
    | project_flows          |
    | project_permissions    |
    | project_properties     |
    | project_versions       |
    | projects               |
    | properties             |
    | schedules              |
    | triggers               |
    +------------------------+
    15 rows in set (0.00 sec)
    
    mysql> 

       然后,再,检测下这个表execution_flows表的状态情况,如下出现错误的状态是正确的,本来这个表就已经是不健康的了。

    mysql> check table execution_flows;
    +-------------------------+-------+----------+----------------------------------------------------------+
    | Table                   | Op    | Msg_type | Msg_text                                                 |
    +-------------------------+-------+----------+----------------------------------------------------------+
    | azkaban.execution_flows | check | warning  | 6 clients are using or haven't closed the table properly |
    | azkaban.execution_flows | check | error    | Wrong bytesec: 40-0-0 at linkstart: 53868                |
    | azkaban.execution_flows | check | error    | Corrupt                                                  |
    +-------------------------+-------+----------+----------------------------------------------------------+
    3 rows in set (0.40 sec)

      

      执行,修复

    mysql> repair table execution_flows;
    +-------------------------+--------+----------+-------------------------------------------------------+
    | Table                   | Op     | Msg_type | Msg_text                                              |
    +-------------------------+--------+----------+-------------------------------------------------------+
    | azkaban.execution_flows | repair | info     | Wrong bytesec:  40-  0-  0 at 53868; Skipped          |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 49260    |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 49472    |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 49476    |
    | azkaban.execution_flows | repair | info     | Found block with too small length at 49628; Skipped   |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 49812    |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 49824    |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 49876    |
    | azkaban.execution_flows | repair | info     | Wrong block with wrong total length starting at 49996 |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 50024    |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 50100    |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 50120    |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 50128    |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 50304    |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 50588    |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 50772    |
    | azkaban.execution_flows | repair | info     | Found block that points outside data file at 51380    |
    | azkaban.execution_flows | repair | warning  | Number of rows changed from 74 to 71                  |
    | azkaban.execution_flows | repair | status   | OK                                                    |
    +-------------------------+--------+----------+-------------------------------------------------------+
    19 rows in set (0.36 sec)

     

    mysql> check table execution_flows;
    +-------------------------+-------+----------+----------+
    | Table                   | Op    | Msg_type | Msg_text |
    +-------------------------+-------+----------+----------+
    | azkaban.execution_flows | check | status   | OK       |
    +-------------------------+-------+----------+----------+
    1 row in set (0.00 sec)

       然后,再

      

      再

    [hadoop@master bin]$ ./azkaban-web-start.sh 
    Using Hadoop from /home/hadoop/app/hadoop-2.6.0
    Using Hive from /home/hadoop/app/hive
    ./..
    /home/hadoop/app/jdk/lib:.:/home/hadoop/app/jdk/lib:/home/hadoop/app/jdk/jre/lib:./../lib/azkaban-2.5.0.jar:./../lib/commons-collections-3.2.1.jar:./../lib/commons-configuration-1.8.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-email-1.2.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.1.jar:./../lib/commons-pool-1.6.jar:./../lib/guava-13.0.1.jar:./../lib/h2-1.3.170.jar:./../lib/httpclient-4.2.1.jar:./../lib/httpcore-4.2.1.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/junit-4.8.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.6.1.jar:./../lib/slf4j-log4j12-1.6.4.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/mysql-connector-java-5.1.21.jar:./../plugins/*/*.jar:/home/hadoop/app/hadoop-2.6.0/conf:/home/hadoop/app/hadoop-2.6.0/*:/home/hadoop/app/hive/conf:/home/hadoop/app/hive/lib/*
    [hadoop@master bin]$ 2017/07/26 09:38:33.582 +0800 ERROR [AzkabanWebServer] [Azkaban] Starting Jetty Azkaban Executor...
    2017/07/26 09:38:33.726 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban settings file from ./../conf
    2017/07/26 09:38:33.748 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban properties file
    2017/07/26 09:38:33.865 +0800 INFO [AzkabanWebServer] [Azkaban] Setting up Jetty Https Server with port:8443 and numThreads:25
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/home/hadoop/app/azkaban/azkaban-web-2.5.0/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/hadoop/app/hive-1.0.0/lib/hive-jdbc-1.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    2017/07/26 09:38:34.222 +0800 INFO [log] [Azkaban] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
    2017/07/26 09:38:34.876 +0800 INFO [AzkabanWebServer] [Azkaban] Loading user manager class azkaban.user.XmlUserManager
    2017/07/26 09:38:35.260 +0800 INFO [XmlUserManager] [Azkaban] Loading user azkaban
    2017/07/26 09:38:35.274 +0800 INFO [XmlUserManager] [Azkaban] Loading user metrics
    2017/07/26 09:38:35.278 +0800 INFO [XmlUserManager] [Azkaban] Loading user admin
    2017/07/26 09:38:38.564 +0800 INFO [AzkabanWebServer] [Azkaban] Loading JDBC for project management
    2017/07/26 09:38:38.569 +0800 INFO [ExecutorManager] [Azkaban] Cleaning old logs from execution_logs
    2017/07/26 09:38:38.590 +0800 INFO [ProjectManager] [Azkaban] Project version retention is set to 3
    2017/07/26 09:38:39.402 +0800 INFO [ExecutorManager] [Azkaban] Cleaning old log files before 2017-05-03T09:38:38.659+08:00
    2017/07/26 09:38:39.532 +0800 INFO [TriggerManager] [Azkaban] TriggerManager loaded.
    2017/07/26 09:38:39.532 +0800 INFO [AzkabanWebServer] [Azkaban] Loading built-in checker and action types
    2017/07/26 09:38:39.560 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker BasicTimeChecker
    2017/07/26 09:38:39.560 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker SlaChecker
    2017/07/26 09:38:39.561 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker ExecutionChecker

     

    扩展

      网上还有说,

    Table '' is marked as crashed and should be repaired 解决方法

    解决方法:

    找到mysql的安装目录的bin/myisamchk工具,在命令行中输入:

    myisamchk -c -r ../data/mysql/user.MYI

    然后myisamchk 工具会帮助你恢复数据表的索引。重新启动mysql,问题解决。

      
       这种方法,我没尝试过。
     
     
     
     
  • 相关阅读:
    Atitit s2018.6 s6 doc list on com pc.docx Atitit s2018.6 s6 doc list on com pc.docx  Aitit algo fix 算法系列补充.docx Atiitt 兼容性提示的艺术 attilax总结.docx Atitit 应用程序容器化总结 v2 s66.docx Atitit file cms api
    Atitit s2018.5 s5 doc list on com pc.docx  v2
    Atitit s2018.5 s5 doc list on com pc.docx  Acc 112237553.docx Acc baidu netdisk.docx Acc csdn 18821766710 attilax main num.docx Atiitt put post 工具 开发工具dev tool test.docx Atiitt 腾讯图像分类相册管家.docx
    Atitit s2018 s4 doc list dvchomepc dvccompc.docx .docx s2018 s4 doc compc dtS44 s2018 s4 doc dvcCompc dtS420 s2018 s4f doc homepc s2018 s4 doc compc dtS44(5 封私信 _ 44 条消息)WebSocket 有没有可能取代 AJAX
    Atitit s2018 s3 doc list alldvc.docx .docx s2018 s3f doc compc s2018 s3f doc homepc sum doc dvcCompc dtS312 s2018 s3f doc compcAtitit PathUtil 工具新特性新版本 v8 s312.docx s2018 s3f doc compcAtitit 操作日
    Atitit s2018.2 s2 doc list on home ntpc.docx  Atiitt uke制度体系 法律 法规 规章 条例 国王诏书.docx Atiitt 手写文字识别 讯飞科大 语音云.docx Atitit 代码托管与虚拟主机.docx Atitit 企业文化 每日心灵 鸡汤 值班 发布.docx Atitit 几大研发体系对比 Stage-Gat
    Atitit 文员招募规范 attilax总结
    Atitit r2017 r6 doc list on home ntpc.docx
    atitit r9 doc on home ntpc .docx
    Atitit.如何文章写好 论文 文章 如何写好论文 技术博客 v4
  • 原文地址:https://www.cnblogs.com/zlslch/p/7237929.html
Copyright © 2011-2022 走看看