zoukankan      html  css  js  c++  java
  • airflow worker 出现 (2006, 'MySQL server has gone away')以及raise AirflowException('Celery command failed')airflow 设置分布式部署主从worker不能协调运行

    OperationalError: (_mysql_exceptions.OperationalError) (2006, 'MySQL server has gone away')
    [SQL: SELECT celery_taskmeta.id AS celery_taskmeta_id, celery_taskmeta.task_id AS celery_taskmeta_task_id, celery_taskmeta.status

    重启能解决上述问题

    [2020-04-20 17:25:40,165: ERROR/ForkPoolWorker-1] None
    [2020-04-20 17:25:40,172: ERROR/ForkPoolWorker-1] Task airflow.executors.celery_executor.execute_command[5253e28b-4219-42bc-81ca-2b8951a7123e] raised unexpected: AirflowException('Celery command failed',)
    Traceback (most recent call last):
    File "/home/ubuntu/.local/lib/python2.7/site-packages/celery/app/trace.py", line 385, in trace_task
    R = retval = fun(*args, **kwargs)
    File "/home/ubuntu/.local/lib/python2.7/site-packages/celery/app/trace.py", line 650, in __protected_call__
    return self.run(*args, **kwargs)
    File "/usr/local/lib/python2.7/dist-packages/airflow/executors/celery_executor.py", line 72, in execute_command
    raise AirflowException('Celery command failed')
    AirflowException: Celery command failed

    上述问题,未找到对应的执行程序,认真检查执行程序是否在所在目录;另外确保slave worker能执行master程序请设置如下操作:

    sudo airflow worker -cn=celery@hosttname -p -D

    master schduler -p 

  • 相关阅读:
    HDU_1006_Tick and Tick
    HDU_1011_Starship Troopers_树型dp
    HDU_1520_Anniversary party_树型dp
    HDU_1176_免费馅饼_16.4.23再做
    HDU_1203_01背包
    HDU_1421_搬寝室_dp
    HDU_1505_矩阵中的最大矩形_dp
    STL_map的使用
    Semantic-UI-React (称 stardust) 对比 Antd
    meteor 为基础,联合 Apollo + React + React-Router
  • 原文地址:https://www.cnblogs.com/zhearts/p/12741744.html
Copyright © 2011-2022 走看看