zoukankan      html  css  js  c++  java
  • azkaban的安装部署

    一、所需环境

    1,JDK

    2,HADOOP

    4,mysql

    3,HIVE

    二、安装azkaban

    1,安装git命令: yum install git

    2,下载azkaban源码:git clone https://github.com/azkaban/azkaban.git

    3,安装到 azkaban-tar目录下

    4,更改配置文件:

    vi 安装目录/azkaban-tar/azkaban/git clone https://github.com/azkaban/azkaban.git

    将final boolean isExecuteAsUser = this.sysProps.getBoolean(EXECUTE_AS_USER, true);
    true修改false 

    5,在azkaban目录下执行:./gradlew distTar  

    此步骤会生成5个压缩包,分别放在下图目录中:

    azkaban-db-3.37.0-10-gdd67b02.tar.gz
    azkaban-exec-server-3.37.0-10-gdd67b02.tar.gz
    azkaban-hadoop-security-plugin-3.37.0-10-gdd67b02.tar.gz
    azkaban-solo-server-3.37.0-10-gdd67b02.tar.gz
    azkaban-web-server-3.37.0-10-gdd67b02.tar.gz 

    6,在当前azkaban文件夹下,将该五个压缩包copy出来,放到 安装目录/azkaban-tar文件夹中。

    cp  azkaban-*/build/distributions/*.tar.gz  安装目录/azkaban-tar

     

    7,将这5个压缩包解压,然后重命令,将这5个压缩包放在另外的 安装目录/azkaban下

    8,登录mysql mysql –u root –p

    9,数据库中执行:create database azkaban;

    10,数据库中执行:use azkaban;

    11, 将azkaban sql表结构导入mysql,数据库中执行:source 安装目录/azkaban/azkaban-db-3.46.0-2-gbb24a72/create-all-sql-3.46.0-2-gbb24a72.sql  

    如果执行失败,则将sql语句复制出来执行

     

    12,部署azkaban-web-server

    将 安装目录/azkaban/azkaban-solo-server/ 下的conf目录拷贝到  安装目录/azkaban/ azkaban-web-server/conf

    命令:cp -r  /安装目录/azkaban/azkaban-solo-server/conf   /安装目录/azkaban/azkaban-web-server/conf 

    配置:azkaban.properties

     

    如下(标红的需要更改,其他的可以对比一下):

     

    # Azkaban Personalization Settings

    azkaban.name=Test

    azkaban.label=My Local Azkaban

    azkaban.color=#FF3601

    azkaban.default.servlet.path=/index

    web.resource.dir=/usr/local/azkaban/azkaban-web-server/web

    default.timezone.id=Asia/Shanghai

    # Azkaban UserManager class

    user.manager.class=azkaban.user.XmlUserManager

    user.manager.xml.file=/usr/local/azkaban/azkaban-web-server/conf/azkaban-users.xml

    # Loader for projects

    executor.global.properties=/usr/local/azkaban/azkaban-web-server/conf/global.properties

    azkaban.project.dir=projects

    database.type=mysql

    mysql.port=3306

    mysql.host=localhost

    mysql.database=azkaban

    mysql.user=root

    mysql.password=root

    mysql.numconnections=100

     

    # Velocity dev mode

    velocity.dev.mode=false

    # Azkaban Jetty server properties.

    jetty.use.ssl=false

    jetty.maxThreads=25

    jetty.port=8081

    # Azkaban Executor settings

    executor.maxThreads=50

    executor.port=12321

    executor.flow.threads=30

    # mail settings

    mail.sender=

    mail.host=

    # User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.

    # enduser -> myazkabanhost:443 -> proxy -> localhost:8081

    # when this parameters set then these parameters are used to generate email links.

    # if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.

    # azkaban.webserver.external_hostname=myazkabanhost.com

    # azkaban.webserver.external_ssl_port=443

    # azkaban.webserver.external_port=8081

    job.failure.email=

    job.success.email=

    lockdown.create.projects=false

    cache.directory=cache

    # JMX stats

    jetty.connector.stats=true

    executor.connector.stats=true

     

    azkaban.native.lib=false

    # Azkaban plugin settings

    azkaban.jobtype.plugin.dir=plugins/jobtypes

     

    13,在conf目录下新增配置文件log4j.properties,内容如下

     

    log4j.rootLogger=INFO,C

    log4j.appender.C=org.apache.log4j.ConsoleAppender

    log4j.appender.C.Target=System.err

    log4j.appender.C.layout=org.apache.log4j.PatternLayout

    log4j.appender.C.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

     

    14,部署 azkaban-exec-server

    将 /安装目录/azkaban/ azkaban-solo-server下的conf文件拷贝到 /安装文件/azkaban/ azkaban-exec-server

    命令:cp –r /安装目录/azkaban/azkaban-solo-server/conf  /安装目录/azkaban/azkaban-exec-server/conf

     

    配置azkaban.properties:(标红的需要更改,其他的可以对比一下)

     

    # Azkaban Personalization Settings

    azkaban.name=Test

    azkaban.label=My Local Azkaban

    azkaban.color=#FF3601

    azkaban.default.servlet.path=/index

    web.resource.dir=web/

    default.timezone.id=Asia/Shanghai

    # Azkaban UserManager class

    user.manager.class=azkaban.user.XmlUserManager

    user.manager.xml.file=conf/azkaban-users.xml

    # Loader for projects

    executor.global.properties=/usr/local/azkaban/azkaban-exec-server/conf/global.properties

    azkaban.project.dir=projects

    database.type=mysql

    mysql.port=3306

    mysql.host=localhost

    mysql.database=azkaban

    mysql.user=root

    mysql.password=root

    mysql.numconnections=100

    # Velocity dev mode

    velocity.dev.mode=false

    # Azkaban Jetty server properties.

    jetty.use.ssl=false

    jetty.maxThreads=25

    jetty.port=8081

    # Azkaban Executor settings

    executor.maxThreads=50

    executor.port=12321

    executor.flow.threads=30

    # mail settings

    mail.sender=

    mail.host=

    # User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.

    # enduser -> myazkabanhost:443 -> proxy -> localhost:8081

    # when this parameters set then these parameters are used to generate email links.

    # if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.

    # azkaban.webserver.external_hostname=myazkabanhost.com

    # azkaban.webserver.external_ssl_port=443

    # azkaban.webserver.external_port=8081

    job.failure.email=

    job.success.email=

    lockdown.create.projects=false

    cache.directory=cache

    # JMX stats

    jetty.connector.stats=true

    executor.connector.stats=true

    # Azkaban plugin settings

    azkaban.jobtype.plugin.dir=plugins/jobtypes

     

    azkaban.native.lib=false

     

    在conf目录下,新建配置log4j.properties文件,内容如下:

     

    log4j.rootLogger=INFO,C

    log4j.appender.C=org.apache.log4j.ConsoleAppender

    log4j.appender.C.Target=System.err

    log4j.appender.C.layout=org.apache.log4j.PatternLayout

    log4j.appender.C.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

     

    15,在hadoop中的/share/Hadoop/common 路径下找到hadoop-common-2.6.4.jar

       在jdk中的/jdk1.8.0_91/db/lib/路径下找到derby.jar

    将这俩个jar包拷贝到azkaban-exec-server和azkaban-web-server
    安装目录下的extlib目录,没有则创建一个,找不到就下载

    16,启动

    进入azkaban-exec-server目录,执行start-exec.sh

    进入azkaban-web-server目录,执行start-web.sh

     

    查看进程:jps

     

    成功执行

    通过http://localhost:8081可访问到web-ui

    16,此时的web界面:

     

    需要更改配置文件解决:

     

    将上图中的web目录的路径加到azkaban.properties配置文件中,如下图

     

    再进入web目录可以看到界面变得美观

     

    17, 用户名密码在 azkaban-web-server/conf/azkaban-users.xml 中配置

  • 相关阅读:
    uva 10369 Arctic Network
    uvalive 5834 Genghis Khan The Conqueror
    uvalive 4848 Tour Belt
    uvalive 4960 Sensor Network
    codeforces 798c Mike And Gcd Problem
    codeforces 796c Bank Hacking
    codeforces 768c Jon Snow And His Favourite Number
    hdu 1114 Piggy-Bank
    poj 1276 Cash Machine
    bzoj 2423 最长公共子序列
  • 原文地址:https://www.cnblogs.com/huxinga/p/8706155.html
Copyright © 2011-2022 走看看