zoukankan      html  css  js  c++  java
  • 【原】centos6.5下cdh4.6 Oozie安装

    0、oozie只需安装在一台服务器上,这里选择在namenode上来安装;安装用户为cloud-user

    1、安装Oozie包:

       sudo yum install -y oozie oozie-client
    2、对于yarn来说,需要修改/etc/oozie/conf/oozie-env.sh中:
       CATALINA_BASE=/usr/lib/oozie/oozie-server
    3、配置mysql:
       yum install -y mysql-server
       创建数据库:
       mysql -u root -p
       mysql> create database oozie;
       mysql> grant all privileges on oozie.* to 'oozie'@'localhost' identified by 'oozie';
       mysql> grant all privileges on oozie.* to 'oozie'@'%' identified by 'oozie';
       mysql> exit
    4、配置oozie-site.xml:
     1 <property>
     2      <name>oozie.service.JPAService.jdbc.driver</name>
     3      <value>com.mysql.jdbc.Driver</value>
     4 </property>
     5 <property>
     6      <name>oozie.service.JPAService.jdbc.url</name>
     7      <value>jdbc:mysql://10.0.0.2:3306/oozie</value>
     8 </property>
     9 <property>
    10      <name>oozie.service.JPAService.jdbc.username</name>
    11      <value>oozie</value>
    12 </property>
    13 <property>
    14      <name>oozie.service.JPAService.jdbc.password</name>
    15      <value>oozie</value>
    16 </property>
    17 <property>
    18      <name>oozie.action.mapreduce.uber.jar.enable</name>
    19      <value>true</value>
    20 </property>
    21 <property>
    22      <name>oozie.service.HadoopAccessorService.supported.filesystems</name>
    23      <value>hdfs,viewfs</value>
    24 </property>
    25   <property>
    26      <name>oozie.service.ProxyUserService.proxyuser.cloud-user.hosts</name>
    27      <value>*</value>
    28 </property>
    29  <property>
    30      <name>oozie.service.ProxyUserService.proxyuser.cloud-user.groups</name>
    31      <value>*</value>
    32 </property>
    5、拷贝JDBC Driver:
        sudo cp /usr/share/java/mysql-connector-java.jar /var/lib/oozie
        sudo cp /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar /var/lib/oozie
        sudo cp /usr/lib/hadoop/lib/hadoop-lzo.jar /var/lib/oozie
    6、创建oozie 数据库scheme,这一步骤需要root来执行:
        su -
      /usr/lib/oozie/bin/ooziedb.sh create -sqlfile oozie-create.sql
        /usr/lib/oozie/bin/ooziedb.sh create -run  
    7、使能web console:
      sudo mv ext-2.2.zip /var/lib/oozie/
        cd /var/lib/oozie/ && sudo unzip ext-2.2.zip
      sudo chown -R oozie:oozie /var/lib/oozie/
    8、安装Oozie ShareLib:
        sudo -u hdfs hadoop fs -mkdir /user/oozie
        sudo -u hdfs hadoop fs -chown oozie:oozie /user/oozie
        mkdir /tmp/ooziesharelib
        cd /tmp/ooziesharelib
        tar xzf /usr/lib/oozie/oozie-sharelib-yarn.tar.gz
        sudo -u oozie hadoop fs -put share /user/oozie/share
    9、启动服务:
        sudo service oozie start
    10、测试map-reduce:
      添加/etc/hadoop/conf/core-site.xml
    1 <property> 
    2    <name>hadoop.proxyuser.oozie.hosts</name> 
    3    <value>10.0.0.2</value> 
    4 </property> 
    5 <property> 
    6    <name>hadoop.proxyuser.oozie.groups</name> 
    7    <value>*</value> 
    8 </property> 
      cp /usr/share/doc/oozie-3.3.2+100/oozie-examples.tar.gz .  
      tar -zxvf oozie-examples.tar.gz
      修改example/apps/map-reduce/job.properties 

      nameNode=hdfs://10.0.0.2:8020
      jobTracker=10.0.0.2:8032 端口改为/etc/hadoop/conf/yarn-site.xml中 'yarn.resourcemanager.address ' 项的值

        hadoop fs -put examples
      oozie job -oozie http://10.0.0.3:11000/oozie -config  ./examples/apps/no-op/job.properties -run
      Job ID : 0000000-140508214215146-oozie-oozi-W
      通过oozie job -oozie http://10.0.0.3:11000/oozie -log 0000000-140508214215146-oozie-oozi-W查看log
      通过oozie job -oozie http://10.0.0.3:11000/oozie -info 0000000-140508214215146-oozie-oozi-W查看信息
      通过http://namenode:11000/oozie查看web console
  • 相关阅读:
    输入流输出流
    WIN32_FIND_DATA
    typeid
    为什么要内存对齐 Data alignment: Straighten up and fly right
    fatal error C1010: 在查找预编译头时遇到意外的文件结尾。是否忘记了向源中添加“#include "stdafx.h
    memcmp
    DPI
    英寸、 Picas、 点、 跨度和 Twips 之间的关系
    如何解决 “fatal error C1083: ”无法打开包括文件
    hdoj 1269迷宫城堡解题报告
  • 原文地址:https://www.cnblogs.com/yuandianliws/p/3718215.html
Copyright © 2011-2022 走看看