zoukankan      html  css  js  c++  java
  • EMR的hive包升级和gateway启动hs2

    升级:


    mkdir /opt/apps/hive
    cd /opt/apps/hive
    wget http://emr.oss-cn-hangzhou.aliyuncs.com/ecm-repo/hive/opay/apache-hive-2.3.5-bin.tar.gz
    md5sum apache-hive-2.3.5-bin.tar.gz
    229e32d99d8d288bdece8c897d6cc625 apache-hive-2.3.5-bin.tar.gz
    tar xvf apache-hive-2.3.5-bin.tar.gz

    1. Gateway升级Hive包
    a.登录Gateway机器,将新的hive包解压到一个目录,比如/opt/apps/hive
    b. cp /usr/lib/hive-current/lib/emr-hive-hook*jar /opt/apps/hive/apache-hive-2.3.5-bin/lib
    c. unlink /usr/lib/hive-current
    d. ln -s /opt/apps/hive/apache-hive-2.3.5-bin /usr/lib/hive-current
    e. 建立其它软连接

    ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-hive-plugin-impl /usr/lib/hive-current/lib/ranger-hive-plugin-impl
    ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-hive-plugin-shim-1.2.0.jar /usr/lib/hive-current/lib/ranger-hive-plugin-shim-1.2.0.jar
    ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-plugin-classloader-1.2.0.jar /usr/lib/hive-current/lib/ranger-plugin-classloader-1.2.0.jar

    ln -s /usr/lib/spark-current/jars/scala-library-2.11.12.jar /usr/lib/hive-current/lib/scala-library-2.11.12.jar
    ln -s /usr/lib/spark-current/jars/spark-core_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-core_2.11-2.4.3.jar
    ln -s /usr/lib/spark-current/jars/spark-network-common_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-network-common_2.11-2.4.3.jar
    ln -s /usr/lib/spark-current/jars/spark-unsafe_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-unsafe_2.11-2.4.3.jar

    f. 测试一下是否正常

    2. HiveServer升级Hive包
    a. 登录emr-header-1, 将新的hive包解压到一个目录,比如/opt/apps/hive
    b. cp /usr/lib/hive-current/lib/emr-hive-hook*jar /opt/apps/hive/apache-hive-2.3.5-bin/lib
    c. unlink /usr/lib/hive-current
    d. ln -s /opt/apps/hive/apache-hive-2.3.5-bin /usr/lib/hive-current
    e. 建立其它软连接

    ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-hive-plugin-impl /usr/lib/hive-current/lib/ranger-hive-plugin-impl
    ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-hive-plugin-shim-1.2.0.jar /usr/lib/hive-current/lib/ranger-hive-plugin-shim-1.2.0.jar
    ln -s /opt/apps/ecm/service/ranger/1.2.0-1.1.0/package/ranger-1.2.0-1.1.0/ranger-1.2.0-1.1.0-hive-plugin/lib/ranger-plugin-classloader-1.2.0.jar /usr/lib/hive-current/lib/ranger-plugin-classloader-1.2.0.jar

    ln -s /usr/lib/spark-current/jars/scala-library-2.11.12.jar /usr/lib/hive-current/lib/scala-library-2.11.12.jar
    ln -s /usr/lib/spark-current/jars/spark-core_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-core_2.11-2.4.3.jar
    ln -s /usr/lib/spark-current/jars/spark-network-common_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-network-common_2.11-2.4.3.jar
    ln -s /usr/lib/spark-current/jars/spark-unsafe_2.11-2.4.3.jar /usr/lib/hive-current/lib/spark-unsafe_2.11-2.4.3.jar

    f. 登录emr-header-2,重复a~d的操作
    g. 页面控制台重启HiveServer
    h. 使用beeline/hue测试是否正常


    gateway上启动hs2: (可选)
    拷贝Ranger相关配置文件
    以root用户执⾏以下命令
    scp root@emr-header-1:/etc/ecm/hive-conf/ranger-hive-audit.xml /etc/ecm/hive-conf
    scp root@emr-header-1:/etc/ecm/hive-conf/ranger-hive-security.xml /etc/ecm/hive-conf
    scp root@emr-header-1:/etc/ecm/hive-conf/ranger-policymgr-ssl.xml /etc/ecm/hive-conf
    scp root@emr-header-1:/etc/ecm/hive-conf/ranger-security.xml /etc/ecm/hive-conf
    chown hadoop:hadoop /etc/ecm/hive-conf/ranger-*
    启动HiveServer2
    以hadoop用户执行以下命令

    HADOOP_OPTS=HIVE_SERVER2_OPTS HADOOP_HEAPSIZE=HIVE_SERVER2_HEAPSIZE hive --service
    hiveserver2 >/var/log/hive/hiveserver2.out 2>/var/log/hive/hiveserver2.err &

  • 相关阅读:
    Java帮助文档的生成
    Java内部类
    Java中利用标签跳出外层循环break
    【转】你真的了解word-wrap和word-break的区别吗?
    Office/Access 2013 扩展支持xbase/DBF 文件
    调用cmd.exe执行pdf的合并(pdftk.exe)
    input 数字输入控制(含小数)
    iis7.5 发布mvc出错的解决办法
    table中超过长度的列,显示省略号
    本地图片的预览和上传
  • 原文地址:https://www.cnblogs.com/hongfeng2019/p/12120596.html
Copyright © 2011-2022 走看看