zoukankan      html  css  js  c++  java
  • 【原创】大数据基础之Hive(3)最简绿色部署

    hadoop部署参考:https://www.cnblogs.com/barneywill/p/10428098.html

    1 拷贝到所有服务器上并解压

    # ansible all-servers -m copy -a 'src=/src/path/to/apache-hive-2.3.4-bin.tar.gz dest=/dest/path/to/'
    # ansible all-servers -m shell -a 'tar xvf /dest/path/to/apache-hive-2.3.4-bin.tar.gz -C /app/path'

    2 拷贝mysql-connector-java.jar

    # ansible all-servers -m shell -a 'cp /path/to/mysql-connector-java.jar /app/path/apache-hive-2.3.4-bin/lib/'

    3 准备配置文件

    hive-site.xml

    <configuration>

        <property>

            <name>javax.jdo.option.ConnectionUserName</name>

            <value>hive</value>

        </property>

        <property>

            <name>javax.jdo.option.ConnectionPassword</name>

            <value>hive</value>

        </property>

       <property>

            <name>javax.jdo.option.ConnectionURL</name>

            <value>jdbc:mysql://node0:3306/hive</value>

        </property>

        <property>

            <name>javax.jdo.option.ConnectionDriverName</name>

            <value>com.mysql.jdbc.Driver</value>

        </property>

        <property>

            <name>hive.metastore.uris</name>

            <value>thrift://node0:9083</value>

        </property>

    </configuration>

    4 将配置同步到所有服务器上

    # ansible all-servers -m copy -a 'src=/path/to/config/ dest=/app/path/apache-hive-2.3.4-bin/conf/'

    5 初始化数据库

    # echo "create database hive;grant all privileges on hive.* to hive@'%' identified by 'hive';"|mysql -uroot -proot

    # su - hadoop
    $ /app/path/apache-hive-2.3.4-bin/bin/schematool -dbType mysql -initSchema

    6 启动metastore

    # su - hadoop
    $ /app/path/apache-hive-2.3.4-bin/bin/hive --service metastore

    7 启动hive thrift server

    # su - hadoop
    $ /app/path/apache-hive-2.3.4-bin/bin/hive --service hiveserver2

    如果运行sql时报Error: Java heap space,即mapper或reducer内存溢出,可以临时调整参数

    set mapreduce.map.memory.mb=3072;
    set mapreduce.map.java.opts=-Xmx2048m;
    set mapreduce.reduce.memory.mb=3072;
    set mapreduce.reduce.java.opts=-Xmx2048m;

    以上配置可以在mapred-site.xml中永久修改

  • 相关阅读:
    随笔2
    随笔
    关于updateElement接口
    随笔1
    本地访问正常,服务器访问乱码 记录
    Redis (error) NOAUTH Authentication required.解决方法
    tomcat启动很慢 停留在 At least one JAR was scanned for TLDs yet contained no TLDs.
    微信公众号消息回复
    微信公众号 报token验证失败
    idea中web.xml报错 Servlet should have a mapping
  • 原文地址:https://www.cnblogs.com/barneywill/p/10429929.html
Copyright © 2011-2022 走看看