zoukankan      html  css  js  c++  java
  • Hive环境搭建

    hive 环境搭建需要hadoop的环境。hadoop环境的搭建不在这里赘述。参考:http://www.cnblogs.com/parkin/p/6952370.html

    1.准备阶段

    hive 官网下载 : http://mirror.bit.edu.cn/apache/hive/hive-2.1.1/

    2.解压缩

    tar -zxvf apache-hive-2.1.1-bin.tar.gz -C /home/q/

    3.hive 配置

      hive-env.sh 配置

    cp hive-env.sh.template hive-env.sh
    vi hive-env.sh 
    #添加以下代码 
    export HADOOP_HOME=/home/q/hadoop-2.6.5 
    export HIVE_HOME=/home/q/apache-hive-2.1.1-bin
    

      hive-site.xml 配置

    cp hive-site.xml.template hive-site.xml 
    vi hive-site.xml #添加以下代码 <!-- metastore 数据库位置,这里采用remote形式,数据库名为hive,用户root, 密码 b6f3g2 ,需先创建好。--> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://192.168.251.114:3306/hive?createDatabaseIfNotExist=true&characterEncoding=UTF-8</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>javax.jdo.option.DetachAllOnCommit</name> <value>true</value> <description>detaches all objects from session so that they can be used after transaction is committed</description> </property> <property> <name>javax.jdo.option.NonTransactionalRead</name> <value>true</value> <description>reads outside of transactions</description> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>b6f3g2</value> </property> <property> <name>javax.jdo.option.Multithreaded</name> <value>true</value> <description>Set this to true if multiple threads access metastore through JDO concurrently.</description> </property> <property> <name>hive.metastore.warehouse.dir</name> <value>/user/hive/warehouse</value> <description>location of default database for the warehouse</description> </property> <property> <name>hive.exec.scratchdir</name> <value>/user/hive/scratch</value> </property> <property> <name>hive.aux.jars.path</name> <value>file:///home/q/apache-hive-2.1.1-bin/auxlib/json-serde-1.3.1-SNAPSHOT-jar-with-dependencies.jar,file:///home/q/apache-hive-2.1.1-bin/auxlib /qunar-udf.jar,file:///home/q/apache-hive-2.1.1-bin/auxlib/brickhouse-0.7.1-SNAPSHOT.jar,file:///home/q/apache-hive-2.1.1-bin/auxlib/joda-time-2.1.jar </value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>localhost</value> </property> <property> <name>hive.exec.dynamic.partition</name> <value>true</value> </property> <property> <name>hive.exec.dynamic.partition.mode</name> <value>nostrict</value> </property> <property> <name>hive.server2.thrift.port</name> <value>12121</value> </property> <property> <name>mapred.max.split.size</name> <value>48000000</value> </property> <property> <name>mapred.min.split.size.per.node</name> <value>24000000</value> </property> <property> <name>mapred.min.split.size.per.rack</name> <value>24000000</value> </property> <property> <name>hive.hadoop.supports.splittable.combineinputformat</name> <value>true</value> </property> <property> <name>hive.exec.max.dynamic.partitions.pernode</name> <value>1000</value> </property> <property> <name>hive.merge.mapredfiles</name> <value>true</value> </property> <property> <name>hive.merge.smallfiles.avgsize</name> <value>32000000</value> </property> <property> <name>hive.hadoop.supports.splittable.combineinputformat</name> <value>true</value> </property> <property> <name>hive.metastore.schema.verification</name> <value>false</value> <description> Enforce metastore schema version consistency. True: Verify that version information stored in metastore matches with one from Hive jars. Also disable automatic schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures proper metastore schema migration. (Default) False: Warn if the version information stored in metastore doesn't match with one from in Hive jars. </description> </property> <property> <name>hive.server2.long.polling.timeout</name> <value>5000</value> <description>Time in milliseconds that HiveServer2 will wait, before responding to asynchronous calls that use long polling</description> </property> </configuration>

     

    4.metastore 数据库初始化 

    cd /home/q/apache-hive-2.1.1-bin/bin 
    schematool -dbType mysql -initSchema

    5. hive 启动

    # CLI 交互式查询 hive 仓库 
    cd /home/q/apache-hive-2.1.1-bin/bin 
    ./hive

    6.hive metastore 服务启动 (主要用于hive client客户端的访问)

    cd /home/q/apache-hive-2.1.1-bin/bin 
    hive --service metastore # -p <port_num> 不指定该参数,则默认端口为9083#

    7.hiveserver2 服务启动(主要用于rpc方式访问hive数据仓库的表)

    cd /home/q/apache-hive-2.1.1-bin/bin
    ./hiveserver2
  • 相关阅读:
    二进制插入 牛客网 程序员面试金典 C++ Python java
    二进制小数 牛客网 程序员面试金典 C++ Python
    二叉树中和为某一值的路径 牛客网 程序员面试金典 C++ Python
    Python matplotlib pylab 画张图
    Python matplotlib pylot和pylab的区别
    Ubuntu 16.04 curl 安装 使用
    Ubuntu 16.04 菜单栏 换位置 挪到左边 挪到下边
    Python 模块feedparser安装使用
    Ubuntu 16.04 下 旋转显示器屏幕 竖屏显示
    从 callback 到 promise
  • 原文地址:https://www.cnblogs.com/parkin/p/7151591.html
Copyright © 2011-2022 走看看