zoukankan      html  css  js  c++  java
  • [会装]Hive安装(基于mysql数据库)

    环境信息:Mac

    安装步骤:

    1. 下载hive组件(我选择的是社区的2.0.1版本)

      http://apache.mirror.globo.tech/hive/hive-2.0.1/

    2. 下载mysql驱动

      https://dev.mysql.com/downloads/connector/j/

    3. 设定环境变量(至于具体放到什么文件夹可以自由指定这里不做赘述)

    export CATALINA_HOME=/Users/apple/Desktop/software/apache-tomcat-8.5.11
    export KYLIN_HOME=/Users/apple/Desktop/hadoop2/apachesrc/kylin/bin/apache-kylin-1.6.0-cdh5.7-bin
    #hadoop config
    #export DERBY_HOME=/Users/apple/Desktop/hadoop2/tools/db-derby-10.4.2.0-bin
    export HIVE_HOME=/Users/apple/Desktop/hadoop2/apachesrc/hive/2.0.1/bin/apache-hive-2.0.1-bin
    export HIVE_CONF_DIR=/Users/apple/Desktop/hadoop2/apachesrc/hive/2.0.1/bin/config
    export HCAT_HOME=$HIVE_HOME/hcatalog
    export HIVE_CONF=/Users/apple/Desktop/hadoop2/apachesrc/hive/2.0.1/bin/config
    export HADOOP_HOME=/Users/apple/Desktop/hadoop2/cdh/hadoop/bin/hadoop-2.6.0-cdh5.4.2
    export HADOOP_CONF_DIR=/Users/apple/Desktop/hadoop2/cdh/hadoop/bin/config
    export HBASE_HOME=/Users/apple/Desktop/hadoop2/cdh/hbase/bin/hbase-1.0.0-cdh5.4.2
    export HBASE_CONF_DIR=/Users/apple/Desktop/hadoop2/cdh/hbase/bin/config
    export PATH=$HIVE_HOME/bin:$HBASE_HOME/bin:$KYLIN_HOME/bin:$HCAT_HOME/bin:$CATALINA_HOME/bin:$SBT_HOME/bin:$MAVEN_HOME/bin:$PROTOBUF_HOME/bin:$JAVA_HOME/bin:$ANT_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

    4.修改(增加)hive配置文件

      修改$HIVE_CONF/hive_site.xml,加入:

      <property>
         <name>javax.jdo.option.ConnectionURL</name>
         <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
         <description>JDBC connect string for a JDBC metastore </description>
      </property>
      <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
        <description>Driver class name for a JDBC metastore</description>
      </property>
      <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>mysql</value>
        <description>Username to use against metastore database</description>
      </property>
      <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>mysql</value>
        <description>password to use against metastore database</description>
      </property>

      <property>

        <name>hive.metastore.warehouse.dir</name>

        <value>/hive/warehouse</value>

        <description>location of default database for the warehouse</description>

      </property>

      在$HIVE_CONF文件夹下创建jpox.properties文件输入:

    javax.jdo.PersistenceManagerFactoryClass =
    
    org.jpox.PersistenceManagerFactoryImpl
    org.jpox.autoCreateSchema = false
    org.jpox.validateTables = false
    org.jpox.validateColumns = false
    org.jpox.validateConstraints = false
    org.jpox.storeManagerType = rdbms
    org.jpox.autoCreateSchema = true
    org.jpox.autoStartMechanismMode = checked
    org.jpox.transactionIsolation = read_committed
    javax.jdo.option.DetachAllOnCommit = true
    javax.jdo.option.NontransactionalRead = true
    javax.jdo.option.ConnectionDriverName = com.mysql.jdbc.Driver
    javax.jdo.option.ConnectionURL = jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true
    javax.jdo.option.ConnectionUserName = mysql
    javax.jdo.option.ConnectionPassword = mysql
    ~                                             

    5. 将mysql驱动拷贝到hive的lib目录中

      mysql-connector-java-5.1.41-bin.jar

     

    5. 初始化memstore数据库:

    schematool -initSchema -dbType mysql

    6. 创建hive工作目录:  

    $ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp 
    $ $HADOOP_HOME/bin/hadoop fs -mkdir /hive/warehouse
    $ $HADOOP_HOME/bin/hadoop fs -chmod g+w /tmp 
    $ $HADOOP_HOME/bin/hadoop fs -chmod g+w /hive/warehouse

      

    7.输入hive:

    Logging initialized using configuration in jar:file:/Users/apple/Desktop/hadoop2/apachesrc/hive/2.0.1/bin/apache-hive-2.0.1-bin/lib/hive-common-2.0.1.jar!/hive-log4j2.properties
    Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. tez, spark) or using Hive 1.X releases.
    hive> show tables;
    OK
    Time taken: 1.201 seconds
    hive> 

    完毕

  • 相关阅读:
    Unity Shader (三)Surface Shader机制
    Unity Shader (二)Cg语言
    Unity Shader (一)ShaderLab 语法
    Pytorch笔记 (3) 科学计算2
    两种方法实现斐波那契数列
    根据输入的整数n使得输出精确到小数n位
    算法竞赛入门 —— 循环结构 习题
    Pytorch笔记 (3) 科学计算1
    算法竞赛入门 (一)语言篇 循环结构
    C++编译错误提示 [Error] name lookup of 'i' changed for ISO '
  • 原文地址:https://www.cnblogs.com/ios123/p/6548288.html
Copyright © 2011-2022 走看看