zoukankan      html  css  js  c++  java
  • hive 安装配置

    hive 安装配置

    检查 mysqld 进程是否已经开启:

    $pgrep mysqld

    如果进程开启,这个命令将会返回该进程的 id

    进入MySQL

    $mysql -u root -p

    安装hive

    解压

    tar–zxvf apache-hive-1.1.0-bin.tar.gz

    profile配置环境变量

    执行命令

    $sudo gedit /etc/profile

    注意CLASSPATH,PATH在原有上增加hive包路径

    #set hive environment

    HIVE_HOME=/home/hadoop/hive-1.1.0

    PATH=$HIVE_HOME/bin:$PATH

    CLASSPATH=$CLASSPATH:$HIVE_HOME/lib

    export HIVE_HOME

    export PATH

    export CLASSPATH

     

    使配置生效:

    $source /etc/profile

    配置hive-env.sh

    gedit /home/yuqiao/hive-0.9.0/conf hive-env.sh

     

    # Set HADOOP_HOME to point to a specific hadoop install directory

    HADOOP_HOME=/home/yuqiao/hadoop-2.7.1

    # Hive Configuration Directory can be controlled by:

    export HIVE_CONF_DIR=/home/yuqiao/hive-0.9.0/conf

    配置hive-site.xml

    创建配置文件

    $cp hive-default.xml.template  hive-site.xml

    hive.metastore.warehouse.dir,hive.exec.scratchdir【 hdfs目录】会自动创建

    在同上的目录下

    <property>

      <name>hive.metastore.warehouse.dir</name>

      <value>/home/yuqiao/hive/warehouse</value>

      <description>location of default database for the warehouse</description>

    </property>

    <property>

      <name>hive.exec.scratchdir</name>

      <value>/home/yuqiao/hive/scratchdir</value>

      <description>Scratch space for Hive jobs</description>

    </property>

    <property>

      <name>hive.querylog.location</name>

      <value>/home/yuqiao/hive-0.9.0/logs</value>

      <description>

        Location of Hive run time structured log file

      </description>

    </property>

    <property>

      <name>javax.jdo.option.ConnectionURL</name>

      <value>jdbc: mysql://master:3306/hive_metadata?createDatabaseIfNotExist=true</value>

      <description>JDBC connect string for a JDBC metastore</description>

    </property>

    <property>

      <name>javax.jdo.option.ConnectionDriverName</name>

      <value>com.mysql.jdbc.Driver</value>

      <description>Driver class name for a JDBC metastore</description>

    </property>

    <property>

      <name>javax.jdo.option.ConnectionUserName</name>

      <value>root</value>

      <description>username to use against metastore database</description>

    </property>

    <property>

      <name>javax.jdo.option.ConnectionPassword</name>

      <value>1234</value>

      <description>password to use against metastore database</description>

    </property>

    <property>

       <name>hive.exec.local.scratchdir</name> 

      <!--拼凑目录-->

       <value>/home/yuqiao/hive-0.9.0/local/${system:user.name}</value>

       <description>Local scratch space for Hive jobs</description>

      </property>

    <property>

       <name>hive.downloaded.resources.dir</name>

       <value>/home/yuqiao/hive-0.9.0/local/${hive.session.id}_resources</value>

       <description>Temporary local directory for added resources in theremote file system.</description>

      </property>

    <property>

       <name>hive.server2.logging.operation.log.location</name>

       <value>/home/yuqiao/hive-0.9.0/logs/operation_logs</value>

       <description>Top leveldirectory where operation logs are stored if logging functionality isenabled</description>

      </property>

     

    在hive下创建local文件夹

    配置log4j

     

    在同上目录下

    创建配置文件:

    cp hive-exec-log4j.properties.template  hive-exec-log4j.properties

    cp hive-log4j.properties.template  hive-log4j.properties

    修改上面两个文件中的配置

    hive.log.dir=/home/hadoop/ hive-1.1.0/logs

    log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter

    注意如果没有logs目录就建立一个 执行命令

    $mkdir /home/hadoop/hive-1.1.0/logs

    添加Mysql驱动包

       把驱动包mysql-connector-java-5.1.24-bin.jar

    放到 $HIVE_HOME/lib 目录下

     

     

    修改profile

     

    启动hive

     

    执行命令

    $start-dfs.sh

    $hive

    有些如insert into需要执行mr,需要打开start-yarn.sh

  • 相关阅读:
    LeetCode——Generate Parentheses
    LeetCode——Best Time to Buy and Sell Stock IV
    LeetCode——Best Time to Buy and Sell Stock III
    LeetCode——Best Time to Buy and Sell Stock
    LeetCode——Find Minimum in Rotated Sorted Array
    Mahout实现基于用户的协同过滤算法
    使用Java对文件进行解压缩
    LeetCode——Convert Sorted Array to Binary Search Tree
    LeetCode——Missing Number
    LeetCode——Integer to Roman
  • 原文地址:https://www.cnblogs.com/YuQiao0303/p/9230533.html
Copyright © 2011-2022 走看看