zoukankan      html  css  js  c++  java
  • Hive2.2.1安装使用

    解压缩hive安装包
    tar zxvf apache-hive-2.1.1-bin.tar.gz

    安装mysql
    sudo yum install mysql-server

    安装 mysql connector
    sudo yum install mysql-connector-java
    该命令会在/usr/share/java/下产生mysql-connector-java.jar

    建立链接
    ln -s /usr/share/java/mysql-connector-java.jar /home/orco/resources/apache-hive-2.1.1-bin/lib/mysql-connector-java.jar
    该命令在hive安装目录的lib目录下建立软链接,指向/usr/share/java/mysql-connector-java.jar

    启动mysql
    sudo service mysqld start

    可通过以下命令验证mysql启动成功(显示mysql进程):
    ps aux | grep mysql


    修改mysql密码
    修改mysql密码(改为newpass,请根据需要自行替换newpass)
    mysql -u root
      mysql> use mysql;
      mysql> UPDATE user SET Password = PASSWORD('newpass') WHERE user = 'root';
      mysql> FLUSH PRIVILEGES;
    允许用户root通过任意机器访问mysql:
      mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'newpass' WITH GRANT OPTION;
      mysql> FLUSH PRIVILEGES;


    配置环境变量
    /home/orco/.bash_profile
    export HIVE_HOME=/home/orco/resources/apache-hive-2.1.1-bin
    export PATH=$HIVE_HOME/bin:$PATH
    运行source生效:
    source ~/.bash_profile

    配置hive,conf/hive-site.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <!--
           Licensed under the Apache License, Version 2.0 (the "License");
             you may not use this file except in compliance with the License.
               You may obtain a copy of the License at
    
        http://www.apache.org/licenses/LICENSE-2.0
    
      Unless required by applicable law or agreed to in writing, software
        distributed under the License is distributed on an "AS IS" BASIS,
          WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
            See the License for the specific language governing permissions and
              limitations under the License. See accompanying LICENSE file.
              -->
    
    <!-- Put site-specific property overrides in this file. -->
    <configuration>
        <property>
            <name>hive.metastore.uris</name>
            <value>thrift://node1:9083</value>
        </property>
        <property>
            <name>hive.server2.thrift.port</name>
            <value>10000</value>
        </property>
        <property>
            <name>javax.jdo.option.ConnectionURL</name>
            <value>jdbc:mysql://node1/metastore?createDatabaseIfNotExist=true</value>
        </property>
        <property>
            <name>javax.jdo.option.ConnectionDriverName</name>
            <value>com.mysql.jdbc.Driver</value>
        </property>
        <property>
            <name>javax.jdo.option.ConnectionUserName</name>
            <value>root</value>
        </property>
        <property>
            <name>javax.jdo.option.ConnectionPassword</name>
            <value>root</value>
        </property>
        <property>
            <name>hive.metastore.schema.verification</name>
            <value>false</value>
        </property>
        <property>
            <name>hive.metastore.warehouse.dir</name>
            <value>/warehouse</value>
        </property>
        <property>
            <name>fs.defaultFS</name>
            <value>hdfs://node1:9000</value>
        </property>
        <property>
            <name>datanucleus.autoCreateSchema</name>
            <value>true</value>
        </property>
        <property>
            <name>datanucleus.autoStartMechanism</name> 
            <value>SchemaTable</value>
        </property>
        <property>
            <name>datanucleus.schema.autoCreateTables</name>
            <value>true</value>
        </property>
    
        <property>
            <name>beeline.hs2.connection.user</name>
            <value>root</value>
            </property>
         <property>
            <name>beeline.hs2.connection.password</name>
            <value>root</value>
         </property>
    </configuration>

    注意:
    hive.metastore.uris中的“node1“含义为metastore所在的机器(启动metastore的方法见下面)
    javax.jdo.option.ConnectionURL中的“node1”为mysql安装机器的hostname
    javax.jdo.option.ConnectionUserName和javax.jdo.option.ConnectionPassword分别为mysql的访问用户和密码
    fs.defaultFS为HDFS的namenode启动的机器地址
    beeline.hs2.connection.user和beeline.hs2.connection.password是beeline方式访问的用户名和密码,可任意指定,但在beeline访问时要写入你指定的这个(具体参考最后面)

    启动metastore,可以配在任意节点上
    nohup hive --service metastore >> /home/orco/resources/apache-hive-2.1.1-bin/metastore.log 2>&1 &

    启动hive server,用来将sql翻译成mr,运行在集群上
    nohup hive --service hiveserver2 >> /home/orco/resources/apache-hive-2.1.1-bin/hiveserver.log 2>&1 &

    最后,通过输入hive命令,进入hive吧

    或者通过beeline进入,方式:

    !connect jdbc:hive2://node1:10000/default  root root

    最后的最后,这些东西在一个节点上(比如自己机器)部署就可以了,metastore和hive server这两个服务必须启动,如果要防止单点故障,则在另一台机器上部一下即可,不需要要集群化,Hive是单机的,只不过他借助hadoop或者hdfs实现分布式

  • 相关阅读:
    BZOJ3510 首都
    洛谷U19464 山村游历(Wander)(LCT)
    [BZOJ4530][Bjoi2014]大融合(LCT)
    POJ 3522 最小差值生成树(LCT)
    AHOI2005航线规划 bzoj1969(LCT缩点)
    P3950 部落冲突
    BZOJ2243 [SDOI2011]染色(LCT)
    P1501 [国家集训队]Tree II
    性能优化之优化加载资源
    Hexo next主题安装algolia
  • 原文地址:https://www.cnblogs.com/sorco/p/7011305.html
Copyright © 2011-2022 走看看