zoukankan      html  css  js  c++  java
  • Hadoop 单机搭建 hive单机搭建

    使用hive版本:apache-hive-2.3.6-bin.tar.gz 下载地址:http://www.apache.org/dyn/closer.cgi/hive

     在安装前必须要先安装好hadoop环境,可参考:Hadoop 单机搭建 hadoop单机搭建

    1、下载解压安装文件

     tar -zxvf apache-hive-2.3.6-bin.tar.gz -C /opt/modules/

    2、配置环境变量

    修改文件
    [hadoop@centos04 modules]$ sudo vi /etc/profile
    
    输入以下内容:
    
    export HIVE_HOME=/opt/modules/apache-hive-2.3.6-bin
    export PATH=$PATH:$HIVE_HOME/bin
    
    刷新使之生效
    [hadoop@centos04 modules]$ source /etc/profile
    
    显示版本号
    [hadoop@centos04 modules]$ hive -version
    which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/modules/jdk1.8.0_181/bin:/opt/modules/hadoop-2.8.2/bin:/opt/modules/hadoop-2.8.2/sbin:/home/hadoop/.local/bin:/home/hadoop/bin:/opt/modules/jdk1.8.0_181/bin:/opt/modules/hadoop-2.8.2/bin:/opt/modules/hadoop-2.8.2/sbin:/opt/modules/apache-hive-2.3.6-bin/bin)
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/opt/modules/apache-hive-2.3.6-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/opt/modules/hadoop-2.8.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
    
    Logging initialized using configuration in jar:file:/opt/modules/apache-hive-2.3.6-bin/lib/hive-common-2.3.6.jar!/hive-log4j2.properties Async: true
    Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
    hive> 

    3、关联hodoop

    Hive依赖hadoop,需要在Hive中指定hadoop的安装目录

    复制Hive安装目录的conf/hive-env.sh.template 文件为hive-env.sh ,然后添加以下内容,指定hadoop安装目录

    复制
    cp hive-env.sh.template hive-env.sh
    
    修改
    vi hive-env.sh
    
    添加
    export HADOOP_HOME=/opt/modules/hadoop-2.8.2/

    4、创建数据仓库

    hadoop fs -mkdir -p /tmp
    hadoop fs -mkdir -p /user/hive/warehouse
    hadoop fs -chmod g+w /tmp
    hadoop fs -chmod g+w /user/hive/warehouse
    
    
    /tmp hive任务在hdfs中的缓存目录
    /user/hive/warehouse hive数据仓库,存储hive创建的数据库
    
    hive默认向这2个目录写入数据
    如果希望任意用户对这2个目录拥有可写权限,g+w 改为 a+w

    5、初始化元数据信息

    从Hive2.1开始,需要运行schematool命令对Hive数据库的元数据进行初始化。默认Hive使用内嵌的Derby数据库来存储元数据信息

    [hadoop@centos04 bin]$ pwd
    /opt/modules/apache-hive-2.3.6-bin/bin
    [hadoop@centos04 bin]$ schematool -dbType derby -initSchema
    [hadoop@centos04 bin]$ schematool -dbType derby -initSchema
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/opt/modules/apache-hive-2.3.6-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/opt/modules/hadoop-2.8.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
    Metastore connection URL:        jdbc:derby:;databaseName=metastore_db;create=true
    Metastore Connection Driver :    org.apache.derby.jdbc.EmbeddedDriver
    Metastore connection User:       APP
    Starting metastore schema initialization to 2.3.0
    Initialization script hive-schema-2.3.0.derby.sql
    Initialization script completed
    schemaTool completed

    注意:Hive创建一个名为metastore_db的Derby数据库

    metastore_db数据库位置默认在初始化命令的执行目录

    [hadoop@centos04 bin]$ ll
    总用量 44
    -rwxr-xr-x. 1 hadoop hadoop  881 8月  13 2019 beeline
    -rw-rw-r--. 1 hadoop hadoop  709 3月  17 15:09 derby.log
    drwxrwxr-x. 3 hadoop hadoop 4096 3月  17 13:51 ext
    -rwxr-xr-x. 1 hadoop hadoop 9838 8月  13 2019 hive
    -rwxr-xr-x. 1 hadoop hadoop 1900 8月  13 2019 hive-config.sh
    -rwxr-xr-x. 1 hadoop hadoop  885 8月  13 2019 hiveserver2
    -rwxr-xr-x. 1 hadoop hadoop  880 8月  13 2019 hplsql
    drwxrwxr-x. 5 hadoop hadoop  133 3月  17 15:09 metastore_db
    -rwxr-xr-x. 1 hadoop hadoop  832 8月  13 2019 metatool
    -rwxr-xr-x. 1 hadoop hadoop  884 8月  13 2019 schematool
    [hadoop@centos04 bin]$ pwd
    /opt/modules/apache-hive-2.3.6-bin/bin
    [hadoop@centos04 bin]$ ll metastore_db/
    总用量 28
    -rw-rw-r--. 1 hadoop hadoop    4 3月  17 15:09 dbex.lck
    -rw-rw-r--. 1 hadoop hadoop   38 3月  17 15:09 db.lck
    drwxrwxr-x. 2 hadoop hadoop   97 3月  17 15:09 log
    -rw-rw-r--. 1 hadoop hadoop  608 3月  17 15:09 README_DO_NOT_TOUCH_FILES.txt
    drwxrwxr-x. 2 hadoop hadoop 8192 3月  17 15:09 seg0
    -rw-rw-r--. 1 hadoop hadoop  930 3月  17 15:09 service.properties
    drwxrwxr-x. 2 hadoop hadoop    6 3月  17 15:09 tmp
    [hadoop@centos04 bin]$ 

    6、启动Hive Cli

    [hadoop@centos04 metastore_db]$ pwd
    /opt/modules/apache-hive-2.3.6-bin/bin
    [hadoop@centos04 metastore_db]$ hive
    which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/modules/jdk1.8.0_181/bin:/opt/modules/hadoop-2.8.2/bin:/opt/modules/hadoop-2.8.2/sbin:/opt/modules/apache-hive-2.3.6-bin/bin:/home/hadoop/.local/bin:/home/hadoop/bin)
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/opt/modules/apache-hive-2.3.6-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/opt/modules/hadoop-2.8.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
    
    Logging initialized using configuration in jar:file:/opt/modules/apache-hive-2.3.6-bin/lib/hive-common-2.3.6.jar!/hive-log4j2.properties Async: true
    Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
    hive> 

    注意:上述命令必须在 metastore_db 所在的目录中执行(初始化命令的执行目录)

    hive> show databases;
    OK
    default
    Time taken: 11.034 seconds, Fetched: 1 row(s)
    hive> 

    7、验证

  • 相关阅读:
    ACM题集以及各种总结大全
    ACM题集以及各种总结大全
    线段树题集
    线段树题集
    POJ 1159 Palindrome【LCS+滚动数组】【水题】
    POJ 1159 Palindrome【LCS+滚动数组】【水题】
    开课博客
    第一周学习进度
    开学测试
    寒假总结
  • 原文地址:https://www.cnblogs.com/xuchen0117/p/12509855.html
Copyright © 2011-2022 走看看