1.hive介绍及安装配置

1.Hive介绍

1. 数据库OLTP 在线事务处理
2. 数据仓库OLAP 在线分析处理 延迟高
3. 类sql方式（HQL）
4. 使用sql方式，用来读写，管理位于分布式存储系统上的大型数据集的数据仓库技术
5. hive是基于Hadoop的一个数据仓库工具，可以将结构化的数据文件映射为一张数据库表，并提供完整的sql查询功能，可以将sql语句转换为MapReduce任务进行运行。其优点是学习成本低，可以通过类SQL语句快速实现简单的MapReduce统计，不必开发专门的MapReduce应用，十分适合数据仓库的统计分析。
6. hive使用的是hdfs做为存储
7. 使用maprecude做为计算模型
8. 用于海量数据计算分析

2.安装

1. 基于hadoop完全分布式环境（搭建过程略）

2. 下载hive包

   #wget https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-2.1.1/apache-hive-2.1.1-bin.tar.gz
   

3. 解压到指定路径和创建软链接

   #tar xf apache-hive-2.1.1-bin.tar.gz -C /soft/
   #ln -s /soft/apache-hive-2.1.1-bin /soft/hive
   

4. 配置环境变量

   #vim  /etc/profile
   添加一下内容
   HIVE_HOME=/soft/hive
   PATH=$PATH:$HIVE_HOME/bin
   #source /etc/profile
   

5. 测试

   #hive --version
   

3.配置mysql相关

1. 安装mysql（过程省略）

2. 创建相关数据库并授权

   mysql>create database dbhive;
   mysql>use hive;
   mysql>grant all on dbhive.* to "hive"@"%" identified by "123456"; 
   

5. 配置hive

1. 修改hive-site.xml(修改mysql相关配置)

   #cd /soft/hive/conf
   #vim hive-site.xml
   

javax.jdo.option.ConnectionPassword 123456 password to use against metastore database javax.jdo.option.ConnectionUserName hive Username to use against metastore database javax.jdo.option.ConnectionURL jdbc:mysql://192.168.10.103:3306/dbhive ```

2. 复制mysql驱动程序

   #cp /root/mysql-connector-java-5.1.38-bin.jar /soft/hive/lib/
   

3. 在mysql中初始化hive的schema

   #cd /soft/hive/bin/
   #./schematool -dbType mysql -initSchema
   which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/soft/jdk/bin:/soft/hadoop/bin:/soft/hadoop/sbin:/soft/hive/bin:/root/bin)
   SLF4J: Class path contains multiple SLF4J bindings.
   SLF4J: Found binding in [jar:file:/soft/apache-hive-2.1.1-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: Found binding in [jar:file:/soft/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
   SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
   Metastore connection URL:	 jdbc:mysql://192.168.10.103:3306/dbhive
   Metastore Connection Driver :	 com.mysql.jdbc.Driver
   Metastore connection User:	 hive
   Starting metastore schema initialization to 2.1.0
   Initialization script hive-schema-2.1.0.mysql.sql
   Initialization script completed
   schemaTool completed
   

4. 查看mysql

   mysql> use dbhive;
   Reading table information for completion of table and column names
   You can turn off this feature to get a quicker startup with -A
   
   Database changed
   mysql> show tables;
   +---------------------------+
   | Tables_in_dbhive          |
   +---------------------------+
   | AUX_TABLE                 |
   | BUCKETING_COLS            |
   | CDS                       |
   | COLUMNS_V2                |
   | COMPACTION_QUEUE          |
   | COMPLETED_COMPACTIONS     |
   | COMPLETED_TXN_COMPONENTS  |
   | DATABASE_PARAMS           |
   | DBS                       |
   | DB_PRIVS                  |
   | DELEGATION_TOKENS         |
   | FUNCS                     |
   | FUNC_RU                   |
   | GLOBAL_PRIVS              |
   | HIVE_LOCKS                |
   | IDXS                      |
   | INDEX_PARAMS              |
   | KEY_CONSTRAINTS           |
   | MASTER_KEYS               |
   | NEXT_COMPACTION_QUEUE_ID  |
   | NEXT_LOCK_ID              |
   | NEXT_TXN_ID               |
   | NOTIFICATION_LOG          |
   | NOTIFICATION_SEQUENCE     |
   | NUCLEUS_TABLES            |
   | PARTITIONS                |
   | PARTITION_EVENTS          |
   | PARTITION_KEYS            |
   | PARTITION_KEY_VALS        |
   | PARTITION_PARAMS          |
   | PART_COL_PRIVS            |
   | PART_COL_STATS            |
   | PART_PRIVS                |
   | ROLES                     |
   | ROLE_MAP                  |
   | SDS                       |
   | SD_PARAMS                 |
   | SEQUENCE_TABLE            |
   | SERDES                    |
   | SERDE_PARAMS              |
   | SKEWED_COL_NAMES          |
   | SKEWED_COL_VALUE_LOC_MAP  |
   | SKEWED_STRING_LIST        |
   | SKEWED_STRING_LIST_VALUES |
   | SKEWED_VALUES             |
   | SORT_COLS                 |
   | TABLE_PARAMS              |
   | TAB_COL_STATS             |
   | TBLS                      |
   | TBL_COL_PRIVS             |
   | TBL_PRIVS                 |
   | TXNS                      |
   | TXN_COMPONENTS            |
   | TYPES                     |
   | TYPE_FIELDS               |
   | VERSION                   |
   | WRITE_SET                 |
   +---------------------------+
   57 rows in set (0.00 sec)
   

5. 登录hive

   #hive