hadoop集群下安装hive

 hive的官网地址为： https://hive.apache.org/index.html

入门指南： https://cwiki.apache.org/confluence/display/Hive/GettingStarted

目录：  https://cwiki.apache.org/confluence/display/Hive/Home

体系结构

一 、解压

[root@master local]# pwd
/local
[root@master local]# tar -zxvf apache-hive-2.3.3-bin.tar.gz
[root@master local]# mkdir /opt/hive
[root@master local]# mv apache-hive-2.3.3-bin/* /opt/hive

二、配置环境变量

#####hive conf
export HIVE_HOME=/opt/hive
export PATH=$HIVE_HOME/bin:$PATH
"/etc/profile" 94L, 2312C written
[root@master bin]# source /etc/profile
[root@master bin]#

2.1 配置hvie文件

[root@master conf]# cp hive-default.xml.template hive-site.xml
[root@master conf]# cp hive-env.sh.template hive-env.sh
[root@master conf]# cp hive-log4j2.properties.template hive-log4j2.properties

 2.2 配置hive-site.xml

替换${system:java.io.tmpdir}为 /opt/hive/iotmp

替换 ${system:user.name}为root

javax.jdo.option.ConnectionURL

javax.jdo.option.ConnectionDriverName

javax.jdo.option.ConnectionUserName

 javax.jdo.option.ConnectionPassword

 

 hive.metastore.schema.verification

 

[root@master hive]# mkdir iotmp
[root@master hive]# cd iotmp/
[root@master iotmp]# pwd
/opt/hive/iotmp

2.3 配置hive-env.sh

# Set HADOOP_HOME to point to a specific hadoop install directory
 HADOOP_HOME=/opt/hadoop

# Hive Configuration Directory can be controlled by:
 export HIVE_CONF_DIR=/opt/hive/conf

# Folder containing extra libraries required for hive compilation/execution can be controlled by:
 export HIVE_AUX_JARS_PATH=/opt/hive/lib

三、 初始化

 [root@master bin]# schematool -initSchema -dbType mysql

缺少mysql的connector, mysql官网有下的。

上传mysql-connector-java-5.1.6-bin.jar到/opt/hive/lib里。

[root@master lib]# pwd
/opt/hive/lib
[root@master lib]# ls mysql*
mysql-connector-java-5.1.6-bin.jar mysql-metadata-storage-0.9.2.jar
[root@master lib]# cd ../bin
[root@master bin]# ll
total 40
-rwxr-xr-x. 1 root root 881 Feb 17 2017 beeline
drwxr-xr-x. 3 root root 4096 Apr 17 01:00 ext
-rwxr-xr-x. 1 root root 9838 Mar 28 04:23 hive
-rwxr-xr-x. 1 root root 1900 Dec 19 2015 hive-config.sh
-rwxr-xr-x. 1 root root 885 Dec 19 2015 hiveserver2
-rwxr-xr-x. 1 root root 880 Feb 17 2017 hplsql
-rwxr-xr-x. 1 root root 832 Dec 19 2015 metatool
-rwxr-xr-x. 1 root root 884 Dec 19 2015 schematool
[root@master bin]# schematool -initSchema -dbType mysql
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://192.168.234.129:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: root
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.mysql.sql
Initialization script completed
schemaTool completed
[root@master bin]#

四 、启动HIVE

启动顺序为zookeeper（全节点启动），hadoop（mater节点启动），hbase（master节点启动），hive

[root@master bin]# /opt/zookeeper/bin/zkServer.sh start

ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/conf/zoo.cfg
Starting zookeeper ... STARTED

[root@master zookeeper]# /opt/hadoop/sbin/start-all.sh 
Starting namenodes on [master]
Starting datanodes
Starting secondary namenodes [node1]
Starting resourcemanager
Starting nodemanagers

[root@master zookeeper]# /opt/hbase/bin/start-hbase.sh 
starting master, logging to /opt/hbase/logs/hbase-root-master-master.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
node1: starting regionserver, logging to /opt/hbase/bin/../logs/hbase-root-regionserver-node1.out
node2: starting regionserver, logging to /opt/hbase/bin/../logs/hbase-root-regionserver-node2.out

[root@master zookeeper]# /opt/hive/bin/hive
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in file:/opt/hive/conf/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive> show tables;
OK
Time taken: 4.057 seconds
hive> show functions;
OK
!
!=
$sum0
%
&
*

 查看版本信息、

[root@master bin]# schematool -dbType mysql -info
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://192.168.234.129:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: root
Hive distribution version: 2.3.0
Metastore schema version: 2.3.0
schemaTool completed

查看进程

master

[root@master bin]# jps
3952 ResourceManager
6448 QuorumPeerMain
6647 HMaster
3544 NameNode
7103 Jps

node1

[root@node1 data]# jps
3840 HRegionServer
4066 Jps
2867 NodeManager
2795 SecondaryNameNode
3755 QuorumPeerMain
2731 DataNode

node2

[root@node2 data]# jps
2784 NodeManager
3525 QuorumPeerMain
3782 Jps
3592 HRegionServer
2715 DataNode
[root@node2 data]#