CentOS 7 安装Hive 3.2

CentOS 7 安装Hive 3.2

Hadoop版本3.2

1.下载并解压Hive(安装在/usr/local下)
tar zxvf apache-hive-3.1.1-bin.tar.gz
mv apache-hive-3.1.1-bin hive-3.1.1

2.环境变量配置
vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_211-amd64
export PATH=$PATH:JAVA_HOME/bin
export HADOOP_HOME=/home/hadoop-3.2.0
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
export HIVE_HOME=/usr/local/hive-3.1.1
export PATH=$PATH:$HIVE_HOME/bin
export CLASSPATH=$CLASSPATH:/home/hadoop-3.2.0/lib/*:.
export CLASSPATH=$CLASSPATH:/usr/local/hive-3.1.1/lib/*:.
export MYSQL_HOME=/usr/local/mysql-5.7.26
export PATH=$PATH:$MYSQL_HOME/bin
export CLASSPATH=$CLASSPATH:/usr/local/hive-3.1.1/lib/mysql-connector-java-8.0.13.jar
配置生效：source /etc/profile

3.配置Hive
cd $HIVE_HOME/conf
cp hive-env.sh.template hive-env.sh
在hive-env.sh中添加
export HADOOP_HOME=/home/hadoop-3.2.0

4.安装mysql
yum install mysql-server
下载不了，只能下载压缩包，安装
tar zvxf /usr/tools/mysql-5.7.26-linux-glibc2.12-x86_64.tar.gz
可以把mysql/bin/mysqld加入到系统服务，以后启动就可用service mysqld start启动了

配置mysql
新建mysql用户、组及目录
---新建一个msyql组
groupadd mysql
新建msyql用户禁止登录shell
useradd -r -s /sbin/nologin -g mysql mysql -d /usr/local/mysql
配置参数，注意生成的临时密码
bin/mysqld --initialize --user=mysql --basedir=/usr/local/mysql-5.7.26 --datadir=/usr/local/mysql-5.7.26/data --pid-file=/usr/local/mysql-5.7.26/data/mysql.pid --explicit_defaults_for_timestamp
2019-05-11T22:24:13.939648Z 1 [Note] A temporary password is generated for root@localhost: a_6+8Ln04+x4

[root@node1 mysql-5.7.26]# bin/mysql_ssl_rsa_setup --datadir=/usr/local/mysql-5.7.26/data
在support-files目录下创建my.cnf，并配置
[mysqld]
user=mysql
port=3306
server_id=1
socket=/tmp/mysql.sock
basedir=/usr/local/mysql-5.7.26
datadir=/usr/local/mysql-5.7.26/data
character-set-server=utf8
[client]
socket=/tmp/mysql.sock
[mysqld_safe]
log-error=/var/log/mysql/mysql.log
pid-file=/usr/local/mysql-5.7.26/data/mysql.pid

mkdir /var/log/mysql
chown -R mysql:mysql /var/log/mysql
chmod -R 755 /var/log/mysql

将support-file中的mysql.server复制到init.d下
cp mysql.server /etc/init.d/mysqld
chmod 755 /etc/init.d/mysqld

修改/etc/init.d/mysqld
basedir=/usr/local/mysql-5.7.26
datadir=/usr/local/mysql-5.7.26/data

启动mysql
service mysqld start

此处可能报错：Starting MySQL. ERROR! The server quit without updating PID file (/var/lib/mysql/node1.test.com.pid)
可能原因：
1.可能mysql用户对目录/var/lib/mysql没有权限，赋予权限
chown -R mysql:mysql /var/lib/mysql
2.可能进程中已存在mysqld
ps -ef|grep mysqld
如果存在，执行kill -9 pid
3.删除残余的binlog.index
将/var/lib/mysql下的binlog.index删除
4.selinux惹的祸，如果是centos系统，默认会开启selinux
解决方法：先临时改为警告模式:[root@www php]# setenforce 0
然后打开/etc/sysconfig/selinux，把SELINUX=enforcing改为SELINUX=disabled，重启电脑

登录，密码为初始化时的临时密码
mysql -u root -p
登入后修改root密码
mysql> set password=password('root');
设置root账户的host地址（才可远程访问）
mysql> grant all privileges on *.* to 'root'@'%' identified by 'root' with grant option;
mysql> flush privileges;

5.创建hive数据及用户
mysql> CREATE DATABASE hive;
mysql> USE hive;
mysql> CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive';
mysql> GRANT ALL ON hive.* TO 'hive'@'localhost' IDENTIFIED BY 'hive';
mysql> GRANT ALL ON hive.* TO 'hive'@'%' IDENTIFIED BY 'hive';
mysql> FLUSH PRIVILEGES;
mysql> quit;

5.下载mysql驱动
mysql-connector-java-8.0.13放到hive/lib下

6.修改hive/conf下的hive-env.sh
export HIVE_CONF_DIR=/usr/local/hive-3.1.1/conf

7.修改hive/conf下的hive-site.xml
在vim编辑界面输入如下命令完成替换：

:%s@${system:java.io.tmpdir}@/tmp/hive@g

cp hive-default.xml.template hive-site.xml

<configuration>
<property>
<name>hive.metastore.uris</name>
<value>thrift://node1:9083</value>
</property>

<property>
<name>hive.exec.scratchdir</name>
<value>/tmp/hive</value>
<description>HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/&lt;username&gt; is created, with ${hive.scratch.dir.permission}.</description>
</property>
<property>
<name>hive.repl.rootdir</name>
<value>/user/hive/repl/</value>
<description>HDFS root dir for all replication dumps.</description>
</property>
<property>
<name>hive.repl.cm.interval</name>
<value>3600s</value>
<description>
Expects a time value with unit (d/day, h/hour, m/min, s/sec, ms/msec, us/usec, ns/nsec), which is sec if not specified.
Inteval for cmroot cleanup thread.
</description>
</property>
<property>
<name>hive.exec.local.scratchdir</name>
<value>/tmp/hive</value>
<description>Local scratch space for Hive jobs</description>
</property>
<property>
<name>hive.downloaded.resources.dir</name>
<value>/tmp/hive/${hive.session.id}_resources</value>
<description>Temporary local directory for added resources in the remote file system.</description>
</property>
<property>
<name>hive.scratch.dir.permission</name>
<value>733</value>
<description>The permission for the user specific scratch directories that get created.</description>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
<description>location of default database for the warehouse</description>
</property>
<!--连接数据库地址，名称 -->
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://node1:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<!--连接数据库驱动 -->
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<!--连接数据库用户名称 -->
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<!--连接数据库用户密码 -->
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>
</property>
<!--客户端显示当前查询表的头信息 -->
<property>
<name>hive.cli.print.header</name>
<value>true</value>
</property>
<!--客户端显示当前数据库名称信息 -->
<property>
<name>hive.cli.print.current.db</name>
<value>true</value>
</property>
</configuration>

8.修改hive-log4j2.properties
cp hive-log4j2.properties.template hive-log4j2.properties
property.hive.log.dir = /var/log/hive

9.初始化数据库
schematool -dbType mysql -initSchema

10.将hive/lib下的jline复制到hadoop下
cp jline-2.12.jar /home/hadoop-3.2.0/share/hadoop/yarn/lib/

11.启动hive
hive
hive>

hive中输入show tables时报错：
java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
查看var/log/hive/hive.log:
Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift.transport.TSocket.open

应该是metastore没有启动，或者已经启动了先用kill -9停止进程，然后重新启动
ps -ef|grep hive
kill进程
hive --service metastore &(&表示后台运行)

这是多用户，remote一体，另外还有Remote分开的：Metastore server(thrift)和Metastore Clinet(Hive CLI)分开