Hadoop之Hive 安装_（hadoop 集群）

Hive mysql的metastore安装准备（***掌握***）

在nameNode1机子上实践：

把hive-0.12.0.tar.gz解压到/itcast/

# tar -zxvf hive-0.12.0.tar.gz -C /itcast

修改/etc/profile文件，将hive加入环境变量

# vim /etc/profile

export JAVA_HOME=/usr/java/jdk1.7.0_79

export HADOOP_HOME=/itcast/hadoop-2.4.1

export HIVE_HOME=/itcast/hive-0.12.0

export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin

Source 一下，让配置生效

# source /etc/profile

安装完毕！

提示：metastore是hive元数据的集中存放地。metastore默认使用内嵌的derby数据库作为存储引擎。Derby引擎的缺点：一次只能打开一个会话，使用Mysql作为外置存储引擎，多用户同时访问。

Ø 安装mysql数据库（***掌握****）

（注意：没网络，可以配置好本地yum源）

# yum -y install mysql-server

启动mysql数据库

# service mysqld start

初始化mysql数据库（默认root的密码为空，按提示一步步完成）

# /usr/bin/mysql_secure_installation

将mysql加入开机启动

# chkconfig mysqld on

允许远程登录mysql

首先登录mysql数据，然后执行下面两行语句：

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'itcast' WITH GRANT OPTION;

　　(

　　grant all privileges on vtdc.* to joe@10.163.225.87 identified by ‘123′;
　　给来自10.163.225.87的用户joe分配可对数据库vtdc所有表进行所有操作的权限，并设定口令为123

　　)

flush privileges;

修改$HIVE_HOME/bin的hive-config.sh，增加以下三行

export JAVA_HOME=/usr/java/jdk1.7.0_79

export HIVE_HOME=/itcast/hive-0.12.0

export HADOOP_HOME=/itcast/hadoop-2.4.1

Ø 配置MySQL的metastore（***掌握***）

上传数据库驱动mysql-connector-java-5.1.28.jar到/itcast/hive-0.12.0/lib

cp hive-default.xml.template hive-site.xml 

修改$HIVE_HOME/conf/hive-site.xml

# vim /itcast/hive-0.12.0/conf/hive-site.xml

<configuration>

<property>

<name>javax.jdo.option.ConnectionURL</name>

<value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>

</property>

<property>

<name>javax.jdo.option.ConnectionDriverName</name>

<value>com.mysql.jdbc.Driver</value>

</property>

<property>

<name>javax.jdo.option.ConnectionUserName</name>

<value>root</value>

</property>

<property>

<name>javax.jdo.option.ConnectionPassword</name>

<value>itcast</value>

</property>

</configuration>

配置完毕！

Ø 测试配置是否正确（***掌握***）

启动hive

# hive

创建数据库

hive> create database test_db;

显示所有数据库

show databases;

使用数据库test_db

hive> use test_db;

创建学生表

hive> create table student(id int,name string);

查看是否在HDFS中存有下面文件夹

http://namenode1:50070/explorer.html#/user/hive/warehouse/test_db.db/student

 

 

 

-------------bug--------------------------------------

1.

Logging initialized using configuration in file:/usr/local/apache-hive-1.1.0-bin/conf/hive-log4j.properties
[ERROR] Terminal initialization failed; falling back to unsupported
java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
    at jline.TerminalFactory.create(TerminalFactory.java:101)
    at jline.TerminalFactory.get(TerminalFactory.java:158)
    at jline.console.ConsoleReader.<init>(ConsoleReader.java:229)
    at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)
    at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)
    at org.apache.hadoop.hive.cli.CliDriver.getConsoleReader(CliDriver.java:773)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:715)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
    at jline.console.ConsoleReader.<init>(ConsoleReader.java:230)
    at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)
    at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)
    at org.apache.hadoop.hive.cli.CliDriver.getConsoleReader(CliDriver.java:773)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:715)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

 

错误原因

Hive has upgraded to Jline2 but jline 0.94 exists in the Hadoop lib.
1

解决方案(Hive on Spark Getting Started)：

1.Delete jline from the Hadoop lib directory (it's only pulled in transitively from ZooKeeper).
2.export HADOOP_USER_CLASSPATH_FIRST=true