zoukankan      html  css  js  c++  java
  • Impala 安装笔记2一hive和mysql安装

    l   安装hivehive-metastore hive-server

    $ sudo yum install hive hive-metastore hive-server

    l   安装mysql-server

    $ sudo yum install mysql-server

    l   启动mysql守护进程

    $ sudo service mysqld start

    l   下载mysql java驱动并复制到hive的lib中

    $ cd ~
    $ wget 'http://cdn.mysql.com/Downloads/Connector-J/mysql-connector-java-5.1.25.tar.gz'
    $ tar xzf mysql-connector-java-5.1.25.tar.gz
    $ sudo cp mysql-connector-java-5.1.25/mysql-connector-java-5.1.25-bin.jar /usr/lib/hive/lib/

    l   Mysql安全配置启动

    $ sudo /usr/bin/mysql_secure_installation

    [...]
    Enter current password for root (enter for none):
    OK, successfully used password, moving on...
    [...]
    Set root password? [Y/n] y
    New password:hadoophive
    Re-enter new password:hadoophive
    Remove anonymous users? [Y/n] Y
    [...]
    Disallow root login remotely? [Y/n] N
    [...]
    Remove test database and access to it [Y/n] Y
    [...]
    Reload privilege tables now? [Y/n] Y
    All done!

     mysql作为hive的元数据存储数据库

    在mysql中建立相应的元数据数据库,执行相应的sql脚本,并且建立供hive连接mysql的用户,赋予其权限。

    $ mysql -u root –p chengyeliang
    mysql> CREATE DATABASE metastore;
    mysql> USE metastore;
    mysql> SOURCE /usr/lib/hive/scripts/metastore/upgrade/mysql/hive-schema-0.10.0.mysql.sql;
    
    mysql> CREATE USER 'hive'@'%' IDENTIFIED BY 'chengyeliang';
    mysql> CREATE USER 'hive'@'localhost' IDENTIFIED BY 'chengyeliang';
    mysql> REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'hive'@'%';
    mysql> REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'hive'@'localhost';
    mysql> GRANT SELECT,INSERT,UPDATE,DELETE,LOCK TABLES,EXECUTE ON metastore.* TO 'hive'@'%';
    mysql> GRANT SELECT,INSERT,UPDATE,DELETE,LOCK TABLES,EXECUTE ON metastore.* TO 'hive'@'localhost';
    mysql> FLUSH PRIVILEGES;
    mysql> quit;

    修改hive的配置文件

    $ sudo vim /etc/hive/conf/hive-site.xml

     1 <?xml version="1.0"?>
     2 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
     3 <configuration>
     4   <property>
     5     <name>javax.jdo.option.ConnectionURL</name>
     6     <value>jdbc:mysql://10.127.130.199:3306/metastore?useUnicode=true&amp;characterEn
     7 coding=UTF-8</value>
     8     <description>JDBC connect string for a JDBC metastore</description>
     9   </property>
    10 
    11   <property>
    12     <name>javax.jdo.option.ConnectionDriverName</name>
    13     <value>com.mysql.jdbc.Driver</value>
    14     <description>Driver class name for a JDBC metastore</description>
    15   </property>
    16 
    17   <property>
    18     <name>javax.jdo.option.ConnectionUserName</name>
    19     <value>hive</value>
    20     <description>username to use against metastore database</description>
    21   </property>
    22 <property>
    23     <name>javax.jdo.option.ConnectionPassword</name>
    24     <value>chengyeliang</value>
    25     <description>password to use against metastore database</description>
    26   </property>
    27 
    28   <property>
    29    <name>localhost.job.tracker</name>
    30    <value>localhost:8031</value>
    31   </property>
    32 
    33   <property>
    34    <name>mapreduce.framework.name</name>
    35    <value>yarn</value>
    36   </property>
    37   <property>
    38           <name>hive.metastore.uris</name>
    39           <value>thrift://localhost:9083</value>
    40           </property>
    41           <property>
    42           <name>hive.metastore.client.socket.timeout</name>
    43           <value>3600</value>
    44           <description>MetaStore Client socket timeout in seconds</description>
    45           </property>
    46 </configuration>

    启动hive-metastore进程

    $ sudo service hive-metastore start

    或者后台启动服务:nohup hive --service metastore&

    创建Hive所需的HDFS目录

    $ sudo -u hdfs hadoop fs -mkdir /user/hive
    $ sudo -u hdfs hadoop fs -chown hive /user/hive
    $ sudo -u hdfs hadoop fs -mkdir /tmp
    $ sudo -u hdfs hadoop fs -chmod 777 /tmp
    $ sudo -u hdfs hadoop fs -chmod o+t /tmp
    $ sudo -u hdfs hadoop fs -mkdir /data
    $ sudo -u hdfs hadoop fs -chown hdfs /data
    $ sudo -u hdfs hadoop fs -chmod 777 /data
    $ sudo -u hdfs hadoop fs -chmod o+t /data

    $ sudo chown -R hive:hive /var/lib/hive

    准备10万行记录的txt文件作为待查询的数据。

    sudo vim /tmp/kv1.txt

    建立一张新表

    hive> CREATE TABLE IF NOT EXISTS chengyeliang ( foo INT,bar STRING ) ROW FORMAT DELIMITED FIELDS TERMINATED BY " " LINES TERMINATED BY " ";

    Time taken: 0.415 seconds

    导入数据到table—chengyeliang

    hive> LOAD DATA LOCAL INPATH '/tmp/kv1.txt' OVERWRITE INTO TABLE chengyeliang;

    Time taken: 1.681 seconds

    上面的路径要换成自己文本文件在hdfs上的绝对路径

    如果上述操作ok,则安装成功。

  • 相关阅读:
    MVC模式-----struts2框架(2)
    MVC模式-----struts2框架
    html的<h>标签
    jsp脚本元素
    LeetCode "Paint House"
    LeetCode "Longest Substring with At Most Two Distinct Characters"
    LeetCode "Graph Valid Tree"
    LeetCode "Shortest Word Distance"
    LeetCode "Verify Preorder Sequence in Binary Search Tree"
    LeetCode "Binary Tree Upside Down"
  • 原文地址:https://www.cnblogs.com/chengyeliang/p/3548531.html
Copyright © 2011-2022 走看看