zoukankan      html  css  js  c++  java
  • 全网最详细Apache Kylin1.5安装(单节点)和测试案例 ---> 现在看来 kylin 需要 安装到Hadoop Master 节点上

    请参考原作者 ,谢 ,http://m.blog.itpub.net/30089851/viewspace-2121221/



    1.版本
    Hadoop2.7.2+HBase1.1.5+Hive2.0.0
    kylin-1.5.1Kylin1.5 (apache-kylin-1.5.1-HBase1.1.3-bin.tar.gz)

    2.Hadoop环境编译以支持Snappy解压缩库
    重新编译hadoop-2.7.2-src的native以支持Snappy解压压缩库

    3.环境准备
    Hadoop-2.7.2+zookeeper-3.4.6完全分布式环境搭建(HDFS、YARN HA)
    Hadoop2.7实战v1.0之HBase1.1.5 HA分布式搭建 
    配置Hadoop2.7.2和Hbase1.1.5支持Snappy解压压缩库

    Hadoop2.7实战v1.0之Hive-2.0.0+MySQL本地模式安装
     
    4.配置Hadoop2.7.2和Hbase1.1.5支持Snappy解压压缩库

    5. 主机规划
    IP                Host                    安装软件                 进程
    172.16.101.55 sht-sgmhadoopnn-01   apache-kylin-1.5.1-HBase1.1.3-bin.tar.gz RunJar
      
    6. 添加hive_dependency和KYLIN_HOME环境变量
    [root@sht-sgmhadoopnn-01 ~]# vi /etc/profile
    export JAVA_HOME=/usr/Java/jdk1.7.0_67-cloudera
    export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

    export HADOOP_HOME=/Hadoop/hadoop
    export HBASE_HOME=/hadoop/hbase
    export ZOOKEEPER_HOME=/hadoop/zookeeper

    export HIVE_HOME=/hadoop/hive
    export hive_dependency=/hadoop/hive/conf:/hadoop/hive/lib/*:/hadoop/hive/hcatalog/share/hcatalog/hive-hcatalog-core-2.0.0.jar                                    

    export KYLIN_HOME=/hadoop/kylin

    export PATH=.:$HADOOP_HOME/bin:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$KYLIN_HOME/bin:$PATH

    "/etc/profile" 99L, 2592C written
     [root@sht-sgmhadoopnn-01 ~]# source /etc/profile


    7.同步/etc/profile到集群其他节点,生效


     8.下载安装配置kylin
    # 在sht-sgmhadoopnn-01上执行以下命令
    [root@sht-sgmhadoopnn-01 ~]# cd /hadoop
    [root@sht-sgmhadoopnn-01 hadoop]# wget https://dist.apache.org/repos/dist/release/kylin/apache-kylin-1.5.2.1/apache-kylin-1.5.2.1-HBase1.x-bin.tar.gz
    [root@sht-sgmhadoopnn-01 hadoop]# tar -zxvf apache-kylin-1.5.1-HBase1.1.3-bin.tar.gz 
    [root@sht-sgmhadoopnn-01 hadoop]# ln -s /hadoop/apache-kylin-1.5.2.1-bin /hadoop/kylin

    [root@sht-sgmhadoopnn-01 hadoop]# vi /hadoop/kylin/bin/kylin.sh

    export KYLIN_HOME=/hadoop/kylin # 改成绝对路径
    export HBASE_CLASSPATH_PREFIX=${tomcat_root}/bin/bootstrap.jar:${tomcat_root}/bin/tomcat-juli.jar:${tomcat_root}/lib/*:$hive_dependency:$HBASE_CLASSPATH_PREFIX  #在路径中添加$hive_dependency


     9.修改kylin.properties
    [root@sht-sgmhadoopnn-01 conf]# vi $KYLIN_HOME/conf/kylin.properties
    ######修改
    # List of web servers in use, this enables one web server instance to sync up with other servers.
    kylin.rest.servers=sht-sgmhadoopnn-01:7070
    ######新增
    kylin.job.jar=$KYLIN_HOME/lib/kylin-job-1.5.2.1.jar
    kylin.coprocessor.local.jar=$KYLIN_HOME /lib/kylin-coprocessor-1.5.2.1.jar


    10.启动kylin
     [root@sht-sgmhadoopnn-01 kylin]#  $KYLIN_HOME/bin/kylin.sh  start
    .........
    .........
    KYLIN_DEBUG_SETTINGS is not set, will not enable remote debuging
    KYLIN_LD_LIBRARY_SETTINGS is not set, Usually it's okay unless you want to specify your own native path
    A new Kylin instance is started by root, stop it using "kylin.sh stop"
    Please visit http://:7070/kylin

    You can check the log at /hadoop/kylin/logs/kylin.log

    ###http://172.16.101.55:7070/kylin   账号: ADMIN    密码:KYLIN


       

     
    11.检查kylin是否启动成功
    [root@sht-sgmhadoopnn-01 kylin]# netstat -nlp |grep 7070
    tcp        0      0 0.0.0.0:7070                0.0.0.0:*                   LISTEN      30939/java         
     [root@sht-sgmhadoopnn-01 kylin]#


     12.导入官网测试案例
    [root@sht-sgmhadoopnn-01 kylin]# $KYLIN_HOME/bin/kylin.sh stop
    [root@sht-sgmhadoopnn-01 kylin]# $KYLIN_HOME/bin/sample.sh
    [root@sht-sgmhadoopnn-01 kylin]# $KYLIN_HOME/bin/kylin.sh  start
    1>.选中'kylin_sales_cube'示例立方体,点击'Actions'->'Build',选择一个截止日期,本试验中选择的是'2016-01-10'
    (具体小时,分,秒随便选,因为最终hive sql的语句类似     WHERE (KYLIN_SALES.PART_DT >= '2012-01-01' AND KYLIN_SALES.PART_DT < '2016-01-10');)

    2>.在'Monitor'标签中通过刷新页面检查进度条,直到100%

    3>.在'Insight'标签中执行下面的SQL查询:
    select part_dt, sum(price) as total_selled, count(distinct seller_id) as sellers from kylin_sales group by part_dt order by part_dt;
    ###耗时2.87s
    4>.在hive中执行同一个SQL查询,验证kylin的查询结果(会开启MapReduce Job计算)
    ###耗时65.205s
    查询执行和结果如图所示

    Build成功后,hive中建立了3+n个表,如图所示(3个官网案例hive表,n个build的hive表)
     

    Build成功后,hbase中建立了1+n个表,如图所示(1个元数据表,n个build的hbase表)

     
  • 相关阅读:
    集合的代数运算
    集合的代数运算
    poj1639 Picnic Planning,K度限制生成树
    C/C++学习站点资源
    Mustache 使用心得总结
    PostgreSQL服务端监听设置及client连接方法
    【线性规划与网络流24题】汽车加油行驶问题 分层图
    linux系统下信号具体解释2
    【数据结构】栈-数组的实现
    EJB究竟是什么,真的那么神奇吗??
  • 原文地址:https://www.cnblogs.com/TendToBigData/p/10501393.html
Copyright © 2011-2022 走看看