zoukankan      html  css  js  c++  java
  • Hadoop2.2.0环境下Sqoop1.99.3安装

    本文转载自http://blog.csdn.net/liuwenbo0920/article/details/40504045

    1.安装准备工作:

    已经装好的hadoop环境是hadoop 2.2.0
    下载的sqoop安装包(注意是hadoop200)

    下载sqoop  wget  http://mirrors.cnnic.cn/apache/sqoop/1.99.6/sqoop-1.99.3-bin-hadoop200.tar.gz

    2.解压文件到工作目录:

    将sqoop解压到目标目录,我的是 /opt/soft/sqoop-1.99.3-bin-hadoop200

    配置环境变量:

    [html] view plain copy
     
    1. export SQOOP_HOME=/opt/soft/sqoop-1.99.3-bin-hadoop200  
    2. export CATALINA_HOME=$SQOOP_HOME/server  
    3. export LOGDIR=$SQOOP_HOME/logs  
    4. export PATH=$SQOOP_HOME/bin:$PATH  

    保存退出即时生效:

    source /etc/profile

    3.修改sqoop配置:
     vi   /opt/soft/sqoop-1.99.3-bin-hadoop200/server/conf/sqoop.properties 

    在文件中找到org.apache.sqoop.submission.engine.mapreduce.configuration.directory,用Hadoop的配置文件目录赋值,最终效果如下:

    org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/opt/soft/hadoop-2.2.0/etc/hadoop

    4.修改配置文件:
    修改配置文件catalina.properties:
    vim $SQOOP_HOME/server/conf/catalina.properties

    在文件中找到common.loader,在其后添加hadoop相关jar包路径,最终效果如下

    [plain] view plain copy 
    1. common.loader=${catalina.base}/lib,  
    2. ${catalina.base}/lib/*.jar,  
    3. ${catalina.home}/lib,  
    4. ${catalina.home}/lib/*.jar,  
    5. ${catalina.home}/../lib/*.jar,  
    6. /opt/soft/hadoop-2.2.0/share/hadoop/common/*.jar,  
    7. /opt/soft/hadoop-2.2.0/share/hadoop/common/lib/*.jar,  
    8. /opt/soft/hadoop-2.2.0/share/hadoop/hdfs/*.jar,  
    9. /opt/soft/hadoop-2.2.0/share/hadoop/hdfs/lib/*.jar,  
    10. /opt/soft/hadoop-2.2.0/share/hadoop/mapreduce/*.jar,  
    11. /opt/soft/hadoop-2.2.0/share/hadoop/mapreduce/lib/*.jar,  
    12. /opt/soft/hadoop-2.2.0/share/hadoop/tools/*.jar,  
    13. /opt/soft/hadoop-2.2.0/share/hadoop/tools/lib/*.jar,  
    14. /opt/soft/hadoop-2.2.0/share/hadoop/yarn/*.jar,  
    15. /opt/soft/hadoop-2.2.0/share/hadoop/yarn/lib/*.jar,  
    16. /opt/soft/hadoop-2.2.0/share/hadoop/httpfs/tomcat/lib/*.jar  


    安装中遇到的问题: 

    我在启动show version --all时,出现GET http://localhost:12000/sqoop/version returned a response status of 404 Not Found

    后来查看日志,发现时log4j的原因。

    由于在catalina.properties配置文件的common.loader属性中配置引入了所有的hadoop相关jar包,而hadoop的lib目录下是有log4j包的,只是版本和sqoop自带的log4j不一致。
    我们需要将/opt/soft/sqoop-1.99.3-bin-hadoop200/server/webapps/sqoop/WEB-INF/lib目录下的log4j-1.2.16.jar移除即可。

    发现还是不能解决,日志仍然抱  java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory ,于是手动将hadoop的jar包全部copy出来,

    然后复制到/opt/soft/sqoop-1.99.3-bin-hadoop200/server/webapps/sqoop/WEB-INF/lib目录下,还原common.loader中修改的内容,然后启动就好了。

    5.下载mysql驱动包

    mysql-connector-java-5.1.16-bin.jar   第三步,jar包相关工作:
    我拷贝到了两个目录  $SQOOP_HOME/server/lib  下

    6.启动/停止sqoop200 

    ./bin/sqoop.sh server start
    ./bin/sqoop.sh server stop

    查看启动日志:

    hadoop@hadoopMaster:/usr/local/sqoop/server/logs$ vim catalina.out 

    7.进入客户端交互目录

    bin/sqoop.sh client
  • 相关阅读:
    Write an algorithm such that if an element in an MxN matrix is 0, its entire row and column is set to 0.
    旋转二维数组
    replace empty char with new string,unsafe method和native implementation的性能比较
    判断一字符串是否可以另一字符串重新排列而成
    移除重复字符的几个算法简单比较
    也来纠结一下字符串翻转
    判断重复字符存在:更有意义一点
    程序员常去网站汇总
    sublime
    针对程序集 'SqlServerTime' 的 ALTER ASSEMBLY 失败,因为程序集 'SqlServerTime' 未获授权(PERMISSION_SET = EXTERNAL_ACCESS)
  • 原文地址:https://www.cnblogs.com/abc8023/p/5367108.html
Copyright © 2011-2022 走看看