zoukankan      html  css  js  c++  java
  • sqoop安装部署(笔记)

    sqoop是一个把关系型数据库数据抽向hadoop的工具。同时,也支持将hive、pig等查询的结果导入关系型数据库中存储。
    由于,笔者部署的hadoop版本是2.2.0,所以sqoop的版本是:sqoop-1.99.3-bin-hadoop200
    1、下载sqoop  wget  http://mirrors.cnnic.cn/apache/sqoop/1.99.3/sqoop-1.99.3-bin-hadoop200.tar.gz
    2、解压sqoop安装文件  tar -zxvf sqoop-1.99.3-bin-hadoop200.tar.gz
    3、配置sqoop环境变量
      export SQOOP_HOME=/home/likehua/sqoop/sqoop
      export PATH = $SQOOP_HOME/bin:$PATH
      修改sqoop——server关于hadoop共享jar的引用
      vi server/conf/catalina.properties
      找到common.loader行,把/usr/lib/hadoop/lib/*.jar改成你的hadoop jar 包目录
    例如:/home/hadoop/hadoop/share/hadoop/yarn/lib/*.jar,
    /home/hadoop/hadoop/share/hadoop/yarn/*.jar,
    /home/hadoop/hadoop/share/hadoop/hdfs/*.jar,
    /home/hadoop/hadoop/share/hadoop/hdfs/lib/*.jar,
    /home/hadoop/hadoop/share/hadoop/mapreduce/*.jar,
    /home/hadoop/hadoop/share/hadoop/mapreduce/lib/*.jar,
    /home/hadoop/hadoop/share/hadoop/common/lib/*.jar,
    /home/hadoop/hadoop/share/hadoop/common/*.jar
      vi server/conf/sqoop.properties
    找到:mapreduce.configuration.directory行,修改值为你的hadoop配置文件目录
    如:/home/likehua/hadoop/hadoop/etc/hadoop
    并且替换@LOGDIR@ 和@BASEDIR@ :
    0,$ s/@LOGDIR@/logs/g
    0,$ s/@BASEDIR@/base/g

    然后找到你的数据库jdbc驱动复制到sqoop/lib目录下,如果不存在则创建.


     vi /etc/profile

    增加以下内容:

    export SQOOP_HOME=/home/hadoop/sqoop

    export PATH=$PATH:$SQOOP_HOME/bin

    export CATALINA_BASE=$SQOOP_HOME/server

    export LOGDIR=$SQOOP_HOME/logs/
     

    启动 [root@db12c sqoop]# ./bin/sqoop.sh server start
    Sqoop home directory: /home/likehua/sqoop/sqoop
    Setting SQOOP_HTTP_PORT:     12000
    Setting SQOOP_ADMIN_PORT:     12001
    Using   CATALINA_OPTS:
    Adding to CATALINA_OPTS:    -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
    Using CATALINA_BASE:   /home/likehua/sqoop/sqoop/server
    Using CATALINA_HOME:   /home/likehua/sqoop/sqoop/server
    Using CATALINA_TMPDIR: /home/likehua/sqoop/sqoop/server/temp
    Using JRE_HOME:        /usr/local/jdk1.7.0
    Using CLASSPATH:       /home/likehua/sqoop/sqoop/server/bin/bootstrap.jar
    (sqoop服务端是一个跑在tomcat上的服务程序)
    关闭 sqoop server :./bin/sqoop.sh server stop
    使用sqoop客户端:
    [root@db12c sqoop]# bin/sqoop.sh client
    Sqoop home directory: /home/likehua/sqoop/sqoop
    Sqoop Shell: Type 'help' or 'h' for help.

    sqoop:000> show version --all#显示版本:show version --all显示连接器:show connector --all创建连接:create connection --cid 1

    client version:
      Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b
      Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013
    server version:
      Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b
      Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013
    Protocol version:
      [1]
    sqoop:000>

  • 相关阅读:
    php实现简单的流程管理
    【百度地图API】如何制作多途经点的线路导航——驾车篇
    利用MFC实现浏览器的定制与扩展(JavaScript与C++交互)
    c++与js脚本交互,C++调用JS函数/JS调用C++函数
    VC/MFC中通过CWebPage类调用javascript函数(给js函数传参,并取得返回值)
    Android中半透明Activity效果另法
    mac java环境
    在Mac osx使用ADT Bundle踩过的坑
    Android自动检测版本及自动升级
    C++编译遇到参数错误(cannot convert parameter * from 'const char [**]' to 'LPCWSTR')
  • 原文地址:https://www.cnblogs.com/likehua/p/3825489.html
Copyright © 2011-2022 走看看