zoukankan      html  css  js  c++  java
  • 【sqoop】安装配置测试sqoop1

    3.1.1 下载sqoop1:
    sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

    3.1.2 解压并查看目录:

    [hadoop@hadoop01 ~]$ tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz --解压
    [hadoop@hadoop01 ~]$ cd sqoop-1.4.7.bin__hadoop-2.6.0
    [hadoop@hadoop01 sqoop-1.4.7.bin__hadoop-2.6.0]$ ls -ll -查看目录
    total 2020
    drwxr-xr-x. 2 hadoop hadoop    4096 Dec 19  2017 bin
    -rw-rw-r--. 1 hadoop hadoop   55089 Dec 19  2017 build.xml
    -rw-rw-r--. 1 hadoop hadoop   47426 Dec 19  2017 CHANGELOG.txt
    -rw-rw-r--. 1 hadoop hadoop    9880 Dec 19  2017 COMPILING.txt
    drwxr-xr-x. 2 hadoop hadoop     150 Dec 19  2017 conf
    drwxr-xr-x. 5 hadoop hadoop     169 Dec 19  2017 docs
    drwxr-xr-x. 2 hadoop hadoop      96 Dec 19  2017 ivy
    -rw-rw-r--. 1 hadoop hadoop   11163 Dec 19  2017 ivy.xml
    drwxr-xr-x. 2 hadoop hadoop    4096 Dec 19  2017 lib
    -rw-rw-r--. 1 hadoop hadoop   15419 Dec 19  2017 LICENSE.txt
    -rw-rw-r--. 1 hadoop hadoop     505 Dec 19  2017 NOTICE.txt
    -rw-rw-r--. 1 hadoop hadoop   18772 Dec 19  2017 pom-old.xml
    -rw-rw-r--. 1 hadoop hadoop    1096 Dec 19  2017 README.txt
    -rw-rw-r--. 1 hadoop hadoop 1108073 Dec 19  2017 sqoop-1.4.7.jar
    -rw-rw-r--. 1 hadoop hadoop    6554 Dec 19  2017 sqoop-patch-review.py
    -rw-rw-r--. 1 hadoop hadoop  765184 Dec 19  2017 sqoop-test-1.4.7.jar
    drwxr-xr-x. 7 hadoop hadoop      73 Dec 19  2017 src
    drwxr-xr-x. 4 hadoop hadoop     114 Dec 19  2017 testdata


    3.2 配置sqoop——mysql连接器:
    下载mysql-connector-java-8.0.16.jar,并将其拷贝至sqoop安装目录下的lib文件夹里

    3.3 配置sqoop环境变量:

    [hadoop@hadoop01 sqoop-1.4.7.bin__hadoop-2.6.0]$ cd conf
    [hadoop@hadoop01 conf]$ ls -ll
    total 28
    -rw-rw-r--. 1 hadoop hadoop 3895 Dec 19  2017 oraoop-site-template.xml
    -rw-rw-r--. 1 hadoop hadoop 1404 Dec 19  2017 sqoop-env-template.cmd
    -rwxr-xr-x. 1 hadoop hadoop 1345 Dec 19  2017 sqoop-env-template.sh
    -rw-rw-r--. 1 hadoop hadoop 6044 Dec 19  2017 sqoop-site-template.xml
    -rw-rw-r--. 1 hadoop hadoop 6044 Dec 19  2017 sqoop-site.xml


    3.3.1 拷贝复制sqoop-env.sh样本,并添加hadoop、hbase、hive、zookeeper的安装目录(注:没有的就不添加)

    [hadoop@hadoop01 conf]$ cp sqoop-env-template.sh sqoop-env.sh
    [hadoop@hadoop01 conf]$ gedit sqoop-env.sh
    修改的内容:
    #Set path to where bin/hadoop is available
    export HADOOP_COMMON_HOME=/home/hadoop/hadoop-3.2.0
    
    #Set path to where hadoop-*-core.jar is available
    export HADOOP_MAPRED_HOME=/home/hadoop/hadoop-3.2.0
    
    #set the path to where bin/hbase is available
    export HBASE_HOME=/home/hadoop/hbase-2.2.1
    
    #Set the path to where bin/hive is available
    export HIVE_HOME=/home/hadoop/apache-hive-3.1.2-bin
    
    #Set the path for where zookeper config dir is
    export ZOOCFGDIR=/home/hadoop/apache-zookeeper-3.5.5


    3.3.2 配置linux环境变量

    [hadoop@hadoop01 conf]$ gedit ~/.bash_profile
    [hadoop@hadoop01 conf]$ source ~/.bash_profile
    新添加:
    #sqoop
    export SQOOP_HOME=/home/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0
    export PATH=$PATH:$SQOOP_HOME/bin


    3.4 验证sqoop是否安装成功

    [hadoop@hadoop01 sqoop-1.4.7.bin__hadoop-2.6.0]$ bin/sqoop help --执行该命令,看到如下信息就表示成功
    Warning: /home/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
    Please set $HCAT_HOME to the root of your HCatalog installation.
    Warning: /home/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
    Please set $ACCUMULO_HOME to the root of your Accumulo installation.
    Warning: /home/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
    Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
    Error: Could not find or load main class org.apache.hadoop.hbase.util.GetJavaProperty
    2019-09-29 23:38:28,571 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
    usage: sqoop COMMAND [ARGS]
    Available commands:
      codegen            Generate code to interact with database records
      create-hive-table  Import a table definition into Hive
      eval               Evaluate a SQL statement and display the results
      export             Export an HDFS directory to a database table
      help               List available commands
      import             Import a table from a database to HDFS
      import-all-tables  Import tables from a database to HDFS
      import-mainframe   Import datasets from a mainframe server to HDFS
      job                Work with saved jobs
      list-databases     List available databases on a server
      list-tables        List available tables in a database
      merge              Merge results of incremental imports
      metastore          Run a standalone Sqoop metastore
      version            Display version information

    See 'sqoop help COMMAND' for information on a specific command.
    [hadoop@hadoop01 sqoop-1.4.7.bin__hadoop-2.6.0]$


    3.5 测试sqoop与mysql的连接
    sqoop list-tables --username User --password 'User_123456' --connect jdbc:mysql://localhost:3306/hahive /**方式1**/
    sqoop list-tables --username User -P --connect jdbc:mysql://localhost:3306/hahive  /**方式2**/
    Enter password:

    如果能执行以上操作,可以看见结果,说明sqoop能连接mysql。

  • 相关阅读:
    centos7手动搭建redis集群
    Xshell突破四个窗口限制
    Redis官方集群规范
    Redis官方集群教程
    centos7 更新阿里YUM源
    gitlab配置ssh
    Java前端控制器模式~
    Java数据访问对象模式
    Java组合实体模式~
    Java业务代理模式~
  • 原文地址:https://www.cnblogs.com/CQ-LQJ/p/11617097.html
Copyright © 2011-2022 走看看