sqoop2的安装配置

1.下载

wget http://mirror.bit.edu.cn/apache/sqoop/1.99.7/sqoop-1.99.7-bin-hadoop200.tar.gz

2.解压

  tar -zxvf sqoop-1.99.7-bin-hadoop200.tar.gz -C /root/sqoop/

3.配置环境变量

 export SQOOP_HOME=/root/sqoop/sqoop-1.99.7-bin-hadoop200
 export CATALINA_BASE=$SQOOP_HOME/server
 export LOGDIR=$SQOOP_HOME/logs/

export PATH=$SQOOP_HOME/bin:$PATH

source ~/.base_profile

source /etc/profile

4.进入$SQOOP_HOME/conf目录，修改sqoop.properties，修改为hadoop的conf目录地址
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/root/hadoop/hadoop-2.7.4/etc/hadoop/

$SQOOP_HOME/conf添加catalina.properties文件，加入本机hadoop的相关jar路径，配置如下：注意修改hadoop路径

common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../lib/*.jar,/root/hadoop/hadoop-2.7.4/share/hadoo
p/common/*.jar,/root/hadoop/hadoop2.7.4/share/hadoop/common/lib/*.jar,/root/hadoop/hadoop2.7.4/share/hadoop/hdfs/*.jar,/root/hadoop/hadoop2.7.4/share/hadoop/hdfs/lib/*.jar,/root/hadoop/hadoop-2.7.4/share/hadoop/mapreduce/*.jar,/root/hadoop/hadoop-2.7.4/share/hadoop/mapreduce/lib/*.jar,/root/hadoop/hadoop-2.7.4/share/hadoop/tools/lib/*.jar,/root/hado
op/hadoop-2.7.4/share/hadoop/yarn/*.jar,/root/hadoop/hadoop-2.7.4/share/hadoop/yarn/lib/*.jar,/root/hadoop/hadoop-2.7.4/share/hadoop/httpfs/tomcat/lib/*.jar

cp mysql-connector-java-5.1.39.jar /root/sqoop/sqoop-1.99.7-bin-hadoop200/server/lib/

5.启动sqoop

[root@host sqoop-1.99.7-bin-hadoop200]# bin/sqoop.sh server start
Setting conf dir: bin/../conf
Sqoop home directory: /root/sqoop/sqoop-1.99.7-bin-hadoop200
Starting the Sqoop2 server...
0    [main] INFO  org.apache.sqoop.core.SqoopServer  - Initializing Sqoop server.
18   [main] INFO  org.apache.sqoop.core.PropertiesConfigurationProvider  - Starting config file poller thread
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/hadoop/hadoop-2.7.4/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/hive/apache-hive-2.1.1/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
Sqoop2 server started.

启动sqoop客户端

[root@host sqoop-1.99.7-bin-hadoop200]# bin/sqoop.sh client

Setting conf dir: bin/../conf

Sqoop home directory: /root/sqoop/sqoop-1.99.7-bin-hadoop200

Sqoop Shell: Type 'help' or 'h' for help.

sqoop:000>

sqoop:000> show connector
+------------------------+---------+------------------------------------------------------------+----------------------+
|          Name          | Version |                           Class                            | Supported Directions |
+------------------------+---------+------------------------------------------------------------+----------------------+
| generic-jdbc-connector | 1.99.7  | org.apache.sqoop.connector.jdbc.GenericJdbcConnector       | FROM/TO              |
| kite-connector         | 1.99.7  | org.apache.sqoop.connector.kite.KiteConnector              | FROM/TO              |
| oracle-jdbc-connector  | 1.99.7  | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO              |
| ftp-connector          | 1.99.7  | org.apache.sqoop.connector.ftp.FtpConnector                | TO                   |
| hdfs-connector         | 1.99.7  | org.apache.sqoop.connector.hdfs.HdfsConnector              | FROM/TO              |
| kafka-connector        | 1.99.7  | org.apache.sqoop.connector.kafka.KafkaConnector            | TO                   |
| sftp-connector         | 1.99.7  | org.apache.sqoop.connector.sftp.SftpConnector              | TO                   |
+------------------------+---------+------------------------------------------------------------+----------------------+

创建链接

sqoop:000>  create link -c  generic-jdbc-connector

Creating link for connector with name generic-jdbc-connector Please fill following values to create new link object Name: mysql2

Database connection

Driver class: com.mysql.jdbc.Driver

Connection String: jdbc:mysql://127.0.0.1:3306/test

Username: root

Password: Fetch Size: **

Connection Properties: There are currently 0 values in the map: entry#

SQL Dialect

Identifier enclose:   此处为空格

New link was successfully created with validation status WARNING and name mysql2

sqoop:000> show link
+-----------+------------------------+---------+
|   Name    |     Connector Name     | Enabled |
+-----------+------------------------+---------+
| mysqllink | generic-jdbc-connector | true    |
| mysql2    | generic-jdbc-connector | true    |
+-----------+------------------------+---------+