【Sqoop1】Sqoop1安装

本篇主要演示在Linux上基于Hadoop安装Sqoop1。

一 安装环境

• 操作系统：Oracle Linux Server release 6.5；
• Java版本：java-1.7.0-openjdk-1.7.0.45；
• Hadoop版本：hadoop-2.7.6；
• Sqoop2版本：1.4.7；

二 安装前准备

1 安装Hadoop

具体安装过程参考：Hadoop 2.7.6安装_伪分布式集群

2 下载Sqoop1以及MySQL-jdbc

Sqoop1下载地址：http://mirror.bit.edu.cn/apache/sqoop/1.4.7/

MySQL Connector下载地址：https://dev.mysql.com/downloads/connector/j/

三 安装配置Sqoop1

1 解压缩安装包

[root@strong ~]# tar zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz
[root@strong ~]# mv sqoop-1.4.7.bin__hadoop-2.6.0 sqoop-1.4.7
[root@strong ~]# chown -R hadoop:hadoop sqoop-1.4.7

2 修改配置文件

[root@strong ~]# cp sqoop-env-template.sh sqoop-env.sh
[hadoop@strong conf]$ vim sqoop-env.sh
#Set path to where bin/hadoop is available
export HADOOP_COMMON_HOME=/usr/local/hadoop

#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=/usr/local/hadoop

#set the path to where bin/hbase is available
export HBASE_HOME=/usr/local/hbase

#Set the path to where bin/hive is available
export HIVE_HOME=/usr/local/apache-hive-2.3.3-bin

export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64

3 拷贝Mysql驱动至Lib目录

[hadoop@strong ~]$ cp /usr/local/apache-hive-2.3.3-bin/lib/mysql-connector-java-5.1.46.jar /usr/local/sqoop-1.4.7/lib/

4 查看版本信息

[hadoop@strong ~]$ sqoop version
Warning: /usr/local/sqoop-1.4.7/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/local/sqoop-1.4.7/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /usr/local/sqoop-1.4.7/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
18/06/26 11:39:57 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
Sqoop 1.4.7
git commit id 2328971411f57f0cb683dfb79d19d4d19d185dd8
Compiled by maugli on Thu Dec 21 15:59:58 STD 2017

注：上面有几个警告，因为没有启用对应的组件，暂时忽略。

5 查看帮助信息

[hadoop@strong ~]$ sqoop help
Warning: /usr/local/sqoop-1.4.7/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/local/sqoop-1.4.7/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /usr/local/sqoop-1.4.7/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
18/06/26 11:42:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
usage: sqoop COMMAND [ARGS]

Available commands:
  codegen            Generate code to interact with database records
  create-hive-table  Import a table definition into Hive
  eval               Evaluate a SQL statement and display the results
  export             Export an HDFS directory to a database table
  help               List available commands
  import             Import a table from a database to HDFS
  import-all-tables  Import tables from a database to HDFS
  import-mainframe   Import datasets from a mainframe server to HDFS
  job                Work with saved jobs
  list-databases     List available databases on a server
  list-tables        List available tables in a database
  merge              Merge results of incremental imports
  metastore          Run a standalone Sqoop metastore
  version            Display version information

See 'sqoop help COMMAND' for information on a specific command.

6 验证对数据库的访问

[hadoop@strong ~]$ sqoop list-databases --connect jdbc:mysql://strong.hadoop.com:3306 --username root --password root
Warning: /usr/local/sqoop-1.4.7/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/local/sqoop-1.4.7/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /usr/local/sqoop-1.4.7/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
18/06/26 11:50:53 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
18/06/26 11:50:53 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/06/26 11:50:54 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
Tue Jun 26 11:50:54 CST 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
mysql
information_schema
performance_schema
sys
hive

至此，Sqoop1环境安装成功，下篇将演示Sqoop1的功能。