【Sqoop1】Sqoop1实战之导出Sqoop export

承接上篇，本篇主要演示Sqoop1的导出，涉及到的命令是Sqoop export。

1 查看export帮助信息

[hadoop@strong ~]$ sqoop help export

2 默认导出

1）在Mysql创建表

mysql> create table lang as select *from language where 1=2;
Query OK, 0 rows affected (1.56 sec)
Records: 0  Duplicates: 0  Warnings: 0

2）查看HDFS文件内容

[hadoop@strong ~]$ hdfs dfs -cat /user/sqoop1/language/part-r-00000
1,2006-02-15 05:02:19.0,English
2,2006-02-15 05:02:19.0,Italian
3,2006-02-15 05:02:19.0,Japanese
4,2006-02-15 05:02:19.0,Mandarin
5,2006-02-15 05:02:19.0,French
6,2006-02-15 05:02:19.0,German
7,2018-06-26 16:51:34.0,Chinese
8,2018-06-26 17:54:54.0,GD

3）执行导出命令

[hadoop@strong ~]$ sqoop export --connect jdbc:mysql://strong.hadoop.com:3306/sakila --username root --password root --table lang --export-dir /user/sqoop1/language

注：由于表没有主键，重复执行后会出现重复记录，避免此情况，可设主键。

4）查看导出结果

mysql> select *from lang order by 1;
+-------------+----------+---------------------+
| language_id | name     | last_update         |
+-------------+----------+---------------------+
|           1 | English  | 2006-02-15 05:02:19 |
|           2 | Italian  | 2006-02-15 05:02:19 |
|           3 | Japanese | 2006-02-15 05:02:19 |
|           4 | Mandarin | 2006-02-15 05:02:19 |
|           5 | French   | 2006-02-15 05:02:19 |
|           6 | German   | 2006-02-15 05:02:19 |
|           7 | Chinese  | 2018-06-26 16:51:34 |
|           8 | GD       | 2018-06-26 17:54:54 |
+-------------+----------+---------------------+
8 rows in set (0.00 sec)

3 以update_key方式导出

1）修改数据

mysql> update lang set name='GuangDongHua',last_update=current_timestamp where language_id=8;
Query OK, 1 row affected (0.45 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> select *from lang order by 1;
+-------------+--------------+---------------------+
| language_id | name         | last_update         |
+-------------+--------------+---------------------+
|           1 | English      | 2006-02-15 05:02:19 |
|           2 | Italian      | 2006-02-15 05:02:19 |
|           3 | Japanese     | 2006-02-15 05:02:19 |
|           4 | Mandarin     | 2006-02-15 05:02:19 |
|           5 | French       | 2006-02-15 05:02:19 |
|           6 | German       | 2006-02-15 05:02:19 |
|           7 | Chinese      | 2018-06-26 16:51:34 |
|           8 | GuangDongHua | 2018-06-27 12:05:22 |
+-------------+--------------+---------------------+
8 rows in set (0.01 sec)

2）执行导出

[hadoop@strong ~]$ sqoop export --connect jdbc:mysql://strong.hadoop.com:3306/sakila --username root --password root --table lang --export-dir /user/sqoop1/language --update-key language_id

3）查看数据

mysql> select *from lang order by 1;
+-------------+----------+---------------------+
| language_id | name     | last_update         |
+-------------+----------+---------------------+
|           1 | English  | 2006-02-15 05:02:19 |
|           2 | Italian  | 2006-02-15 05:02:19 |
|           3 | Japanese | 2006-02-15 05:02:19 |
|           4 | Mandarin | 2006-02-15 05:02:19 |
|           5 | French   | 2006-02-15 05:02:19 |
|           6 | German   | 2006-02-15 05:02:19 |
|           7 | Chinese  | 2018-06-26 16:51:34 |
|           8 | GD       | 2018-06-26 17:54:54 |
+-------------+----------+---------------------+
8 rows in set (0.03 sec)

注：以更新模式对数据进行了更新。

4 指定update-mode为updateonly方式导出

1）修改数据

mysql> delete from lang where language_id=8;
Query OK, 1 row affected (0.26 sec)
mysql> update lang set name='CN',last_update=current_timestamp where language_id=7;
Query OK, 1 row affected (0.12 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> select *from lang order by 1;
+-------------+----------+---------------------+
| language_id | name     | last_update         |
+-------------+----------+---------------------+
|           1 | English  | 2006-02-15 05:02:19 |
|           2 | Italian  | 2006-02-15 05:02:19 |
|           3 | Japanese | 2006-02-15 05:02:19 |
|           4 | Mandarin | 2006-02-15 05:02:19 |
|           5 | French   | 2006-02-15 05:02:19 |
|           6 | German   | 2006-02-15 05:02:19 |
|           7 | CN       | 2018-06-27 12:16:57 |
+-------------+----------+---------------------+
7 rows in set (0.00 sec)

2）进行导出

[hadoop@strong ~]$ sqoop export --connect jdbc:mysql://strong.hadoop.com:3306/sakila --username root --password root --table lang --export-dir /user/sqoop1/language --update-key language_id --update-mode updateonly

3）查看数据

mysql> select *from lang order by 1;
+-------------+----------+---------------------+
| language_id | name     | last_update         |
+-------------+----------+---------------------+
|           1 | English  | 2006-02-15 05:02:19 |
|           2 | Italian  | 2006-02-15 05:02:19 |
|           3 | Japanese | 2006-02-15 05:02:19 |
|           4 | Mandarin | 2006-02-15 05:02:19 |
|           5 | French   | 2006-02-15 05:02:19 |
|           6 | German   | 2006-02-15 05:02:19 |
|           7 | Chinese  | 2018-06-26 16:51:34 |
+-------------+----------+---------------------+
7 rows in set (0.06 sec)

注：发现以updateonly方式导出时，只更新，不会插入新行。

5 指定update-mode为allowinsert方式导出

1）修改数据

mysql> update lang set name='CN',last_update=current_timestamp where language_id=7;
Query OK, 1 row affected (0.14 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> select *from lang order by 1;
+-------------+----------+---------------------+
| language_id | name     | last_update         |
+-------------+----------+---------------------+
|           1 | English  | 2006-02-15 05:02:19 |
|           2 | Italian  | 2006-02-15 05:02:19 |
|           3 | Japanese | 2006-02-15 05:02:19 |
|           4 | Mandarin | 2006-02-15 05:02:19 |
|           5 | French   | 2006-02-15 05:02:19 |
|           6 | German   | 2006-02-15 05:02:19 |
|           7 | CN       | 2018-06-27 12:22:51 |
+-------------+----------+---------------------+
7 rows in set (0.01 sec)

2）进行导出

[hadoop@strong ~]$ sqoop export --connect jdbc:mysql://strong.hadoop.com:3306/sakila --username root --password root --table lang --export-dir /user/sqoop1/language --update-key language_id --update-mode allowinsert

3）查看数据

mysql> select *from lang order by 1;
+-------------+----------+---------------------+
| language_id | name     | last_update         |
+-------------+----------+---------------------+
|           1 | English  | 2006-02-15 05:02:19 |
|           2 | Italian  | 2006-02-15 05:02:19 |
|           3 | Japanese | 2006-02-15 05:02:19 |
|           4 | Mandarin | 2006-02-15 05:02:19 |
|           5 | French   | 2006-02-15 05:02:19 |
|           6 | German   | 2006-02-15 05:02:19 |
|           7 | Chinese  | 2018-06-26 16:51:34 |
|           8 | GD       | 2018-06-26 17:54:54 |
+-------------+----------+---------------------+
8 rows in set (0.00 sec)

注：以allowinsert方式作为更新模式时，可以进行更新和插入操作。