zoukankan      html  css  js  c++  java
  • Hadoop 权威指南学习2 (Sqoop)

    6. Sqoop

    Apache sqoop is an open source tool that allow users to extract data from structured data store into Hadoop or back.

    1) run

    self :        $SQOOP_HOME/bin/sqoop

    standard: sqoop (默认目录是/usr/bin/sqoop)

    2) common commands

        sqoop help           ##  the list of available tools

        sqoop help import #provided with a tool, will get its usage

        sqoop import         # run the tool

        sqoop-toolname     #alternatate way  to run the tool

    3) Sqoop Connector

    Sqoop has an entension framework that makes it possible to import data from and export to ,any external storage system that has bulk

    data transfer capabilities.

    Sqoop Connector is a modular component to use it to enable extracting work.

    4) An import example:

    % sqoop import --connect jdbc:mysql://localhost/hadoopDB --table myTest -m 1
     
    • Sqoop tool will run a MapReduce job that connects to the MySQL db and reads the table.
    • By default, it will call four map tasks. But we could specify just one map task as (-m 1).
    • By default, it will generate comma-delimited text files.

    5) generated code

    Besides importing to HDFS successfully, Sqoop also provides you with a generated Java source file in current local directory.

    • Could be used for some special needs.
    • Coegen tool could generate the source code without performing an import.
    %sqoop codegen  --connect jdbc:mysql://localhost/hadoopDB  --table myTest  --class-name myNeed
    

    6) import process

    A better importing process should use a splitting column to divide table querying aross multiple nodes.

    6)

    7)

  • 相关阅读:
    格式化dataGridview里数据
    XtraGrid gridview基本用法
    winForm中如何控制listView的滚动条高手请进
    WinForm 和 Windows Service 通信 消息队列
    抽象工厂模式
    C#之访问控制修饰符
    JavaScript之变量
    Android之传感器(二)持续更新
    备忘录模式
    JavaScript之构造函数初了解
  • 原文地址:https://www.cnblogs.com/skyEva/p/5471262.html
Copyright © 2011-2022 走看看